[Bug 210259] New: Scan crashed on multi-layer TIFFs

[Franciscus]

https://bugs.kde.org/show_bug.cgi?id=210259

           Summary: Scan crashed on multi-layer TIFFs
           Product: digikam
           Version: unspecified
          Platform: MS Windows
        OS/Version: MS Windows
            Status: UNCONFIRMED
          Severity: crash
          Priority: NOR
         Component: general
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]


Version:           1.0.0 beta 5 (using KDE 4.3.2)
Compiler:          Windows pre-compiled executables Windows pre-compiled
executables
OS:                MS Windows
Installed from:    MS Windows

TIFF is 236MB with three layers. Layers above base layer are in Multiply &
Screen mode; both have layer masks. Crash occurs during startup scan, or if
aborted, during later scan. Established that it was a particular image at fault
by renaming it, moving it, etc. DigiKam options are all set to default.

Metadata exactly the same as all the other single-layered TIFFs produced with a
Nikon LS-5000 (which were no trouble for digiKam).

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=210259


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|general                     |setup
            Version|unspecified                 |1.0.0-beta5




--- Comment #1 from Gilles Caulier <caulier gilles gmail com>  2009-10-12 06:38:00 ---
Can you give us a backtrace to see where is the problem ?

Can you try under linux to see if problem exist too ?

Can you provide a tiff file to test ?

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=210259


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #2 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-10-12 15:42:30 ---
Can we assume that this happens due to the size of this image (doesn't happen
with smaller tiffs?)
Backtrace would be the most interesting piece of information here.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Franciscus]

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #3 from  <francois tiscali co uk>  2009-10-12 23:57:58 ---
Thanks for the interest. Being a noob you'll have to be a bit patient ;)

First I tested the size hypothesis, which appears valid. Using only "normal"
mode for the layers, no mask (but some transparency), digiKam reads a 215MB
file, but chokes on 236MB. Currently uploading files to file sharing site, but
that's going to take a bit of time...

Looking into producing a backtrace under Windows XP.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Franciscus]

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #4 from  <francois tiscali co uk>  2009-10-13 01:38:18 ---
Here are the file:
http://dl.free.fr/qZ7HT26Rk

Can't say the backtrace research has been particularly fruitful :(

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=210259


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #5 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-10-17 14:02:08 ---
Andreas, we have a memory allocation problem with exiv2 here.
I will attach output from massif (heap profiler). When scanning the image in
question, libexiv2 will allocate a total of 1209MB of memory (in seven parts of
around 172MB each)

Note: from the link posted by Francois above, you can download a ZIP file
containing two sample TIFFs.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #6 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-10-17 14:03:39 ---
Created an attachment (id=37634)
 --> (http://bugs.kde.org/attachment.cgi?id=37634)
Relevant output from Massif

(read with automatic line breaking disabled)

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #7 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-10-17 14:04:45 ---
Is this related to bug #183171?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #8 from Andreas Huggel <ahuggel gmx net>  2009-10-17 14:40:38 ---
Marcel,

Exiv2 uses more memory than necessary when writing to TIFFs, that's a known
issue. See here: http://dev.exiv2.org/issues/show/617

However, a normal "intrusive" write operation should not allocate more than
about 2x the size of the image. Try simple commands like those in the issue to
verify. (I can't download such a large file right now, slow internet
connections seem to follow me around...)

Additional memory allocated is likely from suboptimal use of the library.

Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #9 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-10-17 16:39:48 ---
So with a 236MB image, 2*image size is already about 500MB ;-)

So this is indeed an incarnation of bug 183171 (when reading, to store the read
data, we do something supposed to be done only when writing).

Francois: Bug 183171 will be fixed soon, but it requires API changes in
libkexiv2, so the fix will be in KDE4.4 only.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #10 from Andreas Huggel <ahuggel gmx net>  2009-10-17 23:35:48 ---
> So with a 236MB image, 2*image size is already about 500MB ;-)

That's why there is http://dev.exiv2.org/issues/show/617 

In the unstable branch in SVN it is partly fixed, the image is only loaded
once, eventually it will not be necessary to load the image anymore at all. But
that also requires API changes.

Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #11 from Andreas Huggel <ahuggel gmx net>  2009-11-07 18:02:26 ---
The Exiv2 part is now complete and the results are quite drastic. See
http://dev.exiv2.org/issues/show/617

This is still in the unstable branch in the Exiv2 SVN for the moment, but will
go into the trunk and then become 0.19 (hopefully) soon. I'd be grateful if
someone took the trouble to compile digiKam/libkexiv2 against exiv2-unstable
and could report if there are issues with the API changes.

Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=210259


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|1.0.0-beta5                 |1.0.0




--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #12 from Gilles Caulier <caulier gilles gmail com>  2009-12-25 20:09:49 ---
digiKam 1.0.0 is out since few days...

http://www.digikam.org/drupal/node/491

Please try with this version coming with more than 400 bug-fixes.

Thanks in advance

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #13 from Gilles Caulier <caulier gilles gmail com>  2010-01-25 21:49:43 ---
digiKam 1.1. release will be done in few days. Please check if this entry still
valid.

Thanks in advance

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Franciscus]

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #14 from  <francois tiscali co uk>  2010-01-30 23:18:09 ---
Thanks a lot Gilles, Marel and Andreas. Tried 1.0.0 but it crashes every time
at startup so cannot test how well it scans large files. Hopefully I'll have
more luck with 1.1.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=210259


[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #15 from  <frith foottit gmail com>  2010-03-03 04:51:15 ---
Not 100% sure this is the same bug, but I did have a crash on scanning that was
fixed by removing the .TIFFs from my photos folder. The largest one was very
close to exactly 200MB.

Using linux version 1.2.0 on Ubuntu (downloaded from SVN and compiled on 2nd
March 2010), not sure what the library versions are - I just used apt-get
build-dep digikam.

Happy to provide any more info if needed, but I can't quite work out how to get
the debug info properly - I have installed various '-dbg' packages, but the
crash reporter still says that the crash information is not useful.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #16 from Andreas Huggel <ahuggel gmx net>  2010-03-03 05:21:44 ---
What version of exiv2 are you using? Run the following command to check
(assuming you have the exiv2 utility installed, not only the library)

$ exiv2 -V

If it's 0.19, can you post the output of this:

$ ls -la *.TIFF
$ time exiv2 *.TIFF >/dev/null

If it's < 0.19, pls upgrade to 0.19 and re-compile digikam + libkexiv2 against
that

Thanks,
Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #17 from  <frith foottit gmail com>  2010-03-03 10:52:57 ---
Okay, so now I'm stuck.

I ran 'exiv2 -V', and it said that I have version 0.18 (can't remember if there
was a minor number) installed.

So I checked the latest version of exiv2 out of svn (using the instructions on
the website), compiled that via cmake, make, sudo make install. Then I
re-compiled digikam, via cmake, make, sudo make install.

Now if I hit Help->Component information, it says:
(abridged)

digiKam version 1.2.0 (rev.: 1098216)
LibExiv2: 0.18.2

I can't understand why the new version of libexiv2 isn't overwriting the old
version. Any thoughts? I also notice that the command line program 'exiv2'
doesn't get installed - I have to manually run it from the compiled src
directory. Perhaps the install is just silently failing?

I suspect that fixing this would fix the problem, but the solution might end up
being that I have to wait for the next release of Ubuntu - I notice that
version .19 of exiv2 is in the list of packages for Lucid.

Then again, maybe its time to try a different distro...

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=210259





--- Comment #18 from Andreas Huggel <ahuggel gmx net>  2010-03-03 13:30:32 ---
> So I checked the latest version of exiv2 out of svn (using the instructions on
> the website), compiled that via cmake, make, sudo make install.

Good. The current trunk is better than 0.19. But instead of cmake, do

$ make config; ./configure; make; sudo make install

(cmake is highly experimental - see README-cmake)

> Then I
> re-compiled digikam, via cmake, make, sudo make install.
> Now if I hit Help->Component information, it says:
> (abridged)
>
> digiKam version 1.2.0 (rev.: 1098216)
> LibExiv2: 0.18.2
>
> I can't understand why the new version of libexiv2 isn't overwriting the old
> version. Any thoughts?

Did you re-compile libkexiv2? - note the 'k' If not, that's probably the issue.

> I also notice that the command line program 'exiv2'
> doesn't get installed - I have to manually run it from the compiled src
> directory. Perhaps the install is just silently failing?

Compile & install exiv2 as suggested above and it will get installed. But this
won't fix your other problem.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel