[Bug 182738] New: Digikam crash on/after image library scan

http://bugs.kde.org/show_bug.cgi?id=182738

           Summary: Digikam crash on/after image library scan
           Product: digikam
           Version: unspecified
          Platform: Mandriva RPMs
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: crash
          Priority: NOR
         Component: general
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]


Version:           0.10.0-1.rc1.3mdv2009.1 (using KDE 4.2.0)
OS:                Linux
Installed from:    Mandriva RPMs

When I add some image folder to library, Digikam always crash on start.

Konsole give this, in lest lines:
...
digikam(10236)/digikam (core) Digikam::ImageScanner::addImage: Adding newitem
"/mnt/DATA/Obrazy/PC Temp/100CANON/CRW_0390.CRW"
digikam(10236)/KEXIV2 KExiv2Iface::KExiv2::getImageDateTime: DateTime =>
Exif.Photo.DateTimeOriginal =>  QDateTime("niedz. lis 9 19:05:12 2008")
digikam(10236)/KEXIV2 KExiv2Iface::KExiv2::getImageDateTime: DateTime =>
Exif.Photo.DateTimeOriginal =>  QDateTime("niedz. lis 9 19:05:12 2008")
digikam(10236)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation =>
Exif.Image.Orientation =>  8
digikam(10236)/digikam (core) Digikam::DImg::load: "/mnt/DATA/Obrazy/PC
Temp/23-11-2005_0020.JPG"  : JPEG file identified
digikam: tiffcomposite.cpp:697: virtual uint32_t
Exiv2::Internal::TiffMnEntry::doCount() const: Warunek zapewnienia `tiffType()
== ttUndefined' niezostał spełniony.
KCrash: crashing... crashRecursionCounter = 2
KCrash: Application Name = digikam path = <unknown> pid = 10236
sock_file=/home/daniel/.kde4/socket-localhost/kdeinit4__0
digikam: Fatal IO error: client killed



KDE bug pop-up window give this:

Program: digiKam (digikam), sygnał SIGABRT
[Current thread is 1 (Thread 0xb807d8e0 (LWP 11118))]

Thread 2 (Thread 0xb6a2eb90 (LWP 11119)):
[KCrash Handler]
#6  0xffffe424 in __kernel_vsyscall ()
#7  0x419d5c00 in raise () from /lib/i686/libc.so.6
#8  0x419d7668 in abort () from /lib/i686/libc.so.6
#9  0x419ceb5e in __assert_fail () from /lib/i686/libc.so.6
#10 0x436ea6b1 in ?? () from /usr/lib/libexiv2.so.5
#11 0x436e9e8f in ?? () from /usr/lib/libexiv2.so.5
#12 0x436eabe4 in ?? () from /usr/lib/libexiv2.so.5
#13 0x436ee4c0 in ?? () from /usr/lib/libexiv2.so.5
#14 0x436e9f5f in ?? () from /usr/lib/libexiv2.so.5
#15 0x436ec11c in ?? () from /usr/lib/libexiv2.so.5
#16 0x436e9fa7 in ?? () from /usr/lib/libexiv2.so.5
#17 0x436ea000 in ?? () from /usr/lib/libexiv2.so.5
#18 0x436e9fa7 in ?? () from /usr/lib/libexiv2.so.5
#19 0x436ee660 in ?? () from /usr/lib/libexiv2.so.5
#20 0x436e9f5f in ?? () from /usr/lib/libexiv2.so.5
#21 0x436f2949 in ?? () from /usr/lib/libexiv2.so.5
#22 0x436f2cb7 in Exiv2::TiffParser::encode () from /usr/lib/libexiv2.so.5
#23 0x4368e44e in Exiv2::ExifParser::encode () from /usr/lib/libexiv2.so.5
#24 0x42173bfb in ?? () from /usr/lib/libkexiv2.so.7
#25 0x421726eb in KExiv2Iface::KExiv2::getExif () from /usr/lib/libkexiv2.so.7
#26 0x446408ad in Digikam::ImageScanner::loadFromDisk () from
/usr/lib/libdigikamdatabase.so.1
#27 0x44643c62 in Digikam::ImageScanner::newFile () from
/usr/lib/libdigikamdatabase.so.1
#28 0x44637227 in Digikam::CollectionScanner::scanNewFile () from
/usr/lib/libdigikamdatabase.so.1
#29 0x446397fb in Digikam::CollectionScanner::scanAlbum () from
/usr/lib/libdigikamdatabase.so.1
#30 0x446399e8 in Digikam::CollectionScanner::scanAlbum () from
/usr/lib/libdigikamdatabase.so.1
#31 0x44639db3 in Digikam::CollectionScanner::scanAlbumRoot () from
/usr/lib/libdigikamdatabase.so.1
#32 0x4463a5ca in Digikam::CollectionScanner::completeScan () from
/usr/lib/libdigikamdatabase.so.1
#33 0x0822be33 in ?? ()
#34 0x4fdb6f5f in ?? () from /usr/lib/libQtCore.so.4
#35 0x41b0c315 in start_thread () from /lib/i686/libpthread.so.0
#36 0x41a8825e in clone () from /lib/i686/libc.so.6

Thread 1 (Thread 0xb807d8e0 (LWP 11118)):
#0  0xffffe424 in __kernel_vsyscall ()
#1  0x41b0fc45 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib/i686/libpthread.so.0
#2  0x4fdb806b in QWaitCondition::wait () from /usr/lib/libQtCore.so.4
#3  0x4fdb7111 in QThread::wait () from /usr/lib/libQtCore.so.4
#4  0x0822afee in ?? ()
#5  0x0822b0de in ?? ()
#6  0x0822b189 in ?? ()
#7  0x419d8cd9 in exit () from /lib/i686/libc.so.6
#8  0x4280eef8 in ?? () from /usr/lib/libQtGui.so.4
#9  0x41516fb9 in KApplication::xioErrhandler () from /usr/lib/libkdeui.so.5
#10 0x41516ff4 in ?? () from /usr/lib/libkdeui.so.5
#11 0x41bc7f9a in _XIOError () from /usr/lib/libX11.so.6
#12 0x41bd0361 in ?? () from /usr/lib/libX11.so.6
#13 0x0889bf00 in ?? ()
#14 0x00000000 in ?? ()


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

http://bugs.kde.org/show_bug.cgi?id=182738


Gilles Caulier caulier gilles gmail com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email],
                   |                            |[hidden email]
          Component|general                     |Metadata
            Version|unspecified                 |0.10.0-rc1




--- Comment #1 from Gilles Caulier <caulier gilles gmail com>  2009-02-01 16:35:36 ---
Crash appears in Exiv2 library. Which version you use ?

Please give use a better backtrace. Run digikam in GDB as explained here :

http://www.digikam.org/drupal/contrib

Gilles Caulier


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #2 from Daniel <nowa grafika gmail com>  2009-02-01 17:34:02 ---
> Crash appears in Exiv2 library. Which version you use ?

In my system I have this one (rpm connected to exiv2):

libexiv2_4 ver. 0.17.1
libexiv2_5 ver. 0.18
libkexiv2_7 ver. 4.2.0


> Please give use a better backtrace.

For now, I can't give better backtrace info.
GPRS internet connection so debug rpms are to heavy for me now one.


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #3 from Gilles Caulier <caulier gilles gmail com>  2009-02-01 17:38:10 ---
You have both Exiv2 version installed at the same time. This is an origin to
crash (here i have reproduce this dysfunction recently between Exiv2 0.16 and
0.18)

You must remove Exiv2 0.17.1, let's only 0.18, and cleanup and recompile :

libkexiv2
kipi-plugins
digiKam

Gilles Caulier


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #4 from Daniel <nowa grafika gmail com>  2009-02-01 20:13:29 ---
> You must remove Exiv2 0.17.1, let's only 0.18, and cleanup and recompile :

libexiv2_4 removed and still same crash.

I install Digikam from binary rpm, so it should be accurately compiled with
libexiv2_5.

I pick out different, clean folder and then import my photos to it. Digikam
crash when I want to import .jpg, mayby some of them have broke EXIF and
Digikam just crash.

(I import some "test" jpg and Digikam work fine)


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #5 from Andreas Huggel <ahuggel gmx net>  2009-02-02 01:22:39 ---
Daniel,

Can you try to identify the/an image which causes this crash and attach it here
or send it to me at ahuggel at gmx dot net?


Gilles,

What is digiKam doing here:

#23 0x4368e44e in Exiv2::ExifParser::encode () from /usr/lib/libexiv2.so.5
#24 0x42173bfb in ?? () from /usr/lib/libkexiv2.so.7
#25 0x421726eb in KExiv2Iface::KExiv2::getExif () from /usr/lib/libkexiv2.so.7
#26 0x446408ad in Digikam::ImageScanner::loadFromDisk () from
/usr/lib/libdigikamdatabase.so.1

"loadFromDisk()" and "getExif()" sound like read-only operations, but
"Exiv2::ExifParser::encode()" is usually called during writing??


-ahu.


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

http://bugs.kde.org/show_bug.cgi?id=182738


Gilles Caulier caulier gilles gmail com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #6 from Gilles Caulier <caulier gilles gmail com>  2009-02-02 06:15:48 ---
Andreas,

ImageScanner::loadFromDisk() is here:

http://lxr.kde.org/source/extragear/graphics/digikam/libs/database/imagescanner.cpp#594

m_img, is a image conatiner (image dat + metadata)

http://lxr.kde.org/source/extragear/graphics/digikam/libs/database/imagescanner.h#156

http://lxr.kde.org/source/extragear/graphics/digikam/libs/dimg/dimg.h

But here, m_img is used to load a small part of image not whole content. We
collect information to fill database. For me, we never touch image file.

Gilles


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #7 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-02-02 11:24:40 ---
Judging from the backtrace I would assume the crash is from line 606 of
imagescanner.cpp.

The code for KExiv2::getExif is here:
http://lxr.kde.org/source/KDE/kdegraphics/libs/libkexiv2/libkexiv2/kexiv2exif.cpp#073


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #8 from Andreas Huggel <ahuggel gmx net>  2009-02-02 11:54:10 ---
Exiv2 bug 615: http://dev.exiv2.org/issues/show/615

I think this image has a makernote with the wrong tag type and Exiv2 reacts
overly sensitive to that - I'll have to change the assertion to something more
forgiving.

But I'd still like to see the image that causes this.

(This is the first real bug for 0.18 - I almost thought there are none left :)

-ahu.


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

http://bugs.kde.org/show_bug.cgi?id=182738


Marcel Wiesweg marcel wiesweg gmx de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #9 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-02-02 19:41:08 ---
*** Bug 182903 has been marked as a duplicate of this bug. ***


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[S. Burmeister]

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #10 from S. Burmeister <sven burmeister gmx net>  2009-02-02 19:52:18 ---
If bug 182903 is a duplicate, you should be able to trigger this bug with the
rw2 photos I supplied for testing. I do not remember the URL where they were
placed. Having a look at the output of digikam, it works fine indexing until it
hits the first RAW picture.

ate Photos/2006, Oktober, 21/dscf0012.jpg"  : JPEG file identified
digikam(22181)/digikam (core) Digikam::ImageScanner::addImage: Adding new item
"/media/Bueffel/Bilder/Private Photos/2006, Oktober, 21/dscf0012.jpg"
digikam(22181)/KEXIV2 KExiv2Iface::KExiv2::getImageDateTime: DateTime =>
Exif.Photo.DateTimeOriginal =>  QDateTime("Mi Nov 22 18:34:21 2006")
digikam(22181)/KEXIV2 KExiv2Iface::KExiv2::getDigitizationDateTime: DateTime
(Exif digitalized):  Mi Nov 22 18:34:21 2006
digikam(22181)/KEXIV2 KExiv2Iface::KExiv2::getImageOrientation: Orientation =>
Exif.Image.Orientation =>  1
digikam(22181)/digikam (core) Digikam::DImg::load:
"/media/Bueffel/Bilder/Private Photos/2008-12-26 Weihnachten 2008/p1000036.rw2"
 : RAW file identified
digikam: tiffimage.cpp:688: static void
Exiv2::Internal::TiffCreator::getPath(Exiv2::Internal::TiffPath&, uint32_t,
uint16_t, uint32_t): Zusicherung »ts !=0« nicht erfüllt.
digikam: Fatal IO error: client killed


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #11 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-02-02 21:50:42 ---
This is the image:

http://digikam3rdparty.free.fr/TEST_IMAGES/RAW/HORIZONTAL/p1000039.rw2


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #12 from Andreas Huggel <ahuggel gmx net>  2009-02-03 02:29:08 ---
Does the console output in comment #10 go with bug #182903?

That's a different assertion (tiffimage.cpp:688) than the one in the original
bug report here (tiffcomposite.cpp:697). From that console output it is clear
that this is a side effect of my changes from last night, not a duplicate of
this bug. Marcel is right in that it is also triggered by the call to
ExifParser::encode from ImageScanner::loadFromDisk but the root cause here is
different.

With this and Gilles' explanations in comment 6: Although the use case is valid
and these are clearly bugs in Exiv2, I think we need to question the design
now: Is it really desirable to call ExifParser::encode right after decoding the
metadata and internally carry metadata in a binary container m_img? Why not
keep it decoded in the Exiv2 metadata containers?

Since 0.18, ExifParser::encode is quite specific and powerful:

http://dev.exiv2.org/repositories/entry/exiv2/trunk/src/exif.cpp#L463

It tries hard to encode the metadata into a binary block suitable for a JPEG
image. E.g., it deletes certain tags which are not expected in JPEG images and
tries to fit everything into 64kB. This is an expensive operation and may
modify/lose metadata, especially if the source image is not a JPEG image. I
would avoid doing this unless it is really needed. Why not decode the metadata
when digiKam first loads the picture, keep it decoded in a metadata container
all the while, and encode it only when needed for a specific binary target
format.

I think such a design would minimize chances for errors, minimize the loss of
metadata and maximize performance, since decode and encode are the most complex
and expensive operations in Exiv2.

-ahu.


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #13 from Gilles Caulier <caulier gilles gmail com>  2009-02-03 08:08:13 ---
Andreas,

I'm agree that ExifParser::encode() is not the best solution here. But look
code from KExiv2::getExif()

#if (EXIV2_TEST_VERSION(0,17,91))
            Exiv2::Blob blob;
            Exiv2::ExifParser::encode(blob, Exiv2::bigEndian, exif);
            QByteArray ba((const char*)&blob[0], blob.size());
#else
            Exiv2::DataBuf c2 = exif.copy();
            QByteArray ba((const char*)c2.pData_, c2.size_);
#endif

Before Exiv2 0.18, ExifData::copy() been available to. Now api has changed and
you have recommended to use ExifParser::encode() instead... There is another
alternative ?

Gilles


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[S. Burmeister]

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #14 from S. Burmeister <sven burmeister gmx net>  2009-02-03 08:22:51 ---
(In reply to comment #12)
> Does the console output in comment #10 go with bug #182903?

Yes.


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #15 from Andreas Huggel <ahuggel gmx net>  2009-02-03 09:24:43 ---
For this discussion, copy and encode do the same thing and the issues described
apply to both. What I suggest is an image lifecycle in digiKam which looks like
this:

Load image -> decode metadata -> image processing -> encode metadata -> write
image. Image processing should not encode / decode the metadata anymore.

Of course this picture is a simplistic one, without taking all the complexities
of digiKam into account (I can do that, because I don't know them :)

-ahu.


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #16 from Andreas Huggel <ahuggel gmx net>  2009-02-03 16:22:14 ---
A fix for Exiv2 bug 615 (http://dev.exiv2.org/issues/show/615) is in the Exiv2
SVN, r1743 (http://dev.exiv2.org/repositories/revision/exiv2/1743)

I think this fixes the original bug reported here, although we've not seen the
offending image yet.

Daniel, can you please retest with the latest Exiv2 version from SVN and report
if it works now?

-ahu.


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #17 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-02-03 16:22:54 ---
Andreas, this is interesting info. This means that our current getExif
implementation is not lossless and not reversible.

- what you suggest is storing ExifData, IptcData, XmpData directly? This is
your intended workflow?
- these classes use std::vector internally, which means copying is always a
deep copy? (We would have to write a wrapper for Qt-style shallow copies)
- should ExifParser::encode be used for JPEGs only when you say it removes
certain tags?
- there are two ExifParser::encode methods. One has docs and says it does not
do unintrusive writing, suggesting the other method. The other has no docs and
requires a byte* buffer. Should this method be of any interest to us?


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #18 from Andreas Huggel <ahuggel gmx net>  2009-02-04 14:13:24 ---
Re: comment #17: See bug #183171

-ahu.


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

http://bugs.kde.org/show_bug.cgi?id=182738





--- Comment #19 from Gilles Caulier <caulier gilles gmail com>  2009-02-04 14:26:12 ---
Daniel,

can you please test again with the latest Exiv2 version from SVN, recompile and
install it. Do the same with libkexiv2 and report if crash disapear now?

Gilles Caulier


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel