[digikam] [Bug 353674] New: Operations requiring disk access are unreasonably slow

[Nikolaus]

https://bugs.kde.org/show_bug.cgi?id=353674

            Bug ID: 353674
           Summary: Operations requiring disk access are unreasonably slow
           Product: digikam
           Version: 4.4.0
          Platform: Debian stable
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: NOR
         Component: Import
          Assignee: [hidden email]
          Reporter: [hidden email]

When using digikam, all operations that require some disk access seem extremly
slow compared. For example, importing 360 files with 2.1 GB using the "Import
Images" dialog takes about 6 minutes. A simply "cp -a" using the same source
and destination directories, preceded by "echo 3 > /proc/sys/vm/drop_caches"
takes a mere 6.53 seconds.

Reproducible: Always

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=353674

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #1 from Gilles Caulier <[hidden email]> ---
A lots of fixes have been done since 4.4.0 release. Please update to last
4.13.0 and try again.

Also check well all option from Setup/Camera/Behaviors panel. Handling metadata
options can introduce time latency.

Finally, which kind of camera driver your use ? Gphoto or UMS ?

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Nikolaus]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #2 from [hidden email] ---
I'm currently compiling 4.13 but running into some trouble (see mail to
digikam-devel).  I don't think I'm using any camera driver, I'm importing the
files from a local hard disk.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Nikolaus]

https://bugs.kde.org/show_bug.cgi?id=353674

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|4.4.0                       |4.13.0

--- Comment #3 from [hidden email] ---
I've now upgraded to 4.13 and it's indeed faster (~3 minutes instead of 6
minutes). However, it's still extremly slow compared to using "cp" (~6
seconds).

All checkboxes in "Configure -> Camera -> Behavior" are unchecked.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #4 from [hidden email] ---
This file still valid using last digiKam 5.0.0 ?

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|Import                      |Import-PostProcessing

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #5 from [hidden email] ---
(In reply to caulier.gilles from comment #4)
> This file still valid using last digiKam 5.0.0 ?

I see this behaviour with Digikam 5.0.0 on Windows, when importing from an SD
card. Importing is extremely slow, and the GUI becomes unresponsive even though
the processor (Quad core Intel i5) is only at ~15 - 20%.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|4.13.0                      |5.0.0
          Component|Import-PostProcessing       |Import

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #6 from [hidden email] ---
What's about this problem using DK pre-release 5.4.0 bundles ?

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #7 from [hidden email] ---
I don't know about the 5.4 pre-release, but I tried the 5.3.0 appimage and it's
even worse. I don't have exact numbers, but just generating thumbnail takes
about half a second per image (and images are 2-3 MB).

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #8 from [hidden email] ---
(I'm a bit hesitant to download and run software from a Google Drive link on a
public bugtracker, sorry)

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|5.0.0                       |5.3.0

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #9 from Maik Qualmann <[hidden email]> ---
Git commit 043459000f01c0420a7bccb6bdd15696683403d7 by Maik Qualmann.
Committed on 30/01/2017 at 21:42.
Pushed by mqualmann into branch 'master'.

use separate camera thumbnail queue for quickly camera response
Related: bug 363937
FIXED-IN: 5.5.0

M  +2    -1    NEWS
M  +10   -9    utilities/importui/backend/cameracontroller.cpp

https://commits.kde.org/digikam/043459000f01c0420a7bccb6bdd15696683403d7

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #10 from [hidden email] ---
Nikolaus,

I upload myself the file to GDrive. The AppImage is done on same computer (VM)
than official one uploaded to KDE repository (which more complex to upload day
by day this kind of file due to restriction and right).

The file uploaded in GDrive is safe. This is the fast way for the team to
provide a suitable solution for end users to validate last fix.

I just updated the 64 bits AppImage bundle. Please give us a fresh feedback.
It's important...

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #11 from [hidden email] ---
I tried the most recent x64 appimage, and the entire GUI just hangs right after
the flash screen has disappeared. Last messages on the console are:

[...]
digikam.general: mimetype =  ""  ext =  "MP4"
digikam.general: Cannot create thumbnail for
"/home/nikratio/lib/Dropbox/Photos/2016/11 (november)/2016-11-13_16_23_16.mp4"
digikam.general: Thumbnail is null for
"/home/nikratio/lib/Dropbox/Photos/2016/11 (november)/2016-11-13_16_23_16.mp4"
digikam.general: Request to get thumbnail for
"/home/nikratio/lib/Dropbox/Photos/2016/11 (november)/2016-11-13_16_23_16.mp4"
digikam.general: Video duration for  "2016-11-13_16_23_16.mp4" is  24363
seconds
digikam.general: Trying to get thumbnail from  "2016-11-13_16_23_16.mp4"  at
position  4872

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #12 from [hidden email] ---
Run AppImage bundle from a console with "debug" as argument. This will run
digiKam into GDB. When digiKam hand up, press CTRL+C. You must take the GDB
prompt. enter "bt" to see where digiKam loop...

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #13 from [hidden email] ---
My last comment is only valid with 5.5.0 pre version only, available here :

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #14 from [hidden email] ---
I didn't have to press Ctrl-C, it threw me right into the debugger:

digikam.general: Video thumbnail extracted for
"/home/nikratio/lib/Dropbox/Photos/2016/11 (november)/2016-11-13_16_23_16.mp4"
::  QImage(QSize(256,
144),format=6,depth=32,devicePixelRatio=1,bytesPerLine=1024,byteCount=147456)
digikam.general: scan mode: ScanDeferredFiles
digikam.general: total scan value :  25704
digikam.dbengine: Database is locked. Waited 0
[Switching to Thread 0x7fffe3fff700 (LWP 10144)]
Catchpoint 1 (exception thrown), 0x00007fffee688dbd in __cxa_throw ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007fffee688dbd in __cxa_throw () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00000033b5f68fb5 in Exiv2::ImageFactory::open (path=...,
useCurl=<optimized out>)
    at /b/ext_exiv2/ext_exiv2-prefix/src/ext_exiv2/src/image.cpp:834
#2  0x00007ffff67d70e1 in Digikam::MetaEngine::load
(this=this@entry=0x7fffdc1035e0,
    filePath=...)
    at
/b/dktemp/digikam-development/garbagecollection/core/libs/dmetadata/metaengine.cpp:278
#3  0x00007ffff681f986 in Digikam::DMetadata::load
(this=this@entry=0x7fffdc1035e0,
    filePath=...)
    at
/b/dktemp/digikam-development/garbagecollection/core/libs/dmetadata/dmetadata.cpp:96
#4  0x00007fffeea549ba in Digikam::ImageScanner::loadFromDisk (
    this=this@entry=0x7fffe3ffe330)
    at
/b/dktemp/digikam-development/garbagecollection/core/libs/database/item/imagescanner.cpp:1568
#5  0x00007fffeea54b50 in Digikam::ImageScanner::newFile
(this=this@entry=0x7fffe3ffe330,
    albumId=174)
    at
/b/dktemp/digikam-development/garbagecollection/core/libs/database/item/imagescanner.cpp:289
#6  0x00007fffee996f0e in Digikam::CollectionScanner::scanNewFile
(this=this@entry=
    0x7fffe3ffecf0, info=..., albumId=174)
    at
/b/dktemp/digikam-development/garbagecollection/core/libs/database/collection/collectionscanner.cpp:1259
#7  0x00007fffee998b2b in Digikam::CollectionScanner::scanAlbum (
    this=this@entry=0x7fffe3ffecf0, location=..., album=...)
    at
/b/dktemp/digikam-development/garbagecollection/core/libs/database/collection/collectionscanner.cpp:1087
#8  0x00007fffee99877f in Digikam::CollectionScanner::scanAlbum (
    this=this@entry=0x7fffe3ffecf0, location=..., album=...)
    at
/b/dktemp/digikam-development/garbagecollection/core/libs/database/collection/collectionscanner.cpp:1117
[...]

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |UPSTREAM

--- Comment #15 from [hidden email] ---

It's a crash in Exiv2 when image is open for scanning at startup.

Report this problem to Exiv2 bugzilla as UPSTREAM. You will need to identify
the file on disk which has the problem (i suspect 2016-11-13_16_23_16.mp4).

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=353674

--- Comment #16 from Christoph Feck <[hidden email]> ---
Gilles, according to investigation in bug 375952 libexiv2 raises exceptions to
indicate unsupported file formats. I am not sure if it is possible to catch
exceptions within a Qt application, but nevertheless informing you, that this
is not a regular crash bug.

--
You are receiving this mail because:
You are the assignee for the bug.