[Bug 223936] New: Moving photos between albums is slow

[Frédéric COIFFIER]

https://bugs.kde.org/show_bug.cgi?id=223936

           Summary: Moving photos between albums is slow
           Product: digikam
           Version: 1.0.0
          Platform: Compiled Sources
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: NOR
         Component: general
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]


Version:           1.0.0 (using Devel)
Compiler:          gcc (Gentoo 4.4.2 p1.0) 4.4.2

OS:                Linux
Installed from:    Compiled sources

It's a recent problem as I don't remember seeing it in pre-1.0 versions (or
maybe, it's linked to new KDE 4.4 libraries).

When I move 10-20 (RAW or JPEG) photos in an other album, it takes 5~10s before
the view is refreshed. In the past, this action was instantaneous.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=223936


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #1 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-01-23 18:32:34 ---
Which process is using CPU?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Frédéric COIFFIER]

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #2 from Frédéric COIFFIER <frederic coiffier free fr>  2010-01-24 12:44:11 ---
During the first try, I've seen a "virtuoso" process. So, I've disabled the
Nepomuk service.
But, now, during the 10s which lasts the moving operation, I don't see any
process which take lots of CPU. Even the Digikam process is around ~4% (whereas
it takes 11-16% to redraw thumbnails). It seems to wait an asynchronous
event...

In the console, I can see:

...
igikam(3036)/digikam (core) Digikam::AlbumManager::slotKioFileMoved: ...
digikam(3036)/digikam (core) Digikam::AlbumManager::handleKioNotification:
KDirNotify detected file change at ...
digikam(3036)/digikam (core) Digikam::AlbumManager::handleKioNotification:
KDirNotify detected file change at "/hdf/photos/2009-06 Vacances dans le Cantal
Corrèze/2009-06 Vacances dans le Cantal Corrèze - Rush "                        
digikam(3036)/digikam (core) Digikam::ImageScanner::addImage: Adding new item
...
digikam(3036)/digikam (core) Digikam::AlbumManager::slotKioFileMoved: ...      
digikam(3036)/digikam (core) Digikam::ImageScanner::copyFromSource: Recognized
... as copied from 28250
...

But there are lots of these error messages :

Error: Directory Unknown with 64366 entries considered invalid; not read.  
...
Error: Directory Unknown with 25212 entries considered invalid; not read.
...
Error: Directory Unknown with 43693 entries considered invalid; not read.
...
Error: Directory Unknown with 43693 entries considered invalid; not read.


I can send you debug messages of Digikam by mail if you want.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #3 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-01-28 16:11:44 ---
I have identified a problem here which led to a scan of the complete collection
after a copy or move operation, but only for the very first operation after
program startup.
Seems you are compiling SVN; can you check if this fixes your problem?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #4 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-01-28 16:13:19 ---
SVN commit 1081515 by mwiesweg:

Ensure that the initial setup of modification dates of db file directory is
done after the dir watch black list is setup. Otherwise, a complete collection
scan would be triggered for each first use.
Also fix retrieving the directory from the dir watch path, which can be either
a
file or a directory path.

CCBUG: 223936

 M  +20 -12    albummanager.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1081515

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Frédéric COIFFIER]

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #5 from Frédéric COIFFIER <frederic coiffier free fr>  2010-01-28 17:30:23 ---
I've compiled the last SVN version but it doesn't change anything.
Around ~10% of the CPU is shared between Digikam, kio_digikamalbum and
kio_digikamdate (and ~80% of CPU idle).

I also tried a large amount of RAW photos : If I tried to move 228 photos, only
56 are really moved and the other 172 photos stay in their album.

If you need a log file, traces..., explain me (quickly) how to generate them
and I'll send them.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Johannes Wienke-3]

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #6 from Johannes Wienke <languitar semipol de>  2010-01-28 17:40:16 ---
First of all start kdebugdialog and enable full logging for all digikam area
codes.

Afterwards start digikam from a console, do the file operations and close
digikam. Then paste the console log here.

Do you have the same problems when you move the files with dolphin?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Frédéric COIFFIER]

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #7 from Frédéric COIFFIER <frederic coiffier free fr>  2010-01-28 18:03:59 ---
With dolphin, moving 341 RAW files between directories is instantaneous (less
than 1s).

For the trace, I'll do it tonight and I'll send you by mail (because it'll be
big).

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #8 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-01-28 18:17:28 ---
Please also enable the code 50004 and provide messages added to your
~/.xsession-errors file.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #9 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-01-30 17:06:40 ---
A few more questions.

On the console output, there are two main sections: One contains a lot of
handleKioNotification statements. The second section mostly contains
Digikam::ImageScanner output.
Now, during the 10s while you copy files, when you watch the console "live",
which lines are output during the majority of time? Before section 1, in
Section 1, Section 2, or after section 2?

Did you disable Nepomuk in System Settings (Advanced tab)? To rule out it's a
Nepomuk issue.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #10 from Andi Clemens <andi clemens gmx net>  2010-01-30 17:12:37 ---
Do you use EXT4? If so, try turning off barriers in /etc/fstab, for example:
LABEL=media /mnt/data ext4 defaults,noatime,barrier=0  0 2

This helped on my computer, EXT4 + Barriers + Sqlite3 is a slow bastard :D

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Frédéric COIFFIER]

https://bugs.kde.org/show_bug.cgi?id=223936





--- Comment #11 from Frédéric COIFFIER <frederic coiffier free fr>  2010-01-30 19:00:57 ---
Andi, you're right: this bug isn't a regression. It's just that I have change
my filesystem to ext4 some weeks ago.
I've made the try with :
mount -o remount,barrier=0 /my_partition
and Digikam is as fast as previously.

Moreover, I succeed to move ~300 photos without problems (with default ext4
options, Digikam didn't move all selected photos).

For me, the problem is really linked to Sqlite3 + ext4. I knew this problem of
ext4 thanks to Phoronix's articles:
http://feedproxy.google.com/~r/Phoronix/~3/cVGeygVMUC0/vr.php

Marcel, you can close this bug report if you want.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Johannes Wienke-3]

https://bugs.kde.org/show_bug.cgi?id=223936


Johannes Wienke <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |UPSTREAM




--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=223936

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|                            |1.2.0

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=223936

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|general                     |Database
                 CC|                            |[hidden email]
         Resolution|UPSTREAM                    |FIXED
   Version Fixed In|1.2.0                       |5.7.0

--- Comment #12 from [hidden email] ---
Here it crash in KIO while processing changes in database. Since 5.0.0 release,
we have replaced the KIO mechanism to play with database by a multi-threaded
interface. This problem will never be reproducible.

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=223936

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|Database                    |Database-Albums

--
You are receiving this mail because:
You are the assignee for the bug.