digiKam › digikam-devel

[Bug 305359] New: Advanced rename of lots of images is slow

Classic

List

8 messages Options

Options

Sven Heithecker

[Bug 305359] New: Advanced rename of lots of images is slow

https://bugs.kde.org/show_bug.cgi?id=305359

Bug ID: 305359
Severity: wishlist
Version: 2.5.0
Priority: NOR
Assignee: [hidden email]
Summary: Advanced rename of lots of images is slow
Classification: Unclassified
OS: Linux
Reporter: [hidden email]
Hardware: openSUSE RPMs
Status: UNCONFIRMED
Component: Advanced Rename
Product: digikam

I often have to rename a lot of images. I noticed that the renaming function
gets extremly slow after the 1st ~20 images, e.g it needs several minutes to
rename ~340 images. While performing the rename operation, digiKam
continuously tries to update the album/folder view.
I am unsure if this matters, but DigiKam is configured to use "SQLite" internal
data base.
I admit that this is not really a bug, but I think that renaming should be
faster, maybe by disabling the album update until the renaming has finished.

Reproducible: Always

Steps to Reproduce:
1. Create several hundreds of images in one folder/album.
2. Rename all images using the "rename" function including the "###"
placeholder into the new file name.
Actual Results:
Renaming takes several minutes, while digiKam continuously tries to update the
album/folder view.

Expected Results:
The renaming should be faster, maybe by disabling the album update until the
renaming has finished.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

Kenny

[Bug 305359] Advanced rename of lots of images is slow

https://bugs.kde.org/show_bug.cgi?id=305359

Kenny <[hidden email]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[hidden email]

--- Comment #1 from Kenny <[hidden email]> ---
I can confirm, in particular, trying to renaming a large numbers of images over
a slow support (like SD) request a lot of time. Probably, like Sven said, the
problem is the real-time update of the album.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bug 305359] Advanced rename of lots of images is slow

In reply to this post by Sven Heithecker

https://bugs.kde.org/show_bug.cgi?id=305359

--- Comment #2 from Andi Clemens <[hidden email]> ---
I guess this is a general digiKam problem, not renaming in particular. As soon
as image data or metadata information is changed, digiKam updates a lot of
stuff in the background.

Gilles, Marcel,

is there a way to avoid updating immediately?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bug 305359] Advanced rename of lots of images is slow

In reply to this post by Sven Heithecker

https://bugs.kde.org/show_bug.cgi?id=305359

--- Comment #3 from Marcel Wiesweg <[hidden email]> ---
Yes, by introducing delays...which is not good as changes are visible later.

The root of many problems is that any write operation to Sqlite triggers a call
to fsync(), which is very slow and the main bottleneck in this setup. This is a
principal problem. Newer development like Sqlite WAL are steps in the right
direction, but WAL in my experience does not solve the problem.
You can of course switch off the "atomic" mode for you db, but I didn't dare
doing this so far.

I've recently changed the ImageScanner optimizing around that problem so that
there is only one write-to-disk per changed file.

Regarding the problem at hand, we should check console output during the rename
operation, that for each file there's a "recognized as identical..." message
and the file is not actually rescanned.
We should check if the problem is CPU or harddisk, if it's CPU, confirm it is
the main digikam process. If it is, valgrind callgrind will be interesting.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

Gregor Müllegger

[digikam] [Bug 305359] Advanced rename of lots of images is slow

In reply to this post by Sven Heithecker

https://bugs.kde.org/show_bug.cgi?id=305359

Gregor Müllegger <[hidden email]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[hidden email]

--- Comment #4 from Gregor Müllegger <[hidden email]> ---
I tried a SQLite and a MySQL database. It seems even worse with MySQL. When I
rename a bunch of images with a SQLite database, it takes about ~1 second per
file. With MySQL it seems to rename a few files (about three) rapidly but then
needs 4-5 seconds for the next few files and so on.

It definitly seems CPU bound, digikam hogs one of the cores but IO seems pretty
low. If using mysql, mysql will also use a full core for during renaming.
Haven't valigrind'ed anything yet... I hope to find out how to do this to
provide some usefull feedback.

I wonder if none of the core developers have this issue?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

Gregor Müllegger

[digikam] [Bug 305359] Advanced rename of lots of images is slow

In reply to this post by Sven Heithecker

https://bugs.kde.org/show_bug.cgi?id=305359

--- Comment #5 from Gregor Müllegger <[hidden email]> ---
I reseted my DB (i.e. I deleted it) and all the configuration files of digiKam.
This solved the issue for me, it was much faster after that and is even faster
since the last update I got via kubuntu backports.

I didn't lose too much data by this since I synced the tags with the in-file
meta data.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

Frank Steinmetzger

[digikam] [Bug 305359] Advanced rename of lots of images is slow

In reply to this post by Sven Heithecker

https://bugs.kde.org/show_bug.cgi?id=305359

Frank Steinmetzger <[hidden email]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[hidden email]

--- Comment #6 from Frank Steinmetzger <[hidden email]> ---
*** Bug 318363 has been marked as a duplicate of this bug. ***

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

Gilles Caulier-4

[digikam] [Bug 305359] Advanced rename of lots of images is slow

In reply to this post by Sven Heithecker

https://bugs.kde.org/show_bug.cgi?id=305359

Gilles Caulier <[hidden email]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[hidden email]
Resolution|--- |FIXED
Status|UNCONFIRMED |RESOLVED
Version Fixed In| |4.3.0

--- Comment #7 from Gilles Caulier <[hidden email]> ---
Following comment #5, problem was relevant of a broken database.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel