[Bug 303546] New: Re-read metadata from images is very slow

[Michael Holtermann-3]

https://bugs.kde.org/show_bug.cgi?id=303546

            Bug ID: 303546
          Severity: normal
           Version: 2.7.0
          Priority: NOR
          Assignee: [hidden email]
           Summary: Re-read metadata from images is very slow
    Classification: Unclassified
                OS: Linux
          Reporter: [hidden email]
          Hardware: Unlisted Binaries
            Status: UNCONFIRMED
         Component: Metadata
           Product: digikam

Hello,

because of a crash of my digikam database I have to re-import the metadata from
round about 22.000 images back into the MySQL database. I'm using the Ubuntu
PPAs built by Philip on an AMD 64x2 4200+ with 6GB of RAM.

Unfortunately, the process is very slow, shifting round about 1000 pictures per
hour, only.

top shows 84% CPU at mysqld and 35% at digikam, and 3% of MEM, both. iotop
shows nothing.

I guess the meta data reading is done by single kio_digikamtags? I see these
kind of process starting, living for ~6 seconds, and dying.

Any help or tuning hints would be appreciated....

Kind regards,
Michael

Reproducible: Always

Steps to Reproduce:
1. Select 1000 pictures
2. Select "re-read metadata from images"
3. waiting...

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=303546

--- Comment #1 from Marcel Wiesweg <[hidden email]> ---


> top shows 84% CPU at mysqld and 35% at digikam, and 3% of MEM, both. iotop
> shows nothing.

This would indicate the db side as bottleneck, but can be misleading. It will
be quite difficult to benchmark, there are two processes involved and at both
ends CPU or harddisk I/O can be limiting.

>
> I guess the meta data reading is done by single kio_digikamtags? I see these
> kind of process starting, living for ~6 seconds, and dying.

No, this is just updating the tags counter. Scanning is done in a thread of the
main process.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Michael Holtermann-3]

https://bugs.kde.org/show_bug.cgi?id=303546

--- Comment #2 from Michael Holtermann <[hidden email]> ---
Hello Marcel,

thank you for your reply. I'm running mysql-workbench in parallel, but I'm
affraid I'm far away from able to understand the values :-/

I'll attach the status and system variables, assuming that there is something
hidden inside...

Kind regards,
Michael

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Michael Holtermann-3]

https://bugs.kde.org/show_bug.cgi?id=303546

--- Comment #3 from Michael Holtermann <[hidden email]> ---
Created attachment 72538
  --> https://bugs.kde.org/attachment.cgi?id=72538&action=edit
MySQL Global Status and System Variables

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Michael Holtermann-3]

https://bugs.kde.org/show_bug.cgi?id=303546

Michael Holtermann <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #72538|application/octet-stream    |text/plain
          mime type|                            |

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=303546

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|Metadata                    |Database
                 CC|                            |[hidden email]
            Summary|Re-read metadata from       |MySql : Re-read metadata
                   |images is very slow         |from images is very slow

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=303546

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|MySql : Re-read metadata    |MySQL : Re-read metadata
                   |from images is very slow    |from images is very slow

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=303546

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|MySQL : Re-read metadata    |MYSQL : re-read metadata
                   |from images is very slow    |from images is very slow

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=303546

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|Database                    |Database-Mysql

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=303546

--- Comment #4 from [hidden email] ---
This file still valid using last digiKam 5.0.0 ?

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=303546

--- Comment #5 from Michael Holtermann <[hidden email]> ---
Sorry, don't know, I'm on Fedora 23 with Digikam 4.14, currently. I guess it's
a rare use case that did not gain much attraction, so I think it's fair to
close this ticket.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=303546

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
   Version Fixed In|                            |5.1.0
         Resolution|---                         |FIXED
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel