[Bug 216894] New: Slow quadratic runtime generating fingerprints or thumbnails in beta6 during a big import

[Scott A Crosby]

https://bugs.kde.org/show_bug.cgi?id=216894

           Summary: Slow quadratic runtime generating fingerprints or
                    thumbnails in beta6 during a big import
           Product: digikam
           Version: unspecified
          Platform: Debian testing
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: NOR
         Component: general
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]


Version:           1.0.0beta6 (using KDE 4.3.2)
OS:                Linux
Installed from:    Debian testing/unstable Packages

Slow quadratic runtime generating fingerprints or thumbnails in beta6 during a
big import


I am importing about 50k images by pointing digikam at an already-made
directory tree, and digikam works great! However, in order to find
duplicates digikam must generate fingerprints. When doing this, the
program gets progressively slower. I suspect that this is because the
progress dialog continously grows the list of images, never rolling
the first entries off of the list. As the dialog auto-scrolls to the
bottom on every update, it is virtually impossible to see any entries
EXCEPT the last few. This causes digikam to get progressively slower
and slower. After a few thousand images, it is >5x slower.

If I abort fingerprint generation and try to restart and 'scan' for
images that need fingerprints, digikam remains slow.  However, if I
quit digikam entirely and restart and then have it 'scan', then it
generates fingerprints at full speed.... for a while, until the
quadratic slowdown hits again.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=216894


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|general                     |Import
            Version|unspecified                 |1.0.0-beta6




--- Comment #1 from Gilles Caulier <caulier gilles gmail com>  2009-12-01 10:04:56 ---
So, for you, if i understand properly, slower import is due of list from
progress dialog witch take a while to show items. right ?

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Scott A Crosby]

https://bugs.kde.org/show_bug.cgi?id=216894





--- Comment #2 from Scott Crosby <scrosby cs rice edu>  2009-12-01 16:49:40 ---
Yes. It progressively slows down, and the list in the dialog gets progressively
longer. Memory usage also gets progressively bigger. I suspect that the dialog
box is the cause, but I do not know.


Note, I am not doing a digikam 'import'.

Replication instructions:

  1. Find a directory with a lot of images. (10k-50k)
  2. Make a new collection pointing to that directory [Settings -> Configure
DIgikam -> Local Collection -> 'Add Collection']
  3. In the menu, select 'rebuild fingerprints'
  4. Select 'scan'.

Observe how fast it runs, then wait about 5-10 minutes and it is much slower.
Quit and restart digikam and repeat steps 3&4, and it is again fast.

In addition, when I run digikam under oprofile, it appears that the dialog box
may be responsible for 90% of the CPU utilization.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=216894


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #3 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-12-17 21:59:35 ---
You are quite right, the list including pixmap in the progress dialog grows
linearly.
A possible solution would be to remove the scroll bar and remove entries from
the beginning of the list.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Johannes Wienke-3]

https://bugs.kde.org/show_bug.cgi?id=216894


Johannes Wienke <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #4 from Johannes Wienke <languitar semipol de>  2009-12-17 22:05:26 ---
If this is such a performance killer I would really vote for this. Showing only
the last 5 generated fingerprints should really be enough. Who really wants to
scroll through the whole list?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=216894





--- Comment #5 from Gilles Caulier <caulier gilles gmail com>  2009-12-17 22:13:37 ---
Agree...

I have make this dialog in case to identify items if something is wrong...

Why not to do something like scan dialog at kphotoalbum startup ?

Gilles

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Johannes Wienke-3]

https://bugs.kde.org/show_bug.cgi?id=216894





--- Comment #6 from Johannes Wienke <languitar semipol de>  2009-12-17 22:39:14 ---
What does it look like? Never used kphotoalbum ;)

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=216894





--- Comment #7 from Gilles Caulier <caulier gilles gmail com>  2009-12-17 23:00:38 ---
Dialog is similar than digiKam progress dialog, xcpeted that list view do not
exist. There is only a KSqueezedLabel which display current items processed one
by one...

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=216894


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|1.0.0-beta6                 |1.0.0




--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=216894


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |FIXED




--- Comment #8 from Marcel Wiesweg <marcel wiesweg gmx de>  2011-03-05 21:42:24 ---
Fixed since a long time, Gilles changed it as described in #7

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=216894

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|                            |2.0.0

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel