[digikam] [Bug 372435] New: Use multiple cpu cores for duplicate search

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=372435

            Bug ID: 372435
           Summary: Use multiple cpu cores for duplicate search
           Product: digikam
           Version: 5.3.0
          Platform: Other
                OS: Linux
            Status: UNCONFIRMED
          Severity: wishlist
          Priority: NOR
         Component: Searches-Fuzzy
          Assignee: [hidden email]
          Reporter: [hidden email]
  Target Milestone: ---

When searching a big amount of pictures for duplicates the search takes a lot
of time. One cpu core is at 100% the whole time, the others are almost idle.
It would be great if all cpu cores were used to speed the search up.

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=372435

Wolfgang Scheffner <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #1 from Wolfgang Scheffner <[hidden email]> ---
I support this request.

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=372435

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #2 from [hidden email] ---
Look here :

https://quickgit.kde.org/?p=digikam.git&a=blob&&f=utilities%2Fmaintenance%2Fduplicatesfinder.cpp

This is the maintenance tool used to process a duplicates search album, through
the multithreaded DBJob interface.

Currently, multicore support is not set to the DBjob. This can be a simpler
solution, if Haar algorithm from Database is enough re-entrant. I must admit
that i never tested this case, and this is why multi core support is not
enabled here.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=372435

Christian David <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|                            |5.0
             Status|UNCONFIRMED                 |RESOLVED
      Latest Commit|                            |http://commits.kde.org/kmym
                   |                            |oney/64d8749496a04c3be88300
                   |                            |f040ae1c1e14af8103
         Resolution|---                         |FIXED

--- Comment #3 from Christian David <[hidden email]> ---
Git commit 64d8749496a04c3be88300f040ae1c1e14af8103 by Christian Dávid.
Committed on 15/11/2016 at 18:09.
Pushed by christiand into branch 'master'.

Fixed constructors in KBanking

Some class members were not initialized correctly what potentially
caused crashes.
FIXED-IN: 5.0

M  +11   -2    kmymoney/plugins/kbanking/mymoneybanking.cpp

http://commits.kde.org/kmymoney/64d8749496a04c3be88300f040ae1c1e14af8103

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=372435

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
         Resolution|FIXED                       |---
             Status|RESOLVED                    |REOPENED

--- Comment #4 from [hidden email] ---
comment #3 come from another world (:=)))

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=372435

Christian David <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|5.0                         |
      Latest Commit|http://commits.kde.org/kmym |
                   |oney/64d8749496a04c3be88300 |
                   |f040ae1c1e14af8103          |

--- Comment #5 from Christian David <[hidden email]> ---
(In reply to caulier.gilles from comment #4)
> comment #3 come from another world (:=)))

I am sorry (you were fast reopening this bug)!

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=372435

Ralf Habacker <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Latest Commit|                            |http://commits.kde.org/kmym
                   |                            |oney/ee2e88ec3dc89d21ecfdad
                   |                            |cb3131ea6bb4ad7c76
             Status|REOPENED                    |RESOLVED
   Version Fixed In|                            |5.0
         Resolution|---                         |FIXED

--- Comment #6 from Ralf Habacker <[hidden email]> ---
Git commit ee2e88ec3dc89d21ecfdadcb3131ea6bb4ad7c76 by Ralf Habacker.
Committed on 15/11/2016 at 22:54.
Pushed by habacker into branch '4.8'.

Fixed constructors in KBanking

Some class members were not initialized correctly what potentially
caused crashes.
FIXED-IN: 5.0
(cherry picked from commit 64d8749496a04c3be88300f040ae1c1e14af8103)

M  +11   -2    kmymoney/plugins/kbanking/mymoneybanking.cpp

http://commits.kde.org/kmymoney/ee2e88ec3dc89d21ecfdadcb3131ea6bb4ad7c76

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=372435

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|FIXED                       |---
             Status|RESOLVED                    |REOPENED

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=372435

Alister Troup <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #7 from Alister Troup <[hidden email]> ---
This appears still to be an issue - at least on the windows version 5.6.0 x64

I'm running on a xeon 1231 v3 (4 cores plus HT) 16GB RAM, using MarinaDB server
on local Host, SSD as OS/DatabaseData

Currently doing a "Find Duplicates" on a large collection

Digikam.exe

Threads 254
CPU 12
Average CPU 12.42

Process Explorer
msvcrt.dll!ftime64_s+0x180 thread is ~12.5 CPU

Stack capture

ntoskrnl.exe!memset+0x61a
ntoskrnl.exe!KeWaitForMultipleObjects+0xd52
ntoskrnl.exe!KeWaitForSingleObject+0x19f
ntoskrnl.exe!PoStartNextPowerIrp+0xbd0
ntoskrnl.exe!PoStartNextPowerIrp+0x186d
ntoskrnl.exe!PoStartNextPowerIrp+0x1ae7
libdigikamdatabase.dll!ZN7Digikam9HaarIface14calculateScoreERNS_4Haar13SignatureDataES3_RNS1_7WeightsEPPNS1_12SignatureMapE+0xd4
libdigikamdatabase.dll!ZN7Digikam9HaarIface14searchDatabaseEPNS_4Haar13SignatureDataENS0_10SketchTypeER5QListIiENS0_28DuplicatesSearchRestrictionsExi+0xb12
libdigikamdatabase.dll!ZN7Digikam9HaarIface24bestMatchesWithThresholdExPNS_4Haar13SignatureDataEddR5QListIiENS0_28DuplicatesSearchRestrictionsENS0_10SketchTypeE+0xd2
libdigikamdatabase.dll!ZN7Digikam9HaarIface32bestMatchesForImageWithThresholdExddR5QListIiENS0_28DuplicatesSearchRestrictionsENS0_10SketchTypeE+0x158
libdigikamdatabase.dll!ZN7Digikam9HaarIface14findDuplicatesERK4QSetIxEddNS0_28DuplicatesSearchRestrictionsEPNS_20HaarProgressObserverE+0x273
libdigikamdatabase.dll!ZN7Digikam9HaarIface29findDuplicatesInAlbumsAndTagsERK5QListIiES4_NS0_16AlbumTagRelationEddNS0_28DuplicatesSearchRestrictionsEPNS_20HaarProgressObserverE+0x351
libdigikamdatabase.dll!ZN7Digikam9HaarIface23rebuildDuplicatesAlbumsERK5QListIiES4_NS0_16AlbumTagRelationEddNS0_28DuplicatesSearchRestrictionsEPNS_20HaarProgressObserverE+0x67
libdigikamdatabase.dll!ZN7Digikam11SearchesJob3runEv+0x8f2
Qt5Core.dll!ZN11QThreadPool5clearEv+0xe5
Qt5Core.dll!ZN7QThread21setTerminationEnabledEb+0x14c
msvcrt.dll!srand+0x93
msvcrt.dll!ftime64_s+0x1dd
kernel32.dll!BaseThreadInitThunk+0xd
ntdll.dll!RtlUserThreadStart+0x21


Memory is 696 MB private Bytes

I/O is 380B average peaks to 26KB

--
You are receiving this mail because:
You are the assignee for the bug.