Very, very slow tagging
Locked
 
|
DK 5.7.0 (sqlite) on archlinux with 12gb RAM, AMD 8350 (8 cores)
I just tagged 2365 images with 1 tag and now it's "stuck" HTOP: 6 cores: 100% and 2 cores 99.7% & 99.8%) no disk IO, no swap. HTOP: memory used: 1.42G from 11.7G It's been like that for the last 25 minutes. Any way to speed this up? -- sknahT vyS |
|
|
It's been running for 2 hours and 29 minutues and it's still running!
Help! On Thu, 2 Nov 2017 14:11:16 -0700 [hidden email] wrote: > DK 5.7.0 (sqlite) on archlinux with 12gb RAM, AMD 8350 (8 cores) > > I just tagged 2365 images with 1 tag and now it's "stuck" > > HTOP: 6 cores: 100% and 2 cores 99.7% & 99.8%) no disk IO, no swap. > > HTOP: memory used: 1.42G from 11.7G > > It's been like that for the last 25 minutes. > > Any way to speed this up? > > -- > sknahT > > vyS -- sknahT vyS |
|
|
In reply to this post by digikam-2
What is going on in the terminal ? Sent from my Samsung Galaxy smartphone. -------- Original message -------- From: [hidden email] Date: 2017-11-02 5:27 PM (GMT-07:00) To: [hidden email] Subject: Re: Very, very slow tagging Help! On Thu, 2 Nov 2017 14:11:16 -0700 [hidden email] wrote: > DK 5.7.0 (sqlite) on archlinux with 12gb RAM, AMD 8350 (8 cores) > > I just tagged 2365 images with 1 tag and now it's "stuck" > > HTOP: 6 cores: 100% and 2 cores 99.7% & 99.8%) no disk IO, no swap. > > HTOP: memory used: 1.42G from 11.7G > > It's been like that for the last 25 minutes. > > Any way to speed this up? > > -- > sknahT > > vyS -- sknahT vyS |
|
|
In reply to this post by digikam-2
If you can reproduce it, run digikam in gdb and hit ctrl-c after a while
of such high activity. Then get the backtrace ("bt") and send the result - there hopefully is a clue what it is doing. On 03/11/17 00:27, [hidden email] wrote: > It's been running for 2 hours and 29 minutues and it's still running! > > Help! > > On Thu, 2 Nov 2017 14:11:16 -0700 > [hidden email] wrote: > >> DK 5.7.0 (sqlite) on archlinux with 12gb RAM, AMD 8350 (8 cores) >> >> I just tagged 2365 images with 1 tag and now it's "stuck" >> >> HTOP: 6 cores: 100% and 2 cores 99.7% & 99.8%) no disk IO, no swap. >> >> HTOP: memory used: 1.42G from 11.7G >> >> It's been like that for the last 25 minutes. >> >> Any way to speed this up? >> >> -- >> sknahT >> >> vyS > |
|
|
What's happen with digiKam universal AppImage Linux bundle ? Current pre-version 5.8.0 is avaialble here : Gilles Caulier 2017-11-03 11:53 GMT+01:00 Simon Frei <[hidden email]>: If you can reproduce it, run digikam in gdb and hit ctrl-c after a while |
|
|
On Fri, 3 Nov 2017 15:13:52 +0100
Gilles Caulier <[hidden email]> wrote: > What's happen with digiKam universal AppImage Linux bundle ? Current > pre-version 5.8.0 is avaialble here : > > https://files.kde.org/digikam/ Tried both, same -- sknahT vyS |
|
|
In reply to this post by Simon Frei
On Fri, 3 Nov 2017 11:53:14 +0100
Simon Frei <[hidden email]> wrote: > If you can reproduce it, run digikam in gdb and hit ctrl-c after a > while of such high activity. Then get the backtrace ("bt") and send > the result > - there hopefully is a clue what it is doing. Later tonight, I'll try. 1. Do I run gdb digikam? 2. How do I get the backtrace? What's bt? As you can see, I've never done this before... -- sknahT vyS |
|
|
On 03/11/17 15:48, [hidden email] wrote:
> On Fri, 3 Nov 2017 11:53:14 +0100 > Simon Frei <[hidden email]> wrote: > >> If you can reproduce it, run digikam in gdb and hit ctrl-c after a >> while of such high activity. Then get the backtrace ("bt") and send >> the result >> - there hopefully is a clue what it is doing. > Later tonight, I'll try. > > 1. Do I run gdb digikam? > 2. How do I get the backtrace? What's bt? > > As you can see, I've never done this before... > https://www.digikam.org/contribute/ |
|
|
In reply to this post by Simon Frei
OK:
The file is available at: https://drive.google.com/open?id=0B7HTmJCO2gG7OWFOUGtGTEFNTFE It starts at line: 386 ................................................ digikam.general: Writting tags digikam.general: -------------------------- New Keywords ("animals", "dogs", "dog photography") digikam.metaengine: xmlACDSee "<Categories><Category Assigned=\"1\">animals</Category><Category Assigned=\"1\">dogs</Category><Category Assigned=\"1\">dog photography</Category></Categories>" digikam.metaengine: "/photos/digikam/streets/dogs/dogs-2016/dogs-vancouver-20160630-2812.cr2" ==> Read Iptc Keywords: () digikam.metaengine: "/photos/digikam/streets/dogs/dogs-2016/dogs-vancouver-20160630-2812.cr2" ==> New Iptc Keywords: ("animals", "dogs", "dog photography") digikam.metaengine: MetaEngine::metadataWritingMode 2 digikam.metaengine: Will write Metadata to file "/photos/digikam/streets/dogs/dogs-2016/dogs-vancouver-20160630-2812.cr2" digikam.metaengine: wroteComment: false digikam.metaengine: wroteEXIF: true digikam.metaengine: wroteIPTC: true digikam.metaengine: wroteXMP: true ................................................ then by line: 786: ................................................ digikam.database: Scanning took 5156 ms digikam.metaengine: File time stamp restored digikam.metaengine: Metadata for file "dogs-vancouver-20160409-1631.cr2" written to file. digikam.metaengine: Will write XMP sidecar for file "dogs-vancouver-20160409-1631.cr2" digikam.general: Detected change, triggering rescan of "/photos/digikam/streets/dogs/dogs-2016/" digikam.metaengine: wroteComment: false digikam.metaengine: wroteEXIF: true digikam.metaengine: wroteIPTC: true digikam.metaengine: wroteXMP: true digikam.metaengine: File time stamp restored digikam.metaengine: Metadata for file "dogs-vancouver-20160409-1631.cr2" written to XMP sidecar. digikam.general: Detected change, triggering rescan of "/photos/digikam/streets/dogs/dogs-2016/" digikam.database: Starting scan! digikam.database: Finishing took 3731 ms [Thread 0x7ffee7fff700 (LWP 4684) exited] ....................................................................... Then the scanning takes longer and longer... ....................................................................... Database stats: digikam version 5.7.0 Images: JPG: 191 RAW-CR2: 2188 RAW-RAF: 558 total: 2937 : Total Items: 2937 Albums: 12 Tags: 32 : Database backend: QSQLITE Database Path: /photos/photos/ ............................................................... On Fri, 3 Nov 2017 11:53:14 +0100 Simon Frei <[hidden email]> wrote: > If you can reproduce it, run digikam in gdb and hit ctrl-c after a > while of such high activity. Then get the backtrace ("bt") and send > the result > - there hopefully is a clue what it is doing. > > On 03/11/17 00:27, [hidden email] wrote: > > It's been running for 2 hours and 29 minutues and it's still > > running! > > > > Help! > > > > On Thu, 2 Nov 2017 14:11:16 -0700 > > [hidden email] wrote: > > > >> DK 5.7.0 (sqlite) on archlinux with 12gb RAM, AMD 8350 (8 cores) > >> > >> I just tagged 2365 images with 1 tag and now it's "stuck" > >> > >> HTOP: 6 cores: 100% and 2 cores 99.7% & 99.8%) no disk IO, no > >> swap. > >> > >> HTOP: memory used: 1.42G from 11.7G > >> > >> It's been like that for the last 25 minutes. > >> > >> Any way to speed this up? > >> > >> -- > >> sknahT > >> > >> vyS > > > -- sknahT vyS |
|
|
Do you use a local drive or a network drive on which the images are? The used
time from the image scanner are very very long. Maik Am Freitag, 3. November 2017, 18:59:13 CET schrieb [hidden email]: > OK: > > The file is available at: > https://drive.google.com/open?id=0B7HTmJCO2gG7OWFOUGtGTEFNTFE > > It starts at line: 386 > > ................................................ > digikam.general: Writting tags > digikam.general: -------------------------- New Keywords ("animals", "dogs", > "dog photography") digikam.metaengine: xmlACDSee "<Categories><Category > Assigned=\"1\">animals</Category><Category > Assigned=\"1\">dogs</Category><Category Assigned=\"1\">dog > photography</Category></Categories>" digikam.metaengine: > "/photos/digikam/streets/dogs/dogs-2016/dogs-vancouver-20160630-2812.cr2" > ==> Read Iptc Keywords: () digikam.metaengine: > "/photos/digikam/streets/dogs/dogs-2016/dogs-vancouver-20160630-2812.cr2" > ==> New Iptc Keywords: ("animals", "dogs", "dog photography") > digikam.metaengine: MetaEngine::metadataWritingMode 2 > digikam.metaengine: Will write Metadata to file > "/photos/digikam/streets/dogs/dogs-2016/dogs-vancouver-20160630-2812.cr2" > digikam.metaengine: wroteComment: false > digikam.metaengine: wroteEXIF: true > digikam.metaengine: wroteIPTC: true > digikam.metaengine: wroteXMP: true > ................................................ > > then by line: 786: > > ................................................ > digikam.database: Scanning took 5156 ms > digikam.metaengine: File time stamp restored > digikam.metaengine: Metadata for file "dogs-vancouver-20160409-1631.cr2" > written to file. digikam.metaengine: Will write XMP sidecar for file > "dogs-vancouver-20160409-1631.cr2" digikam.general: Detected change, > triggering rescan of "/photos/digikam/streets/dogs/dogs-2016/" > digikam.metaengine: wroteComment: false > digikam.metaengine: wroteEXIF: true > digikam.metaengine: wroteIPTC: true > digikam.metaengine: wroteXMP: true > digikam.metaengine: File time stamp restored > digikam.metaengine: Metadata for file "dogs-vancouver-20160409-1631.cr2" > written to XMP sidecar. digikam.general: Detected change, triggering rescan > of "/photos/digikam/streets/dogs/dogs-2016/" digikam.database: Starting > scan! > digikam.database: Finishing took 3731 ms > [Thread 0x7ffee7fff700 (LWP 4684) exited] > > ....................................................................... > > Then the scanning takes longer and longer... > > ....................................................................... > Database stats: > > digikam version 5.7.0 > Images: > JPG: 191 > RAW-CR2: 2188 > RAW-RAF: 558 > total: 2937 > > Total Items: 2937 > Albums: 12 > Tags: 32 > > Database backend: QSQLITE > Database Path: /photos/photos/ > ............................................................... > > On Fri, 3 Nov 2017 11:53:14 +0100 > > Simon Frei <[hidden email]> wrote: > > If you can reproduce it, run digikam in gdb and hit ctrl-c after a > > while of such high activity. Then get the backtrace ("bt") and send > > the result > > - there hopefully is a clue what it is doing. > > > > On 03/11/17 00:27, [hidden email] wrote: > > > It's been running for 2 hours and 29 minutues and it's still > > > running! > > > > > > Help! > > > > > > On Thu, 2 Nov 2017 14:11:16 -0700 > > > > > > [hidden email] wrote: > > >> DK 5.7.0 (sqlite) on archlinux with 12gb RAM, AMD 8350 (8 cores) > > >> > > >> I just tagged 2365 images with 1 tag and now it's "stuck" > > >> > > >> HTOP: 6 cores: 100% and 2 cores 99.7% & 99.8%) no disk IO, no > > >> swap. > > >> > > >> HTOP: memory used: 1.42G from 11.7G > > >> > > >> It's been like that for the last 25 minutes. > > >> > > >> Any way to speed this up? > > >> > > >> > > >> vyS |
|
|
On Fri, 03 Nov 2017 20:59:42 +0100
Maik Qualmann <[hidden email]> wrote: > Do you use a local drive or a network drive on which the images > are? The used time from the image scanner are very very long. local drive: sdb1 [froggy@dodoite ~]$ df -h Filesystem Size Used Avail Use% Mounted on dev 5.9G 0 5.9G 0% /dev run 5.9G 1.3M 5.9G 1% /run /dev/sda2 29G 17G 12G 60% / tmpfs 5.9G 0 5.9G 0% /dev/shm tmpfs 5.9G 0 5.9G 0% /sys/fs/cgroup /dev/sdb1 1.8T 785G 956G 46% /photos <<<<<<<<<<< tmpfs 5.9G 16K 5.9G 1% /tmp /dev/sda1 247M 82M 163M 34% /boot /dev/sda3 1.8T 1.1T 590G 66% /home tmpfs 1.2G 16K 1.2G 1% /run/user/1000 -- sknahT vyS |
|
|
Are you sure that the drive is not broken? I see times between 2 to 10
seconds, but also up to 17 seconds in the log for reading the image information. These times should probably not be greater than 200 ms on your machine. I think the problem is outside of digiKam. Maik Am Freitag, 3. November 2017, 21:43:53 CET schrieb [hidden email]: > On Fri, 03 Nov 2017 20:59:42 +0100 > > Maik Qualmann <[hidden email]> wrote: > > Do you use a local drive or a network drive on which the images > > are? The used time from the image scanner are very very long. > > local drive: sdb1 > > [froggy@dodoite ~]$ df -h > Filesystem Size Used Avail Use% Mounted on > dev 5.9G 0 5.9G 0% /dev > run 5.9G 1.3M 5.9G 1% /run > /dev/sda2 29G 17G 12G 60% / > tmpfs 5.9G 0 5.9G 0% /dev/shm > tmpfs 5.9G 0 5.9G 0% /sys/fs/cgroup > /dev/sdb1 1.8T 785G 956G 46% /photos <<<<<<<<<<< > tmpfs 5.9G 16K 5.9G 1% /tmp > /dev/sda1 247M 82M 163M 34% /boot > /dev/sda3 1.8T 1.1T 590G 66% /home > tmpfs 1.2G 16K 1.2G 1% /run/user/1000 |
|
|
On Fri, 03 Nov 2017 22:06:26 +0100
Maik Qualmann <[hidden email]> wrote: > Are you sure that the drive is not broken? I see times between 2 to > 10 seconds, but also up to 17 seconds in the log for reading the > image information. These times should probably not be greater than > 200 ms on your machine. I think the problem is outside of digiKam. unlikely to be the drive, it's a 3 years old drive 2Tb. The other thing is that it's a small database (2100 images) with most of it in memory, it's CPU cycles that max out: the 8 cores at 100%. smartctl reports: no error logged ..................................... SMART overall-health self-assessment test result: PASSED ... ... plenty of other stuff ... ... ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 167 163 021 Pre-fail Always - 6641 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 910 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 063 063 000 Old_age Always - 27466 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 845 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 87 193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 1799713 194 Temperature_Celsius 0x0022 123 105 000 Old_age Always - 27 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 21 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 1 SMART Error Log Version: 1 No Errors Logged ..................................... -- sknahT vyS |
|
|
Your value for Load_Cycle_Count is very extreme. WD hard drive? Do a reading
test with hdparm -tT /dev/sdb1 Maik Am Freitag, 3. November 2017, 22:22:23 CET schrieb [hidden email]: > On Fri, 03 Nov 2017 22:06:26 +0100 > > Maik Qualmann <[hidden email]> wrote: > > Are you sure that the drive is not broken? I see times between 2 to > > 10 seconds, but also up to 17 seconds in the log for reading the > > image information. These times should probably not be greater than > > 200 ms on your machine. I think the problem is outside of digiKam. > > unlikely to be the drive, it's a 3 years old drive 2Tb. > > The other thing is that it's a small database (2100 images) with most of it > in memory, it's CPU cycles that max out: the 8 cores at 100%. > > smartctl reports: no error logged > > ..................................... > SMART overall-health self-assessment test result: PASSED > > ... > ... plenty of other stuff ... > ... > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED > WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 > Pre-fail Always - 0 3 Spin_Up_Time 0x0027 167 > 163 021 Pre-fail Always - 6641 4 Start_Stop_Count > 0x0032 100 100 000 Old_age Always - 910 5 > Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always > - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age > Always - 0 9 Power_On_Hours 0x0032 063 063 000 > Old_age Always - 27466 10 Spin_Retry_Count 0x0032 > 100 100 000 Old_age Always - 0 11 > Calibration_Retry_Count 0x0032 100 100 000 Old_age Always > - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age > Always - 845 192 Power-Off_Retract_Count 0x0032 200 200 > 000 Old_age Always - 87 193 Load_Cycle_Count 0x0032 > 001 001 000 Old_age Always - 1799713 194 > Temperature_Celsius 0x0022 123 105 000 Old_age Always > - 27 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age > Always - 0 197 Current_Pending_Sector 0x0032 200 200 > 000 Old_age Always - 21 198 Offline_Uncorrectable 0x0030 > 200 200 000 Old_age Offline - 0 199 > UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always > - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age > Offline - 1 > > SMART Error Log Version: 1 > No Errors Logged > > > ..................................... |
|
|
On Fri, 03 Nov 2017 22:50:37 +0100
Maik Qualmann <[hidden email]> wrote: > Your value for Load_Cycle_Count is very extreme. WD hard drive? Do > a reading test with hdparm -tT /dev/sdb1 /dev/sdb1: Timing cached reads: 9002 MB in 2.00 seconds = 4511.15 MB/sec Timing buffered disk reads: 338 MB in 3.00 seconds = 112.58 MB/sec -- sknahT vyS |
«
Return to digikam-users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |