Hi all,
using 4.9 I'm getting it being really slow when re-opening an album. Images in such album are jpg around 15 MB each, thumbnails have been already done in the past, but the table view still renders a small thumbnail every 2-3 seconds. Clicking on an image requires several seconds (20-30) for it to open, and once it is shown thumbnails in the neighborhood are displayed as well. Other albums are working fine, and all are on the same removable hard disk. Is there some tunable I can check to understand what is slowing down the thumbnail view? Thanks, Luca _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
Hi, The thumbnails in digiKam are stored in a dedicated database sqlite file. Look my root collection on my host linux computer mounted on a SSD : [gilles@localhost photos]$ pwd /mnt/data2/photos [gilles@localhost photos]$ ls -al total 1332572 drwxrwxrwx 5 gilles gilles 4096 Sep 30 22:11 ./ drwxr-xr-x 6 root root 4096 Jul 6 18:04 ../ drwxrwxr-x 55 gilles gilles 4096 Sep 5 2014 AGNES/ -rw-r--r-- 1 gilles gilles 64644096 Sep 30 22:11 digikam4.db drwxrwxrwx 4 gilles gilles 4096 Dec 4 2013 GILLES/ drwxrwxr-x 53 gilles gilles 4096 May 11 23:13 TESTS/ -rw-r--r-- 1 gilles gilles 1299880960 Sep 30 22:11 thumbnails-digikam.db [gilles@localhost photos]$ du -hs 229G . The thumbnails-digikam.db only store wavelets compressed thumbnails. As you can see it's huge, because i use WQHD flat screens here and I turned on the HD thumbs support (512*512 pixels instead 256*256). A DB file is used instead file-system to speedup thumbnails access in all cases : local, remote, and removable collections. For ex, if disk is not available, displaying thumbs still possible. To test if DB is corrupted, you can delete/rename safely the DB file for thumbnails. There is no extra metatada in this file. It will be re-created automatically at next session, but this take a while to regenerate thumbs until there are cached in DB. You can use the Maintenance tool to recreate all thumbnails, but take a coffee and be patient. Best Gilles Caulier 2015-10-01 9:04 GMT+02:00 Luca Ferrari <[hidden email]>: Hi all, _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
On Thu, Oct 1, 2015 at 10:41 AM, Gilles Caulier
<[hidden email]> wrote: > To test if DB is corrupted, you can delete/rename safely the DB file for > thumbnails. There is no extra metatada in this file. It will be re-created > automatically at next session, but this take a while to regenerate thumbs > until there are cached in DB. You can use the Maintenance tool to recreate > all thumbnails, but take a coffee and be patient. I'm trying to rebuild the thumbnail database (and only that on all available cores, but after 3+ hours only 2% is done and the database is about 12 MB (digikam4.db = 50 MB). I will let it working for hours, but this is going to take much more than a coffee.... Any suggestion? Luca _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
No idea. here, with 200 Gb of photo on a SSD, it take 10mns... JPG +RAW. Gilles Caulier 2015-10-05 12:06 GMT+02:00 Luca Ferrari <[hidden email]>: On Thu, Oct 1, 2015 at 10:41 AM, Gilles Caulier _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
On Mon, Oct 5, 2015 at 12:13 PM, Gilles Caulier
<[hidden email]> wrote: > No idea. > > here, with 200 Gb of photo on a SSD, it take 10mns... JPG +RAW. After 5 hours at 2% and without any increment in the thumbnails.db file I stopped the process (digikam was not blocked). Now I've restored the old thumbnail database and start a rebuild with only changed items, but after 1+ hour it is still at 1%. I've around 200GB on an USB drive (jpg+raf and some mp4). Is there a way to check what the program is doing and if there some lock anywhere? Luca _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
yes. Follow instructions into "For hangs out or dysfunctions" section from : Gilles Caulier PS : did you store DB files on removable USB drive or in local computer ? 2015-10-05 14:01 GMT+02:00 Luca Ferrari <[hidden email]>: On Mon, Oct 5, 2015 at 12:13 PM, Gilles Caulier _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
On Mon, Oct 5, 2015 at 2:14 PM, Gilles Caulier <[hidden email]> wrote:
> yes. Follow instructions into "For hangs out or dysfunctions" section from : > > https://www.digikam.org/contrib Thanks, at least I'm seeing that the process is runnig. So far, after 30 mins it is still at 0% and the console reports a line every 30 seconds: digikam(10312)/digikam (core) Digikam::DImg::load: "/media/luca/Marco_A_0014.jpg" : JPEG file identified digikam(10312)/digikam (core) Digikam::DImg::load: "/media/luca/Marco_A_0015.jpg" : JPEG file identified Such images are around 50 MB each. The preamble before the process starts is: digikam(10312)/digikam (core) Digikam::MaintenanceMngr::setSettings: wholeAlbums : true wholeTags : false Albums : 1124 Tags : 0 useMutiCoreCPU : true newItems : false thumbnails : true scanThumbs : false fingerPrints : false scanFingerPrints : true duplicates : false similarity : 90 faceManagement : false faceScannedHandling : 0 qualitySort : false quality : EnableSorter : false DetectBlur : true DetectNoise : true DetectCompression : true DetectOverexposure :true LowQRejected : true MediumQPending : true HighQAccepted : true Speed : 1 Rejected Threshold: 10 Pending Threshold : 40 Accepted Threshold: 60 Blur Weight : 100 Noise Weight : 100 Compression Weight: 100 qualityScanMode : 0 metadataSync : false syncDirection : 0 digikam(10312)/digikam (core) Digikam::MaintenanceMngr::stage1: stage1 digikam(10312)/digikam (core) Digikam::MaintenanceMngr::stage2: stage2 I'll leave the process go and see if it hungs somewhere. Is there a way to instrument digikam to produce faster/low quality thumbnails for larger images? Luca _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
I'm also getting now a lot of:
digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. digikam(11191)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile. for each image, and each image is now processed at a speed of 10 secs (size of image is around 5 MB). So far I'm stuck: after 3+ hours I'm still at 2% of the process, meaning it will going on for days before it ends. I'm recovering the old thumbnail database (that could be corrupt) and try to rebuild some parts of it. Any suggestion is welcome. Thanks, Luca On Mon, Oct 5, 2015 at 3:36 PM, Luca Ferrari <[hidden email]> wrote: > On Mon, Oct 5, 2015 at 2:14 PM, Gilles Caulier <[hidden email]> wrote: >> yes. Follow instructions into "For hangs out or dysfunctions" section from : >> >> https://www.digikam.org/contrib > > Thanks, at least I'm seeing that the process is runnig. So far, after > 30 mins it is still at 0% and the console reports a line every 30 > seconds: > > digikam(10312)/digikam (core) Digikam::DImg::load: > "/media/luca/Marco_A_0014.jpg" : JPEG file identified > digikam(10312)/digikam (core) Digikam::DImg::load: > "/media/luca/Marco_A_0015.jpg" : JPEG file identified > > Such images are around 50 MB each. > > The preamble before the process starts is: > > > digikam(10312)/digikam (core) Digikam::MaintenanceMngr::setSettings: > wholeAlbums : true > wholeTags : false > Albums : 1124 > Tags : 0 > useMutiCoreCPU : true > newItems : false > thumbnails : true > scanThumbs : false > fingerPrints : false > scanFingerPrints : true > duplicates : false > similarity : 90 > faceManagement : false > faceScannedHandling : 0 > qualitySort : false > quality : > EnableSorter : false > DetectBlur : true > DetectNoise : true > DetectCompression : true > DetectOverexposure :true > LowQRejected : true > MediumQPending : true > HighQAccepted : true > Speed : 1 > Rejected Threshold: 10 > Pending Threshold : 40 > Accepted Threshold: 60 > Blur Weight : 100 > Noise Weight : 100 > Compression Weight: 100 > > qualityScanMode : 0 > metadataSync : false > syncDirection : 0 > > digikam(10312)/digikam (core) Digikam::MaintenanceMngr::stage1: stage1 > digikam(10312)/digikam (core) Digikam::MaintenanceMngr::stage2: stage2 > > > I'll leave the process go and see if it hungs somewhere. > Is there a way to instrument digikam to produce faster/low quality > thumbnails for larger images? > > Luca Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
did you store DB files on removable USB drive or in local computer ? Which Exiv2 version you use ? Go to Help/Components Info for details. Did you use multicore option in Maintenance tool ? If you turn off this option, this improve the speed? Gilles Caulier 2015-10-05 17:43 GMT+02:00 Luca Ferrari <[hidden email]>: I'm also getting now a lot of: _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
On Mon, Oct 5, 2015 at 7:29 PM, Gilles Caulier <[hidden email]> wrote:
> did you store DB files on removable USB drive or in local computer ? > On the local computer, thinking this would improve speed without consuming bandwidth along the usb path. > Which Exiv2 version you use ? Go to Help/Components Info for details. > digiKam version 4.2.0 CPU cores: 2 Demosaic GPL2 pack support: Unknown Demosaic GPL3 pack support: Unknown Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibEigen: 3.2.1 LibExiv2: 0.24 LibJPEG: 80 LibJasper: 1.900.1 LibKDE: 4.14.1 LibKExiv2: 2.3.1 LibKGeoMap: 2.0.0 LibKdcraw: 2.4.2 LibLCMS: 2060 LibLensFun: 0.2.8-0 LibPGF: 6.12.24 - external shared library LibPNG: 1.2.51 LibQt: 4.8.6 LibRaw: 0.16.0 LibTIFF: LIBTIFF, Version 4.0.3 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.18.21 (0.19 Alpha) Parallelized PGF codec: No Parallelized demosaicing: Unknown RawSpeed codec support: Unknown Database backend: QSQLITE Kipi-Plugins: 4.2.0 LibGphoto2: 2.5.4 LibKface: 3.4.0 LibKipi: 2.1.0 LibOpenCV: 2.4.9 > Did you use multicore option in Maintenance tool ? If you turn off this > option, this improve the speed? Yes, but no changes are perceivable in time and speed. However, with the multi core option enabled the other programs slow down too, so something is surely running on more cores. I admit I'm using an old version of digikam, but it is the only one I'm able to install on this machine so far. Thanks, Luca _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
yes 4.2.0 is old. New one is 4.13.0. Also, look to update Exiv2 from 0.24 to 0.25. 0.24 is very problematic. To extract thumbnails from images we use Exiv2 Gilles Caulier 2015-10-06 8:40 GMT+02:00 Luca Ferrari <[hidden email]>: On Mon, Oct 5, 2015 at 7:29 PM, Gilles Caulier <[hidden email]> wrote: _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
On Tue, Oct 6, 2015 at 10:05 AM, Gilles Caulier
<[hidden email]> wrote: > yes 4.2.0 is old. New one is 4.13.0. > > Also, look to update Exiv2 from 0.24 to 0.25. 0.24 is very problematic. To > extract thumbnails from images we use Exiv2 I've compiled 4.12 on my machine, with Exiv 0.25 as reported below. I've launched the thumbanil rebuild on the whole album set but after 20+ minutes the program is still frozend and the debug output reports only the stage 2. Is it possible there is something not working with my setup? wholeAlbums : true wholeTags : false Albums : 1124 Tags : 0 useMutiCoreCPU : false newItems : false thumbnails : true scanThumbs : false fingerPrints : false scanFingerPrints : true duplicates : false similarity : 90 faceManagement : false faceScannedHandling : 0 qualitySort : false quality : EnableSorter : false DetectBlur : true DetectNoise : true DetectCompression : true DetectOverexposure :true LowQRejected : true MediumQPending : true HighQAccepted : true Speed : 1 Rejected Threshold: 10 Pending Threshold : 40 Accepted Threshold: 60 Blur Weight : 100 Noise Weight : 100 Compression Weight: 100 qualityScanMode : 0 metadataSync : false syncDirection : 0 digikam(14333)/digikam (core) Digikam::MaintenanceMngr::stage1: stage1 digikam(14333)/digikam (core) Digikam::MaintenanceMngr::stage2: stage2 This is the component info: digiKam version 4.12.0 CPU cores: 2 Demosaic GPL2 pack support: Unknown Demosaic GPL3 pack support: Unknown Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibEigen support: no LibExiv2: 0.25 LibJPEG: 80 LibJasper support: no LibKDE: 4.14.1 LibKExiv2: 2.4.0 LibKGeoMap: 3.1.0 LibKdcraw: 2.4.2 LibLCMS: 2060 LibLensFun support: no LibLqr support: no LibPGF: 6.14.12 LibPNG: 1.2.51 LibQt: 4.8.6 LibRaw: 0.17.0 LibTIFF: LIBTIFF, Version 4.0.3 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.19.0 (stable release) Parallelized demosaicing: Unknown RawSpeed codec support: Unknown Baloo support: no Database backend: QSQLITE Kdepimlibs support: Yes Kipi-Plugins: 4.12.0 LibGphoto2 support: no LibKface: 3.5.0 LibKipi: 2.2.0 LibOpenCV: 2.4.11 Sqlite2 support: no _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
On Thu, Oct 8, 2015 at 11:04 AM, Luca Ferrari <[hidden email]> wrote:
> I've compiled 4.12 on my machine, with Exiv 0.25 as reported below. > I've launched the thumbanil rebuild on the whole album set but after > 20+ minutes the program is still frozend and the debug output reports > only the stage 2. > Is it possible there is something not working with my setup? So I got it working in a reasonable time: I made the thumbnail rebuild in chunks of albums (around 200 albums per chunk) and the program keeps responsive, showing progress and completing each chunk. The thumbnail database is growing and it's now 750 MB in size, I'm running the last chunk. Luca _______________________________________________ Digikam-users mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-users |
Free forum by Nabble | Edit this page |