digiKam freezes when scanning a new collection
Locked
digiKam freezes when scanning a new collection
 
|
I have a Core i5 CPU with 4 cores and when I add a collection from an external hard drive, digiKam becomes unresponsive.
Would it be possible touse a separate thread to scan new collections? |
Re: digiKam freezes when scanning a new collection
|
|
All is multithreaded in digiKam : scan items and database registration. If you use a remote database, network latency can be a problem. Didi you try to use a local database ? Gilles Caulier 2017-01-22 6:52 GMT+01:00 Dan Dascalescu <[hidden email]>:
|
Re: digiKam freezes when scanning a new collection
|
|
Gilles Caulier <[hidden email]> wrote:
> 2017-01-22 6:52 GMT+01:00 Dan Dascalescu <[hidden email]>: > > > I have a Core i5 CPU with 4 cores and when I add a collection from an > > external hard drive, digiKam becomes unresponsive. > > > > Would it be possible touse a separate thread to scan new collections? > > > > All is multithreaded in digiKam : scan items and database registration. > > If you use a remote database, network latency can be a problem. Didi you > try to use a local database ? > directories to digikam. I have a similar CPU and 8Gb RAM with digikam running under Linux. The digikam window simply freezes until it has scanned the new directories. It's not noticeable when adding small collections as it only takes a fraction of a second to scan and add them, it's only noticeable when adding a big block of thousands of images. I'm doing this all on an internal disk drive by the way. -- Chris Green · |
Re: digiKam freezes when scanning a new collection
|
|
I have the same problem on a i7 CPU with 16GB Ram on a MacBookPro, straight after a new installation. The collection is said to be reading, but in fact digiKam becomes unresponsive. After a day I had to kill the process. Can't get digiKam to work at all now. On 22/01/2017 12:59, Chris Green wrote: > Gilles Caulier <[hidden email]> wrote: >> 2017-01-22 6:52 GMT+01:00 Dan Dascalescu <[hidden email]>: >> >>> I have a Core i5 CPU with 4 cores and when I add a collection from an >>> external hard drive, digiKam becomes unresponsive. >>> >>> Would it be possible touse a separate thread to scan new collections? >>> >> All is multithreaded in digiKam : scan items and database registration. >> >> If you use a remote database, network latency can be a problem. Didi you >> try to use a local database ? >> > Well I see exactly the same issue when adding a big block of new > directories to digikam. I have a similar CPU and 8Gb RAM with digikam > running under Linux. The digikam window simply freezes until it has > scanned the new directories. > > It's not noticeable when adding small collections as it only takes a > fraction of a second to scan and add them, it's only noticeable when > adding a big block of thousands of images. > > I'm doing this all on an internal disk drive by the way. > > |
Re: digiKam freezes when scanning a new collection
|
|
In reply to this post by Gilles Caulier-4
On Sat, Jan 21, 2017 at 9:58 PM, Gilles Caulier <[hidden email]> wrote:
There is no network involved; the external hard drive is connected via USB. I also have 16GB of RAM (though Chrome eats up most of it). |
Re: digiKam freezes when scanning a new collection
|
|
I think the problem is the registration in DB of non image files and all hidden files. There is patch in bugzilla to introduce a settings to filter unwanted dirs/files while scanning. It's not yet finalized and need to be review. Look here : Gilles Caulier 2017-01-23 2:16 GMT+01:00 Dan Dascalescu <[hidden email]>:
|
Re: digiKam freezes when scanning a new collection
|
|
What makes you think that? This is also an issue when importing a
folder that only contains image files.
Pure speculation on my side: To me it seems that there is something wrong with priority handling (however that is done). Finding new files, which does happen in background, still influences other digikam functionalities to the point where the UI freezes. This looks like the background process takes all resources and any normal operations have to wait, while it should be the other way around: Normal operations (e.g. user interaction with GUI) should always get the resources it needs and background process whatever resources are left. On 23/01/17 10:14, Gilles Caulier
wrote:
|
Re: digiKam freezes when scanning a new collection
|
|
Although I do not see long hangs/freezes when digikam scans new
collections or for new images I do see that in general the process slows down digikam so that the gui either is very very slow or even non-responsive while the process finishes. In my case the directories being scanned contain only image files from my cameras. While the scanning of new collections is not used a huge amount the process looking for new images in existing collections does slow down the initial use of digikam on every start of the program. I am running using an AMD quad core processor with 8meg memory running openSUSE Tumbleweed. Stuart On 23/01/17 10:01, Simon Frei wrote: > What makes you think that? This is also an issue when importing a folder > that only contains image files. > Pure speculation on my side: To me it seems that there is something > wrong with priority handling (however that is done). Finding new files, > which does happen in background, still influences other digikam > functionalities to the point where the UI freezes. This looks like the > background process takes all resources and any normal operations have to > wait, while it should be the other way around: Normal operations (e.g. > user interaction with GUI) should always get the resources it needs and > background process whatever resources are left. > > On 23/01/17 10:14, Gilles Caulier wrote: >> I think the problem is the registration in DB of non image files and >> all hidden files. >> >> There is patch in bugzilla to introduce a settings to filter unwanted >> dirs/files while scanning. It's not yet finalized and need to be >> review. Look here : >> >> https://bugs.kde.org/show_bug.cgi?id=123097 >> >> Gilles Caulier >> > -- Website: http://www.stella-maris.org.uk or: http://www.broadstairs.org |
Re: digiKam freezes when scanning a new collection
|
|
The scan process run already in a separated thread. The registration of data to the database is also delegate to a thread. Database access can introduce time latency with concurrent request. This is true with remote database and network access, as i can see here with my computer. I tested here the scan of a whole collection from scratch with many type of images and videos. More than 30.000 items located in a SSD. Computer is an i7 3Ghz with 32Gb of RAM. There is no visible time latency. I run digiKam from a console and i can see all files scanned step by step very quickly. While this process is done, the progress manager report the advance of scanning as expected. This is done with a sqlite database, also located on a SSD. To identify where is the problem, i recommend to always run digiKam from a console. On MacOS it's simple, as under Linux. For Windows, the use of and extra application to catch console trace as debugview is mandatory. It's explained on web site contrib page : Gilles Caulier 2017-01-23 11:53 GMT+01:00 Stuart T Rogers <[hidden email]>: Although I do not see long hangs/freezes when digikam scans new collections or for new images I do see that in general the process slows down digikam so that the gui either is very very slow or even non-responsive while the process finishes. In my case the directories being scanned contain only image files from my cameras. While the scanning of new collections is not used a huge amount the process looking for new images in existing collections does slow down the initial use of digikam on every start of the program. I am running using an AMD quad core processor with 8meg memory running openSUSE Tumbleweed. |
Re: digiKam freezes when scanning a new collection
|
|
I updated the patch from bug : It's ready for testing with current implementation from git/master. This add new settings in database config panel : Gilles Caulier 2017-01-23 12:41 GMT+01:00 Gilles Caulier <[hidden email]>:
|
Re: digiKam freezes when scanning a new collection
|
|
In reply to this post by Gilles Caulier-4
Running digikam 5.4 from a konsole I see
digikam.dbengine: Database is locked. Waited 0 digikam.dbengine: Database is locked. Waited 250 digikam.dbengine: Database is locked. Waited 500 digikam.dbengine: Database is locked. Waited 750 digikam.dbengine: Database is locked. Waited 1000 digikam.dbengine: Database is locked. Waited 1250 digikam.dbengine: Database is locked. Waited 1500 digikam.dbengine: Database is locked. Waited 1750 digikam.dbengine: Database is locked. Waited 2000 digikam.dbengine: Database is locked. Waited 2250 digikam.dbengine: Database is locked. Waited 2500 digikam.dbengine: Database is locked. Waited 2750 digikam.dbengine: Database is locked. Waited 3000 digikam.dbengine: Database is locked. Waited 3250 digikam.dbengine: Database is locked. Waited 3500 digikam.dbengine: Database is locked. Waited 3750 digikam.dbengine: Database is locked. Waited 4000 digikam.dbengine: Database is locked. Waited 4250 digikam.dbengine: Database is locked. Waited 4500 digikam.dbengine: Database is locked. Waited 4750 on the first run after a boot up of Linux, subsequent starts do not see this but I dont often run digikam more than once on any given day. I should add that my database is the same one I used to use on digikam V4, I have never deliberately recreated it. Because I dont use digikam all the time I guess this database locked is happening whenever I start digikam for the first time after any reboot and it is this which appears to make digikam sluggish for me. Stuart On 23/01/17 11:41, Gilles Caulier wrote: > The scan process run already in a separated thread. > > The registration of data to the database is also delegate to a thread. > > Database access can introduce time latency with concurrent request. This > is true with remote database and network access, as i can see here with > my computer. > > I tested here the scan of a whole collection from scratch with many type > of images and videos. More than 30.000 items located in a SSD. Computer > is an i7 3Ghz with 32Gb of RAM. There is no visible time latency. I run > digiKam from a console and i can see all files scanned step by step very > quickly. While this process is done, the progress manager report the > advance of scanning as expected. This is done with a sqlite database, > also located on a SSD. > > To identify where is the problem, i recommend to always run digiKam from > a console. On MacOS it's simple, as under Linux. For Windows, the use of > and extra application to catch console trace as debugview is mandatory. > It's explained on web site contrib page : > > https://www.digikam.org/contrib > > Gilles Caulier > > 2017-01-23 11:53 GMT+01:00 Stuart T Rogers <[hidden email] > <mailto:[hidden email]>>: > > Although I do not see long hangs/freezes when digikam scans new > collections or for new images I do see that in general the process > slows down digikam so that the gui either is very very slow or even > non-responsive while the process finishes. In my case the > directories being scanned contain only image files from my cameras. > While the scanning of new collections is not used a huge amount the > process looking for new images in existing collections does slow > down the initial use of digikam on every start of the program. I am > running using an AMD quad core processor with 8meg memory running > openSUSE Tumbleweed. > > Stuart > > > On 23/01/17 10:01, Simon Frei wrote: > > What makes you think that? This is also an issue when importing > a folder > that only contains image files. > Pure speculation on my side: To me it seems that there is something > wrong with priority handling (however that is done). Finding new > files, > which does happen in background, still influences other digikam > functionalities to the point where the UI freezes. This looks > like the > background process takes all resources and any normal operations > have to > wait, while it should be the other way around: Normal operations > (e.g. > user interaction with GUI) should always get the resources it > needs and > background process whatever resources are left. > > On 23/01/17 10:14, Gilles Caulier wrote: > > I think the problem is the registration in DB of non image > files and > all hidden files. > > There is patch in bugzilla to introduce a settings to filter > unwanted > dirs/files while scanning. It's not yet finalized and need to be > review. Look here : > > https://bugs.kde.org/show_bug.cgi?id=123097 > <https://bugs.kde.org/show_bug.cgi?id=123097> > > Gilles Caulier > > > > -- > Website: http://www.stella-maris.org.uk > or: http://www.broadstairs.org > > -- Website: http://www.stella-maris.org.uk or: http://www.broadstairs.org |
Re: digiKam freezes when scanning a new collection
|
|
In reply to this post by Simon Frei
On Mon, Jan 23, 2017 at 2:01 AM, Simon Frei <[hidden email]> wrote:
I also saw the freeze when scanning a folder with 98%+ images and maybe 1% videos. I don't have as powerful a machine as Gilles though. My laptop is a Core i5 with 16GB RAM, and maybe 6GB free. |
Re: digiKam freezes when scanning a new collection
|
|
I know that a problem still exist, in some conditions in database interface which not commit data by transaction stage, but only and end of database session. This bloat the memory and if swap is required, this have very bad effect on GUI. I NEVER reproduce the problem here. I sniffed the memory to catch the problem. This never happen. The problem also exist with face detection and recognition, or while finger prints computation for similarity tool. I cannot reproduce, and i don't understand why it appears with some system and not mine. This is true for sqlite and mysql. Gilles Caulier 2017-01-24 21:22 GMT+01:00 Dan Dascalescu <[hidden email]>:
|
Re: digiKam freezes when scanning a new collection
|
|
I've just reproduced the problem while adding a collection from the local hard disk. The collection has 10k+ files, but I do not think that's relevant, because digiKam froze as soon as I closed the dialog for adding the collection. Here's the log from the console: ➜ ~ /c/download/Linux/digikam-5.5.0-01-x86-64.appimage VideoDecoder::Register(..., CUDA) Singleton 0x2022160 created... VideoRenderer::Register(..., OpenGLWindow) Singleton 0x2022770 created... SubtitleProcessor::Register(..., FFmpeg) Singleton 0x201d840 created... AudioDecoder::Register(..., FFmpeg) Singleton 0x2022d80 created... AudioEncoder::Register(..., FFmpeg) Singleton 0x20232b0 created... ImageConverter::Register(..., FFmpeg) Singleton 0x20233c0 created... MediaIO::Register(..., QIODevice) Singleton 0x20229e0 created... MediaIO::Register(..., QFile) AudioOutputBackend::Register(..., null) Singleton 0x2022b70 created... VideoDecoder::Register(..., FFmpeg) VideoEncoder::Register(..., FFmpeg) Singleton 0x2023e20 created... registerRenderers........... VideoRenderer::Register(..., OpenGLWidget) VideoRenderer::Register(..., QGLWidget2) VideoRenderer::Register(..., Widget) VideoRenderer::Register(..., GraphicsItem) digikam.widgets: Breeze icons ressource file found digikam.general: AlbumWatch use QFileSystemWatcher digikam.general: Database Parameters: Type: "QSQLITE" DB Core Name: "/c/picsvids/digikam4.db" DB Thumbs Name: "/c/picsvids/thumbnails-digikam.db" DB Face Name: "/c/picsvids/recognition.db" Connect Options: "" Host Name: "" Host port: -1 Internal Server: false Internal Server Path: "" Internal Server Serv Cmd: "" Internal Server Init Cmd: "" Username: "" Password: "" digikam.dbengine: Loading SQL code from config file "/tmp/.mount_kzvjVa/usr/share/digikam/database/dbconfig.xml" digikam.dbengine: Checking XML version ID => expected: 3 found: 3 digikam.coredb: Core database: running schema update digikam.coredb: Core database: have a structure version 8 digikam.coredb: Core database: makeUpdates 8 to 8 digikam.database: Creating new Location "/picsvids" uuid "volumeid:?uuid=c6ac55edac55d90f" digikam.database: Creating new Location "/picsvids" uuid "volumeid:?uuid=289352fd5a743680" digikam.database: location for "/c/picsvids" is available true digikam.database: location for "" is available false KMemoryInfo: Platform identified : "LINUX" KMemoryInfo: TotalRam: 16729649152 digikam.general: Allowing a cache size of 200 MB digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret = 0 digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret = 0 digikam.thumbsdb: Thumbs database: have a structure version "3" digikam.general: Thumbnails database ready for use digikam.dimg: ("/usr/share/color/icc", "/home/dandv/.local/share/icc") digikam.dimg: No X.org XICC profile installed for screen 0 digikam.general: Camera XML data: "/home/dandv/.local/share/digikam/cameras.xml" digikam.facedb: FaceDB SelectFaceSetting val ret = 0 digikam.facedb: FaceDB SelectFaceSetting val ret = 0 digikam.facedb: Face database: have a structure version "2" digikam.facesengine: Face database ready for use digikam.general: Face PipeLine: add database writer digikam.general: Face PipeLine: add faces trainer digikam.facesengine: Face database ready for use digikam.general: Face PipeLine: add database writer digikam.general: Face PipeLine: add faces trainer digikam.geoiface: "setting backend marble" digikam.general: AudioOutput backends: ("null") digikam.general: Stacked View Mode : 0 digikam.geoiface: "setting backend marble" digikam.general: "browse_album" digikam.general: "browse_tag" digikam.general: "browse_labels" digikam.general: "browse_date" digikam.general: "browse_timeline" digikam.general: "browse_search" digikam.general: "browse_fuzzysearch" digikam.general: "browse_gpssearch" digikam.general: "browse_people" digikam.widgets: Paths to color scheme : ("/tmp/.mount_kzvjVa/usr/share/digikam/colorschemes") digikam.widgets: "" :: "" digikam.widgets: "" :: "" QFSFileEngine::open: No file name specified digikam.dimg: Root lens database dir: "/tmp/.mount_kzvjVa/usr/share/lensfun" digikam.dimg: Load lens database file: "6x6.xml" digikam.dimg: Load lens database file: "actioncams.xml" digikam.dimg: Load lens database file: "compact-canon.xml" digikam.dimg: Load lens database file: "compact-casio.xml" digikam.dimg: Load lens database file: "compact-fujifilm.xml" digikam.dimg: Load lens database file: "compact-kodak.xml" digikam.dimg: Load lens database file: "compact-konica-minolta.xml" digikam.dimg: Load lens database file: "compact-leica.xml" digikam.dimg: Load lens database file: "compact-nikon.xml" digikam.dimg: Load lens database file: "compact-olympus.xml" digikam.dimg: Load lens database file: "compact-panasonic.xml" digikam.dimg: Load lens database file: "compact-pentax.xml" digikam.dimg: Load lens database file: "compact-ricoh.xml" digikam.dimg: Load lens database file: "compact-samsung.xml" digikam.dimg: Load lens database file: "compact-sigma.xml" digikam.dimg: Load lens database file: "compact-sony.xml" digikam.dimg: Load lens database file: "contax.xml" digikam.dimg: Load lens database file: "generic.xml" digikam.dimg: Load lens database file: "mil-canon.xml" digikam.dimg: Load lens database file: "mil-fujifilm.xml" digikam.dimg: Load lens database file: "mil-nikon.xml" digikam.dimg: Load lens database file: "mil-olympus.xml" digikam.dimg: Load lens database file: "mil-panasonic.xml" digikam.dimg: Load lens database file: "mil-pentax.xml" digikam.dimg: Load lens database file: "mil-samsung.xml" digikam.dimg: Load lens database file: "mil-samyang.xml" digikam.dimg: Load lens database file: "mil-sigma.xml" digikam.dimg: Load lens database file: "mil-sony.xml" digikam.dimg: Load lens database file: "mil-tamron.xml" digikam.dimg: Load lens database file: "mil-zeiss.xml" digikam.dimg: Load lens database file: "misc.xml" digikam.dimg: Load lens database file: "rf-leica.xml" digikam.dimg: Load lens database file: "slr-canon.xml" digikam.dimg: Load lens database file: "slr-hasselblad.xml" digikam.dimg: Load lens database file: "slr-konica-minolta.xml" digikam.dimg: Load lens database file: "slr-leica.xml" digikam.dimg: Load lens database file: "slr-nikon.xml" digikam.dimg: Load lens database file: "slr-olympus.xml" digikam.dimg: Load lens database file: "slr-panasonic.xml" digikam.dimg: Load lens database file: "slr-pentax.xml" digikam.dimg: Load lens database file: "slr-ricoh.xml" digikam.dimg: Load lens database file: "slr-samsung.xml" digikam.dimg: Load lens database file: "slr-samyang.xml" digikam.dimg: Load lens database file: "slr-schneider.xml" digikam.dimg: Load lens database file: "slr-sigma.xml" digikam.dimg: Load lens database file: "slr-soligor.xml" digikam.dimg: Load lens database file: "slr-sony.xml" digikam.dimg: Load lens database file: "slr-tamron.xml" digikam.dimg: Load lens database file: "slr-tokina.xml" digikam.dimg: Load lens database file: "slr-ussr.xml" digikam.dimg: Load lens database file: "slr-vivitar.xml" digikam.dimg: Load lens database file: "slr-zeiss.xml" digikam.dimg: variant: QVariant(Digikam::LensFunIface::DevicePtr, ) digikam.dimg: dev: AEE :: AEE DV :: 6 digikam.dimg: Search for camera "" - "" ==> false digikam.dimg: Search for lens "" ==> false digikam.dimg: updating data digikam.dimg: updating data digikam.general: Using 4 CPU core to run threads digikam.general: new search text settings: "" : hasResult = false , validRows = 0 QFSFileEngine::open: No file name specified digikam.geoiface: ---- digikam.general: Added root album called: "picsvids" digikam.general: Added root album called: "HDD picsvids" digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.general: Using 4 CPU core to run threads digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.general: Action Thread run 1 new jobs digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.geoiface: ---- digikam.geoiface: ---- digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.geoiface: ---- digikam.general: Cancel Main Thread digikam.geoiface: ---- digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.geoiface: ---- digikam.general: Cancel Main Thread digikam.geoiface: ---- digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.general: changing to SAlbum 0x3b67e30 digikam.database: " ( (Images.id NOT IN (SELECT imageid FROM ImageTags)) ) " digikam.database: Search query: "SELECT DISTINCT Images.id, Images.name, Images.album, Albums.albumRoot, ImageInformation.rating, Images.category, ImageInformation.format, ImageInformation.creationDate, Images.modificationDate, Images.fileSize, ImageInformation.width, ImageInformation.height, ImagePositions.latitudeNumber, ImagePositions.longitudeNumber FROM Images LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid LEFT JOIN ImageMetadata ON Images.id=ImageMetadata.imageid LEFT JOIN VideoMetadata ON Images.id=VideoMetadata.imageid LEFT JOIN ImagePositions ON Images.id=ImagePositions.imageid INNER JOIN Albums ON Albums.id=Images.album WHERE Images.status=1 AND ( ( (Images.id NOT IN (SELECT imageid FROM ImageTags)) ) );" () digikam.geoiface: "ROADMAP" digikam.geoiface: "ROADMAP" digikam.geoiface: "setting backend marble" digikam.geoiface: "ROADMAP" digikam.geoiface: ---- digikam.general: Cancel Main Thread digikam.geoiface: ---- digikam.database: Search result: 576548 digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.geoiface: "ROADMAP" digikam.geoiface: ---- digikam.general: One job is done digikam.general: One job is done digikam.general: One job is done digikam.general: One job is done digikam.general: One job is done digikam.general: One job is done digikam.geoiface: ---- Recreating ksycoca file ("/home/dandv/.cache/ksycoca5_en_dDLWeWuQRKktFB_jz4jIAAZxO6U=", version 303) Saving kipi.plugins: Plugin_Dropbox Plugin Loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_dropboxui.rc" kipi.library: Loaded plugin "Dropbox" kipi.plugins: Plugin_Imageshack plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_imageshackui.rc" kipi.library: Loaded plugin "Imageshack" kipi.plugins: Plugin_GoogleServices Plugin Loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_googleservicesui.rc" kipi.library: Loaded plugin "Google Services" kipi.plugins: Plugin_Rajce plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_rajceui.rc" kipi.library: Loaded plugin "Rajce" kipi.plugins: Plugin_Smug plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_smugui.rc" kipi.library: Loaded plugin "Smug" kipi.plugins: Plugin_SendImages plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_kmlexportui.rc" kipi.library: Loaded plugin "KMLExport" kipi.plugins: Plugin_Flickr plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_flickrui.rc" kipi.library: Loaded plugin "Flickr" kipi.plugins: Plugin_YandexFotki plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_yandexfotkiui.rc" kipi.library: Loaded plugin "YandexFotki" kipi.plugins: Plugin_Facebook plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_facebookui.rc" kipi.library: Loaded plugin "Facebook" kipi.plugins: Plugin_Piwigo plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_piwigoui.rc" kipi.library: Loaded plugin "Piwigo" kipi.plugins: Plugin_MediaWiki plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_wikimediaui.rc" kipi.library: Loaded plugin "MediaWiki export" kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_flashexportui.rc" kipi.library: Loaded plugin "FlashExport" kipi.plugins: Imgur plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_imgurui.rc" kipi.library: Loaded plugin "Imgur" kipi.plugins: Plugin_SendImages plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_sendimagesui.rc" kipi.library: Loaded plugin "SendImages" kipi.plugins: Plugin_PrintImages plugin loaded kipi.library: UI file : "/tmp/.mount_kzvjVa/usr/share/kxmlgui5/kipi/kipiplugin_printimagesui.rc" kipi.library: Loaded plugin "PrintImages" kipi.plugins: Reading data from kipirc file.. digikam.geoiface: ---- digikam.general: Cancel Main Thread digikam.general: One job is done digikam.general: One job is done QFSFileEngine::open: No file name specified digikam.geoiface: "setting backend marble" digikam.geoiface: "ROADMAP" digikam.facesengine: Face database ready for use digikam.general: Face PipeLine: add database writer digikam.general: Face PipeLine: add faces trainer digikam.facesengine: Face database ready for use digikam.general: Face PipeLine: add database writer digikam.general: Face PipeLine: add faces trainer QFSFileEngine::open: No file name specified digikam.geoiface: "setting backend marble" digikam.geoiface: "ROADMAP" digikam.geoiface: "setting backend marble" digikam.geoiface: "ROADMAP" digikam.general: scan mode: ScanDeferredFiles digikam.general: total scan value : 8502 digikam.dimg: "/c/picsvids/Krystle/India/2017-01-23 fingering.jpg" : JPEG file identified digikam.database: Adding new item "/c/picsvids/Krystle/India/2017-01-23 fingering.jpg" digikam.database: Scanning took 5 ms digikam.database: Finishing took 2 ms digikam.dimg: "/c/picsvids/Krystle/India/2017-01-23 rural 2.jpg" : JPEG file identified digikam.database: Adding new item "/c/picsvids/Krystle/India/2017-01-23 rural 2.jpg" digikam.metaengine: Orientation => Exif.Image.Orientation => 1 digikam.database: Scanning took 3 ms digikam.database: Finishing took 1 ms digikam.dimg: "/c/picsvids/Krystle/India/2017-01-23 rural.jpg" : JPEG file identified digikam.database: Adding new item "/c/picsvids/Krystle/India/2017-01-23 rural.jpg" digikam.metaengine: Orientation => Exif.Image.Orientation => 1 digikam.database: Scanning took 3 ms digikam.database: Finishing took 0 ms digikam.database: items to tag () digikam.database: Complete scan took: 984 msecs. digikam.general: Event is dispatched to desktop notifier through DBUS digikam.dimg: "/c/picsvids/Krystle/India/2017-01-23 fingering.jpg" : JPEG file identified digikam.dimg: "/c/picsvids/Krystle/India/2017-01-23 rural.jpg" : JPEG file identified digikam.metaengine: Orientation => Exif.Image.Orientation => 1 digikam.dimg: "/c/picsvids/Krystle/India/2017-01-23 rural 2.jpg" : JPEG file identified digikam.metaengine: Orientation => Exif.Image.Orientation => 1 digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.general: One job is done digikam.general: Testing "mysql_install_db" ... digikam.general: Testing "mysqld" ... digikam.general: Testing "mysql_install_db" ... digikam.general: Testing "mysqld" ... digikam.general: Testing "././/bin/mysql_install_db" ... digikam.general: All Binaries Found : false digikam.general: Testing "././/bin/mysqld" ... digikam.general: All Binaries Found : false digikam.general: Testing "././/sbin/mysql_install_db" ... digikam.general: All Binaries Found : false digikam.general: Testing "././/sbin/mysqld" ... digikam.general: All Binaries Found : false digikam.metaengine: Cannot get Exif Tags list using Exiv2 (Error # 23 : Invalid ifdId 8 digikam.dimg: "/tmp/.mount_kzvjVa/usr/share/digikam/data/sample-aix.png" : PNG file identified digikam.metaengine: Loading image history "" digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2010-07-27 06:40:05.000 PDT Qt::TimeSpec(LocalTime)) libpng warning: iCCP: known incorrect sRGB profile digikam.general: new search text settings: "" : hasResult = true , validRows = 165 digikam.general: No DB settings changes. Do nothing... digikam.database: addLocation QUrl("file:///c/download/x") digikam.database: Creating new Location "/download/x" uuid "volumeid:?uuid=c6ac55edac55d90f" digikam.database: location for "/c/picsvids" is available true digikam.database: location for "" is available false digikam.database: location for "/c/download/x" is available true digikam.general: Added root album called: "x" digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.general: Using 4 CPU core to run threads digikam.general: Cancel Main Thread digikam.general: Using 4 CPU core to run threads digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.general: Using 4 CPU core to run threads digikam.general: Action Thread run 1 new jobs digikam.general: Action Thread run 1 new jobs digikam.general: Action Thread run 1 new jobs digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: User of DWItemDelegate should not delete widgets created by createItemWidgets! digikam.general: scan mode: CompleteCollectionScan digikam.general: One job is done digikam.general: One job is done digikam.general: One job is done digikam.general: One job is done digikam.general: One job is done digikam.dbengine: Database is locked. Waited 0 digikam.dimg: "/c/download/new/0cb7f4f712178598d73465fe5fb99d0a.jpg" : JPEG file identified digikam.database: Adding new item "/c/download/new/0cb7f4f712178598d73465fe5fb99d0a.jpg" digikam.database: Scanning took 2 ms digikam.database: Finishing took 2 ms digikam.dimg: "/c/download/new/20150103_173008.jpg" : JPEG file identified digikam.database: Adding new item "/c/download/new/20150103_173008.jpg" digikam.database: Recognized "/c/download/new/20150103_173008.jpg" as identical to item 9200 digikam.database: Scanning took 3 ms digikam.database: Finishing took 2 ms digikam.dimg: "/c/download/new/20150103_173027.jpg" : JPEG file identified digikam.database: Adding new item "/c/download/new/20150103_173027.jpg" digikam.database: Recognized "/c/download/new/20150103_173027.jpg" as identical to item 9201 digikam.database: Scanning took 3 ms digikam.database: Finishing took 1 ms ... The log keeps going on and on with the same type of messages. I searched for "error" and most of the errors I see seem related to invalid metadata in some images: digikam.metaengine: Exiv2 ( 3 ) : XMP Toolkit error 203: Duplicate property or field node digikam.metaengine: Exiv2 ( 2 ) : Failed to decode XMP metadata. digikam.metaengine: Cannot load metadata from file (Error # 53 : Not a valid ICC Profile digikam.metaengine: Exiv2 ( 2 ) : Updating namespace URI for prefix0 from MSImagingV1/ to http://xmp.gettyimages.com/gift/1.0/ digikam.metaengine: Exiv2 ( 3 ) : XMP Toolkit error 101: Schema namespace URI and prefix mismatch digikam.metaengine: Exiv2 ( 2 ) : Failed to decode XMP metadata. digikam.metaengine: Exiv2 ( 3 ) : XMP Toolkit error 201: XML parsing failure But also at some point: digikam.database: Scanning took 3 ms digikam.database: Finishing took 1 ms digikam.dbengine: Database is locked. Waited 10000 digikam.dbengine: Detected locked database file. There is an active transaction. Waited but giving up now. digikam.dbengine: Failure executing query: "SELECT value FROM ImageProperties WHERE imageid=? and property=?;" Error messages: "Unable to fetch row" "database table is locked: ImageProperties" 6 1 Bound values: (QVariant(qlonglong, 36539), QVariant(QString, "city")) digikam.dbengine: Database is locked. Waited 0 and some other time: digikam.dbengine: Detected locked database file. There is an active transaction. Waited but giving up now. digikam.dbengine: Failure executing query: "SELECT value FROM ImageProperties WHERE imageid=? and property=?;" Error messages: "Unable to fetch row" "database table is locked: ImageProperties" 6 1 Bound values: (QVariant(qlonglong, 36539), QVariant(QString, "countryCode")) digikam.dbengine: Database is locked. Waited 0 digikam.database: Scanning took 4 ms digikam.database: Finishing took 0 ms Does digiKam search for duplicates while adding each file maybe? ("Recognized as identical to item...") On Tue, Jan 24, 2017 at 12:53 PM, Gilles Caulier <[hidden email]> wrote:
|
Re: digiKam freezes when scanning a new collection
|
|
The Exiv2 error about XMP is not the problem here. This message is the problem : digikam.dbengine: Database is locked. Waited 0 a transaction is in progress and lock the concurrent DB access to other one. While a scanning, there is no reason for that. Even if scan run in a separated thread, it's not (yet) multi-core compliant. So one separated thread to register items in database. One item registered, one quiery in DB. The finger-prints and face detection recognition are not processed while scanning. So there is no other DB access than item registration while scanning. There is no reason to see the DB locked, especially if you use a local DB, as sqlite or mysql internal. For a remote mysql/mariadb, the network can be a problem. What do you use as DB ? Which OS do you use ? Gilles Caulier 2017-01-25 4:33 GMT+01:00 Dan Dascalescu <[hidden email]>:
|
Re: digiKam freezes when scanning a new collection
|
|
Local SQLite database on Ubuntu 16.04.1 64-bit, stored on an NTFS partition if that matters. On Wed, Jan 25, 2017 at 2:05 AM, Gilles Caulier <[hidden email]> wrote:
|
«
Return to digikam-users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |