unresponsive while scanning for new items

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

unresponsive while scanning for new items

Simon Frei
Hi
I do not have enough detailed information to file a bug and something
similar might already be reported. Still I wanted to inform you about
it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam
(metadata writing with exiftool). So it took quite some time searching
for new items after startup (notification on the bottom left), but never
used cpu or ram very hard. After some time however digikam became
unresponsive, just a grey frame. The commandline did show that it was
still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did
load the computer, but not overload. After a certain time digikam
resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Gilles Caulier-4
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Simon Frei
Compiled from software-compilation. Git core is on commit c9f02e0.

On 22/07/16 19:14, Gilles Caulier wrote:
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Gilles Caulier-4
From my experience sqlite use must be limited to 100Gb of images. For more, use Mysql. You can use internal server, no need a remote server. In this case, it will be used as sqlite.

Gilles Caulier

2016-07-22 19:31 GMT+02:00 Simon Frei <[hidden email]>:
Compiled from software-compilation. Git core is on commit c9f02e0.


On 22/07/16 19:14, Gilles Caulier wrote:
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Gilles Caulier-4
The time latency from gui are done certainly by sqlite lock because a lots lots data need to be managed in database.

Did you store the DB file in a fast device as a SSD. This improve also the performance until a limit of course.

In all case, as you use git/master code, Mysql must give better results. We will interested to have a comparison.

Gilles Caulier

2016-07-22 19:35 GMT+02:00 Gilles Caulier <[hidden email]>:
From my experience sqlite use must be limited to 100Gb of images. For more, use Mysql. You can use internal server, no need a remote server. In this case, it will be used as sqlite.

Gilles Caulier

2016-07-22 19:31 GMT+02:00 Simon Frei <[hidden email]>:
Compiled from software-compilation. Git core is on commit c9f02e0.


On 22/07/16 19:14, Gilles Caulier wrote:
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel




_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Simon Frei
Thanks for the information.
We discussed the issue of database location earlier already. In the meantime I locate the database in ram, so access is fast (digikam as a whole is a lot faster now). My issue is not that digikam was slow, it is that it became completely. unresponsive for no apparent reason (no performance limit).
For clarification (I have no knowledge of database programming): Internal mysql uses the same "system" as sqlite and the recent patches discussed were only concerning external mysql?
Trying mysql is on my list, but only after I reorganised my collection. If sqlite really proves to be unusable with lots of data, I may switch earlier.

On 22/07/16 19:38, Gilles Caulier wrote:
The time latency from gui are done certainly by sqlite lock because a lots lots data need to be managed in database.

Did you store the DB file in a fast device as a SSD. This improve also the performance until a limit of course.

In all case, as you use git/master code, Mysql must give better results. We will interested to have a comparison.

Gilles Caulier

2016-07-22 19:35 GMT+02:00 Gilles Caulier <[hidden email]>:
From my experience sqlite use must be limited to 100Gb of images. For more, use Mysql. You can use internal server, no need a remote server. In this case, it will be used as sqlite.

Gilles Caulier

2016-07-22 19:31 GMT+02:00 Simon Frei <[hidden email]>:
Compiled from software-compilation. Git core is on commit c9f02e0.


On 22/07/16 19:14, Gilles Caulier wrote:
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel





_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Gilles Caulier-4


2016-07-22 21:19 GMT+02:00 Simon Frei <[hidden email]>:
Thanks for the information.
We discussed the issue of database location earlier already. In the meantime I locate the database in ram, so access is fast (digikam as a whole is a lot faster now). My issue is not that digikam was slow, it is that it became completely. unresponsive for no apparent reason (no performance limit).
For clarification (I have no knowledge of database programming): Internal mysql uses the same "system" as sqlite and the recent patches discussed were only concerning external mysql?

yes there are. sqlite still unchanged.

Mysql database is more designed to work with huge data, not sqlite. This is why this kind of DB exists.
Don't forget the low level interface from Qt which is different slightly different than sqlite.

Gilles Caulier
 
Trying mysql is on my list, but only after I reorganised my collection. If sqlite really proves to be unusable with lots of data, I may switch earlier.


On 22/07/16 19:38, Gilles Caulier wrote:
The time latency from gui are done certainly by sqlite lock because a lots lots data need to be managed in database.

Did you store the DB file in a fast device as a SSD. This improve also the performance until a limit of course.

In all case, as you use git/master code, Mysql must give better results. We will interested to have a comparison.

Gilles Caulier

2016-07-22 19:35 GMT+02:00 Gilles Caulier <[hidden email]>:
From my experience sqlite use must be limited to 100Gb of images. For more, use Mysql. You can use internal server, no need a remote server. In this case, it will be used as sqlite.

Gilles Caulier

2016-07-22 19:31 GMT+02:00 Simon Frei <[hidden email][hidden email]>:
Compiled from software-compilation. Git core is on commit c9f02e0.


On 22/07/16 19:14, Gilles Caulier wrote:
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email][hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel





_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Simon Frei
I wanted to give you an additional piece of information: I just had the same issue again and could test what file is accessed:
The unresponsivness (grey window) occurs while digikam is reading (sic!) from thumbnails-digikam.db.

On 22/07/16 22:03, Gilles Caulier wrote:


2016-07-22 21:19 GMT+02:00 Simon Frei <[hidden email]>:
Thanks for the information.
We discussed the issue of database location earlier already. In the meantime I locate the database in ram, so access is fast (digikam as a whole is a lot faster now). My issue is not that digikam was slow, it is that it became completely. unresponsive for no apparent reason (no performance limit).
For clarification (I have no knowledge of database programming): Internal mysql uses the same "system" as sqlite and the recent patches discussed were only concerning external mysql?

yes there are. sqlite still unchanged.

Mysql database is more designed to work with huge data, not sqlite. This is why this kind of DB exists.
Don't forget the low level interface from Qt which is different slightly different than sqlite.

Gilles Caulier
 
Trying mysql is on my list, but only after I reorganised my collection. If sqlite really proves to be unusable with lots of data, I may switch earlier.


On 22/07/16 19:38, Gilles Caulier wrote:
The time latency from gui are done certainly by sqlite lock because a lots lots data need to be managed in database.

Did you store the DB file in a fast device as a SSD. This improve also the performance until a limit of course.

In all case, as you use git/master code, Mysql must give better results. We will interested to have a comparison.

Gilles Caulier

2016-07-22 19:35 GMT+02:00 Gilles Caulier <[hidden email]>:
From my experience sqlite use must be limited to 100Gb of images. For more, use Mysql. You can use internal server, no need a remote server. In this case, it will be used as sqlite.

Gilles Caulier

2016-07-22 19:31 GMT+02:00 Simon Frei <[hidden email]>:
Compiled from software-compilation. Git core is on commit c9f02e0.


On 22/07/16 19:14, Gilles Caulier wrote:
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel





_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel




_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Gilles Caulier-4
the thumb DB can growing up very quickly. Do you have enough free space on your device ?

Gilles Caulier

2016-07-25 23:32 GMT+02:00 Simon Frei <[hidden email]>:
I wanted to give you an additional piece of information: I just had the same issue again and could test what file is accessed:
The unresponsivness (grey window) occurs while digikam is reading (sic!) from thumbnails-digikam.db.


On 22/07/16 22:03, Gilles Caulier wrote:


2016-07-22 21:19 GMT+02:00 Simon Frei <[hidden email]>:
Thanks for the information.
We discussed the issue of database location earlier already. In the meantime I locate the database in ram, so access is fast (digikam as a whole is a lot faster now). My issue is not that digikam was slow, it is that it became completely. unresponsive for no apparent reason (no performance limit).
For clarification (I have no knowledge of database programming): Internal mysql uses the same "system" as sqlite and the recent patches discussed were only concerning external mysql?

yes there are. sqlite still unchanged.

Mysql database is more designed to work with huge data, not sqlite. This is why this kind of DB exists.
Don't forget the low level interface from Qt which is different slightly different than sqlite.

Gilles Caulier
 
Trying mysql is on my list, but only after I reorganised my collection. If sqlite really proves to be unusable with lots of data, I may switch earlier.


On 22/07/16 19:38, Gilles Caulier wrote:
The time latency from gui are done certainly by sqlite lock because a lots lots data need to be managed in database.

Did you store the DB file in a fast device as a SSD. This improve also the performance until a limit of course.

In all case, as you use git/master code, Mysql must give better results. We will interested to have a comparison.

Gilles Caulier

2016-07-22 19:35 GMT+02:00 Gilles Caulier <[hidden email][hidden email]>:
From my experience sqlite use must be limited to 100Gb of images. For more, use Mysql. You can use internal server, no need a remote server. In this case, it will be used as sqlite.

Gilles Caulier

2016-07-22 19:31 GMT+02:00 Simon Frei <[hidden email][hidden email]>:
Compiled from software-compilation. Git core is on commit c9f02e0.


On 22/07/16 19:14, Gilles Caulier wrote:
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email][hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email][hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel





_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel




_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Simon Frei
Yes that is not a problem. And the weird thing is it is not even writing to it, but reading.

On 25/07/16 23:38, Gilles Caulier wrote:
the thumb DB can growing up very quickly. Do you have enough free space on your device ?

Gilles Caulier

2016-07-25 23:32 GMT+02:00 Simon Frei <[hidden email]>:
I wanted to give you an additional piece of information: I just had the same issue again and could test what file is accessed:
The unresponsivness (grey window) occurs while digikam is reading (sic!) from thumbnails-digikam.db.


On 22/07/16 22:03, Gilles Caulier wrote:


2016-07-22 21:19 GMT+02:00 Simon Frei <[hidden email]>:
Thanks for the information.
We discussed the issue of database location earlier already. In the meantime I locate the database in ram, so access is fast (digikam as a whole is a lot faster now). My issue is not that digikam was slow, it is that it became completely. unresponsive for no apparent reason (no performance limit).
For clarification (I have no knowledge of database programming): Internal mysql uses the same "system" as sqlite and the recent patches discussed were only concerning external mysql?

yes there are. sqlite still unchanged.

Mysql database is more designed to work with huge data, not sqlite. This is why this kind of DB exists.
Don't forget the low level interface from Qt which is different slightly different than sqlite.

Gilles Caulier
 
Trying mysql is on my list, but only after I reorganised my collection. If sqlite really proves to be unusable with lots of data, I may switch earlier.


On 22/07/16 19:38, Gilles Caulier wrote:
The time latency from gui are done certainly by sqlite lock because a lots lots data need to be managed in database.

Did you store the DB file in a fast device as a SSD. This improve also the performance until a limit of course.

In all case, as you use git/master code, Mysql must give better results. We will interested to have a comparison.

Gilles Caulier

2016-07-22 19:35 GMT+02:00 Gilles Caulier <[hidden email]>:
From my experience sqlite use must be limited to 100Gb of images. For more, use Mysql. You can use internal server, no need a remote server. In this case, it will be used as sqlite.

Gilles Caulier

2016-07-22 19:31 GMT+02:00 Simon Frei <[hidden email]>:
Compiled from software-compilation. Git core is on commit c9f02e0.


On 22/07/16 19:14, Gilles Caulier wrote:
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel





_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel




_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel




_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: unresponsive while scanning for new items

Gilles Caulier-4
The thumb database file store wavelets compressed image (thumbnails) in a table. If file is corrupted for a specific reason, you can remove it, and it will be recreated automatically. There is no extra properties stored in this database.

I just take time to recreate it. That all.

Try to rename this file in the way tp force re-creation, and look the result.

Gilles Caulier

2016-07-25 23:47 GMT+02:00 Simon Frei <[hidden email]>:
Yes that is not a problem. And the weird thing is it is not even writing to it, but reading.


On 25/07/16 23:38, Gilles Caulier wrote:
the thumb DB can growing up very quickly. Do you have enough free space on your device ?

Gilles Caulier

2016-07-25 23:32 GMT+02:00 Simon Frei <[hidden email]>:
I wanted to give you an additional piece of information: I just had the same issue again and could test what file is accessed:
The unresponsivness (grey window) occurs while digikam is reading (sic!) from thumbnails-digikam.db.


On 22/07/16 22:03, Gilles Caulier wrote:


2016-07-22 21:19 GMT+02:00 Simon Frei <[hidden email][hidden email]>:
Thanks for the information.
We discussed the issue of database location earlier already. In the meantime I locate the database in ram, so access is fast (digikam as a whole is a lot faster now). My issue is not that digikam was slow, it is that it became completely. unresponsive for no apparent reason (no performance limit).
For clarification (I have no knowledge of database programming): Internal mysql uses the same "system" as sqlite and the recent patches discussed were only concerning external mysql?

yes there are. sqlite still unchanged.

Mysql database is more designed to work with huge data, not sqlite. This is why this kind of DB exists.
Don't forget the low level interface from Qt which is different slightly different than sqlite.

Gilles Caulier
 
Trying mysql is on my list, but only after I reorganised my collection. If sqlite really proves to be unusable with lots of data, I may switch earlier.


On 22/07/16 19:38, Gilles Caulier wrote:
The time latency from gui are done certainly by sqlite lock because a lots lots data need to be managed in database.

Did you store the DB file in a fast device as a SSD. This improve also the performance until a limit of course.

In all case, as you use git/master code, Mysql must give better results. We will interested to have a comparison.

Gilles Caulier

2016-07-22 19:35 GMT+02:00 Gilles Caulier <[hidden email][hidden email]>:
From my experience sqlite use must be limited to 100Gb of images. For more, use Mysql. You can use internal server, no need a remote server. In this case, it will be used as sqlite.

Gilles Caulier

2016-07-22 19:31 GMT+02:00 Simon Frei <[hidden email][hidden email]>:
Compiled from software-compilation. Git core is on commit c9f02e0.


On 22/07/16 19:14, Gilles Caulier wrote:
Which digiKam version do you use ?

Gilles Caulier

2016-07-22 17:32 GMT+02:00 Simon Frei <[hidden email][hidden email]>:
Hi
I do not have enough detailed information to file a bug and something similar might already be reported. Still I wanted to inform you about it. Maybe it helps in some way.
I just made changes to a big set of images (350G) outside of digikam (metadata writing with exiftool). So it took quite some time searching for new items after startup (notification on the bottom left), but never used cpu or ram very hard. After some time however digikam became unresponsive, just a grey frame. The commandline did show that it was still working in the background, the following was displayed repeatedly:
digikam.dimg: "*somefile*"  : JPEG file identified
digikam.metaengine: Orientation => Exif.Image.Orientation => 1
So digikam was still functioning, just not the gui and still digikam did load the computer, but not overload. After a certain time digikam resurfaced and worked fine.
Cheers,
Simon
_______________________________________________
Digikam-devel mailing list
[hidden email][hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email][hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel





_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel




_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel




_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel


_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel



_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel