All albums are removed if there is a network disconnection
Locked
12
12
 
|
Hello,
I am using Digikam 5.8.0 in Windows 10. After (painfully) scanning around 100,000 pictures in a network share via wifi, I suffered a momentarily disconnection (less than a minute), and Digikam, which was open at that moment, deleted all albums from my collections. Now it is scanning again from the beginning, which will take a few days. I browsed the SQLite database file to confirm that albums were indeed empty (although filenames appeared in the Image database). Shouldn't be there some protection when a collection in a network share is temporally unavailable, so it doesn't automatically remove everything from the database? Like marking the album as "unavailable" or greyed out, but still allowing to see the thumbnails. Of course, if a specific folder no longer exists (but the network share is accessible) it makes sense to remove it from the collection, but if the entire collection is not available it is likely due to a network error, and shouldn't be emptied right away in my opinion. Thanks in advance. -- Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html |
Re: All albums are removed if there is a network disconnection
|
|
On mardi 30 janvier 2018 12:42:13 CET woenx wrote:
> Hello, > > I am using Digikam 5.8.0 in Windows 10. After (painfully) scanning around > 100,000 pictures in a network share via wifi, I suffered a momentarily > disconnection (less than a minute), and Digikam, which was open at that > moment, deleted all albums from my collections. Now it is scanning again > from the beginning, which will take a few days. > > I browsed the SQLite database file to confirm that albums were indeed empty > (although filenames appeared in the Image database). > > Shouldn't be there some protection when a collection in a network share is > temporally unavailable, so it doesn't automatically remove everything from > the database? Like marking the album as "unavailable" or greyed out, but > still allowing to see the thumbnails. Of course, if a specific folder no > longer exists (but the network share is accessible) it makes sense to remove > it from the collection, but if the entire collection is not available it is > likely due to a network error, and shouldn't be emptied right away in my > opinion. You did declare your collections as residing on network shares, didn't you? The first option is only for collections on your local hard disk(s); which should not go off-line. Remco |
Re: All albums are removed if there is a network disconnection
|
|
On 30.01.2018 14:55, Remco Viëtor wrote:
> The first option is only for collections on your local hard disk(s); which > should not go off-line. > Even on local hard disks collections can go off line - in case of a defect or simply a problem while mounting. Sorry, the collection should never ever be emptied without user interaction confirming the action, not now, not ever! regards Karl Günter |
|
|
In reply to this post by Remco Viëtor
Yes, of course, they were added as network shares (the third option). The other two options were collections in the local hard drive and collections in external drives, and I didn't use any of these two.
|
|
|
In reply to this post by Karl Günter Wünsch
I am wondering if the new database cleanup tool does that... When you started digikam did you notice how long did database cleanup take? There is usually a notification pops up in the corner. It usually takes a second or two. I am assuming the operation would take way longer if it was to remove thousands of pictures from database Sent from my Samsung Galaxy smartphone. -------- Original message -------- From: Karl Günter Wünsch <[hidden email]> Date: 2018-01-30 7:01 AM (GMT-07:00) To: [hidden email] Subject: Re: All albums are remo > The first option is only for collections on your local hard disk(s); which > should not go off-line. > Even on local hard disks collections can go off line - in case of a defect or simply a problem while mounting. Sorry, the collection should never ever be emptied without user interaction confirming the action, not now, not ever! regards Karl Günter |
|
|
Digikam was open and scanning a few pending folders when that happened. Usually when I start the program, the cleanup is very fast (one second or so), but i didn't see any notificacions when the collections were emptied as Digikam was minimized
at that moment.
El dia 30 gen. 2018 3:04 p. m., Andrey Goreev <[hidden email]> va escriure:
|
|
|
It is hard to see what exactly happened without the terminal output. I believe you are on Windows, Am I correct? I am not sure if there is a way to get digikam terminal output on Windows... Best regards, On Tue, Jan 30, 2018 at 7:13 AM, Marc Palaus <[hidden email]> wrote:
|
Re: All albums are removed if there is a network disconnection
|
|
In reply to this post by Karl Günter Wünsch
On mardi 30 janvier 2018 15:01:00 CET Karl Günter Wünsch wrote:
> On 30.01.2018 14:55, Remco Viëtor wrote: > > The first option is only for collections on your local hard disk(s); which > > should not go off-line. > > Even on local hard disks collections can go off line - in case of a > defect or simply a problem while mounting. Sorry, the collection should > never ever be emptied without user interaction confirming the action, > not now, not ever! True, at least in theory, for defective disks. I have no idea what digikam would do in that case. But... In case of a problem mounting the disk, your collection is on a /removable/ medium, and /you should declare it as such/. (a problem mounting a local disk would have shown up at system start, long before digikam is started). In this case the collections were on a network share, and that also should have been declared in setup. For both network shares and removable media, you tell digikam "this collection is not necessarily always present", whereas for a local disk, you basically say "this collection will always be available". At least for local collections, digikam is notified of changes through external programs (e.g. editors like darktable), and updates the database on the fly.In other words: a file disappears from disk => it is removed from the database. Also, on reflection, where does OP store the database? (and is it SQLite, or MySQL?). At least in the version I use (5.5.0), SQLite is the default, and it is explicitly NOT supported on network shares. Remco |
|
|
In reply to this post by woenx
On mardi 30 janvier 2018 15:13:25 CET Marc Palaus wrote:
> Digikam was open and scanning a few pending folders when that happened. > Usually when I start the program, the cleanup is very fast (one second or > so), but i didn't see any notificacions when the collections were emptied > as Digikam was minimized at that moment. Could you tell us the following: - what database options do you use and where do you store the database - how did you define your collections (i.e. local, removable medium, network share)? Until we know at least that much, all we can do is guess... |
Re: All albums are removed if there is a network disconnection
|
|
In reply to this post by Remco Viëtor
On 30.01.2018 16:12, Remco Viëtor wrote:
> At least for local collections, digikam is notified of changes through external > programs (e.g. editors like darktable), and updates the database on the fly.In > other words: a file disappears from disk => it is removed from the database. IMHO that is a horrible mistake to make. The user loses information about the file because it may be unavailable just for a short moment or because of an honest mistake. What if he restores the file? You have effectively lost the information in the database because of that stupid behavior... -- regards Karl Günter |
|
|
Yes, I am using windows 10 (64 bit). I have another laptop with Ubuntu, so I
could try as well if a console output is required. I store the database using SQLite, I just used the default option during the installation. The database is stored locally at "C:\Users\myusername\Pictures". Pictures are in a network share (using samba in a Debian server), but the database is in the local hard drive. The options to explore the folders at startup and to clean the database are activated in the configuration menu. I just replicated this behavior. Digikam was scanning new folders (in three network shared collections), I deactivated the wifi network (the only one) and after 10-15 seconds, all albums disappeared from Digikam, without showing any notification. I will try now with my Ubuntu computer and with a console output. I'll post the results later. If you need me to run some additional tests, I can just backup my current progress and do it. -- Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html |
Re: All albums are removed if there is a network disconnection
|
|
You can also try turning off the database cleanup option if possible and try again Sent from my Samsung Galaxy smartphone. -------- Original message -------- From: woenx <[hidden email]> Date: 2018-01-30 9:37 AM (GMT-07:00) To: [hidden email] Subject: Re: All albums are removed if there is a network disconnection could try as well if a console output is required. I store the database using SQLite, I just used the default option during the installation. The database is stored locally at "C:\Users\myusername\Pictures". Pictures are in a network share (using samba in a Debian server), but the database is in the local hard drive. The options to explore the folders at startup and to clean the database are activated in the configuration menu. I just replicated this behavior. Digikam was scanning new folders (in three network shared collections), I deactivated the wifi network (the only one) and after 10-15 seconds, all albums disappeared from Digikam, without showing any notification. I will try now with my Ubuntu computer and with a console output. I'll post the results later. If you need me to run some additional tests, I can just backup my current progress and do it. -- Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html |
Re: All albums are removed if there is a network disconnection
|
|
In reply to this post by Karl Günter Wünsch
On mardi 30 janvier 2018 16:58:38 CET Karl Günter Wünsch wrote:
> On 30.01.2018 16:12, Remco Viëtor wrote: > > At least for local collections, digikam is notified of changes through > > external programs (e.g. editors like darktable), and updates the database > > on the fly.In other words: a file disappears from disk => it is removed > > from the database. > IMHO that is a horrible mistake to make. The user loses information > about the file because it may be unavailable just for a short moment or > because of an honest mistake. What if he restores the file? You have > effectively lost the information in the database because of that stupid > behavior... Unavailable through an honest mistake, possible, that's why most OSes nowaways have a trashcan system (and digikam implements its own, as it has to work across OSes). On restore, the data is reread, including sidecars (if it is removed from the database on moving the files to thrash). But a local file *cannot* be "unavailable just for a short moment", contrary to files on a network share (esp. WiFi) or removable media. And to avoid all misunderstanding, I'm talking from my experience (under Linux) and not in any way connected with the Digikam maintainers/coders. Remco |
|
|
In reply to this post by AndriusWild
On windows: I disabled the database cleanup option, restarted Digikam, and
disconnected the wifi network. Same result, all albums just disappeared in a few seconds. I also disabled the option to scan at startup, with same outcome. These two options do not make a difference in this behavior. On my linux machine, running ubuntu 16.043 LTS and Digikam 5.8.0 (using the appimage file, since the PPA is not updated), with the same conditions (source as shared folder, explore and clean database at startup, and SQLite db stored locally at /home/username/Images/), I waited until it started scanning new folders... I disconnected the wifi... and nothing. Nothing happened, the albums are still there, unaffected. If I turn on the wifi again, the scan continues, with no error messages whatsoever. So it seems this bug only affects the windows version. PS: Btw, one bug I found in the linux release: the font size option is not applied until you enter and exit the configuration menu in digikam, and needs to be done every time, which is annoying. -- Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html |
Re: All albums are removed if there is a network disconnection
|
|
In reply to this post by Remco Viëtor
Remco, You statement has a point however I am not sure if you can bring information from sidecars back to the database.https://bugs.kde.org/show_bug.cgi?id=389619 Best regards, On Tue, Jan 30, 2018 at 10:11 AM, Remco Viëtor <[hidden email]> wrote: On mardi 30 janvier 2018 16:58:38 CET Karl Günter Wünsch wrote: |
Re: All albums are removed if there is a network disconnection
|
|
In reply to this post by woenx
Woenx, I suggest you to open relevant bug reports in bugzilla (one for the network collections issue with digikam on windows, one for the font size changing issue on linux)Thank you for taking the time to investigate on this. In the meantime, as a workaround you might want to consider creating a virtual machine with some light linux distro installed to be able to use digikam from your windows laptop/desktop. I have been doing so for a couple of years before switched to linux completely. I used this distro: https://sourceforge.net/projects/runtu/files/runtu%2016.04/XFCE/runtu-xfce-16.04.3-amd64_20170806.iso/download I chose it because 1. I have known it for years 2. It is light (679Mb) 3. It is rock solid and just works. Best regards, Andrey On Tue, Jan 30, 2018 at 10:24 AM, woenx <[hidden email]> wrote: On windows: I disabled the database cleanup option, restarted Digikam, and |
|
|
While submitting the bug, I found one
<https://bugs.kde.org/show_bug.cgi?id=377849> which already describes this problem. I'll just add a comment on it and link to this thread. I'll also report the other bug in linux. Thanks! -- Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html |
Re: All albums are removed if there is a network disconnection
|
|
In reply to this post by AndriusWild
On mardi 30 janvier 2018 18:26:38 CET Andrey Goreev wrote:
> Remco, > > You statement has a point however I am not sure if you can bring > information from sidecars back to the database. > I just opened a bug report: > https://bugs.kde.org/show_bug.cgi?id=389619 > > Long story short, digikam was not able to read information from a sidecar > and write it back to the database. > I tested on an mp4 file. I think it should be relevant to any format exiv2 > can't read metadata from including raw images. > Please try to reproduce it if you have a chance. > > Best regards, restored on resoring a raw file /even if I do not restore the sidecar file/. That is, I recovered caption, title, rating, pick label, colour label and tags. That is with rescan and database cleanup on start-up enabled. That is with digikam 5.5.0 appimage under OpenSuse Leap 42.3 (some of the reported bugs on newer versions made me a bit hesitant to update). So it looks like digikam does not remove 'deleted' items from the database while they are in the Trash. I'll do some more tests later to see what exactly happens. Remco |
Re: All albums are removed if there is a network disconnection
|
|
In reply to this post by Remco Viëtor
On 30.01.18 18:11, Remco Viëtor wrote:
> But a local file *cannot* be "unavailable just for a short moment", And here I beg to differ. What if I move the files to another mounted drive? Still available but no longer in the place they were before. And IIRC some information aren't stored in the sidecar files but rather in the almost completely inaccessible (and ignored by most backup solutions as well as many tools implementing file system operations) extended attributes as the semantic desktop mandates, it's a pain to try and keep those information with the original files if at all possible (the networked drives don't support extended attributes and neither does the NTFS driver (even though that concept is available there too)... That mindset you just expressed has driven me away from digikam (after I ran into similar issues years ago - I honestly thought they had been addressed) and into the arms of the subscription of a commercial DAM (along with a switch to first Windows and then after a while to MacOS) - because they (unlike digikam) have embraced the concept that any information in the DAM should persist until such a time when the user actively chooses to remove it. There, even if I remove a file, move it to another drive, move it to a NAS or anything else - yes it will lose original file but it will show me that that connection has been broken and I can go ahead and reestablish it - I will not lose any information stored in the database. Why can't digikam do the same thing? If the file goes missing, amend the still existing preview with an icon telling the user that the original has vanished. If he then wants to do something with the original prompt him with the file dialog so that he can locate the original and thus regain access to it. If a whole folder has gone missing (by for example renaming to reflect the location) then locating one file would automatically reestablish database connection for the rest. That way digikam would be much more robust, it just doesn't bode well if the database information are still lost so easily... |
Re: All albums are removed if there is a network disconnection
|
|
In reply to this post by Remco Viëtor
On 30.01.18 19:23, Remco Viëtor wrote:
> On mardi 30 janvier 2018 18:26:38 CET Andrey Goreev wrote: > Very quick test using the digikam trash, shows that (afaik) all metadata are > restored on resoring a raw file /even if I do not restore the sidecar file/. > That is, I recovered caption, title, rating, pick label, colour label and > tags. That is with rescan and database cleanup on start-up enabled. > > That is with digikam 5.5.0 appimage under OpenSuse Leap 42.3 (some of the > reported bugs on newer versions made me a bit hesitant to update). > > So it looks like digikam does not remove 'deleted' items from the database > while they are in the Trash. I'll do some more tests later to see what exactly > happens. within digikam that's fine. But often people forget that certain folders must not be accessed outside the DAM - and then they get royally screwed... |
«
Return to digikam-users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |