Missing thumbnails, maybe digikam4.db involved

[userfriendly]

Hello,

i'm using digikam for a decade (!) now. I really like it :-)

My current version here is 5.2 from openSuse Leap 42.2.
I've also used 4.14 on another box (Leap 42.1) to see if the problem described below exists also (it does).

Problem description:
--------------------

Unfortunately it seems that during a big tagging session (and maybe due to a fast shutdown of my laptop after closing digikam) the database is messed up something.

- All Images are stored in folder like: "images\year\year-month" or "images\year\year-month_special event", e.g.: "images\2015\2015-3_trip_to_grandparents"
- The digikam4.db, recognition.db and thumbnail.db are stored at "images\"
- I store the tags also in the files.
- Now the folders after "/images/2016/2016-02" won't show all thumbnails in digikam. E.g. "/images/2016/2016-03" contains 373 images but only 21 of them are shown.
- The images are perfectly viewable with dolphin, showfoto, gwenview,... They even show the correct tags.
- I've also checked the integrity and optimized the db (https://userbase.kde.org/Digikam/Check_Databas). No errors have been reported.
                sqlite3 -line digikam4.db 'pragma integrity_check;' -> "ok"
                sqlite3 -line thumbnails-digikam.db 'pragma integrity_check;' > "ok"
- I tried to delete the tumbnails.db and used the maintenance tools of digikam to recreate fingerprints, thumbnails and looking for new images. No change.
- I tried to recreate a folder:
        - I made a new folder "/images/2016/2016-3_tripNEW" and imported images fromm a usb-stick.
        - Now every single image is visible
        - I tag the images and close digikam
        - I reopen digikam and I see all the thumbnails for a second or so while digikam is looking for new images. BUT directly after that most thumbnails disappear!
        - This results in the same missing (and the same few showing) images in both folders: the old "/images/2016/2016-3_trip" and "/images/2016/2016-3_tripNEW"
        - I dosen't help to delete "/images/2016/2016-3_trip" before starting with "/images/2016/2016-3_tripNEW"
        - So there is some kind of background-process recognising the old and new images as the same images (same fingerprints? hash in database?) and than running in the same issue of not displaying most of them (but not all of them).
       
       
I am somewhat frustrated now. What can I do?
Unfortunately I do not have a recent backup of the digikam4.db. I would loose a lot of hard tagging-work.

What to do now?
---------------

Fortunately (?) the problem starts with images starting at "/images/2016/2016-03". Older folders seem to be (still) correct. I've checked this quite extensively.

Is there a way to delete all references in the database starting from that folder?
Or even more rigorous: Delete every entry (tags, fingerprints, hash,...) containing images in the db with references starting from "/images/2016"?

How to achieve this?
While I know that should be doeable using spl here, I've no idea how to do it, exept of opening the db in e.g. sqlbrowser.
If this works out I will only have to re-tag the last year. And if all references from 2016 are gone (in the database) there should be no 'automated loss' of thumbnails anymore (?)

Do you have any other ideas to solve this issue?

Thanks, Andreas

[ShutterBug]

I recently encountered this problem, as well, using DigiKam 4.12 on Ubuntu
16.04.3.  Just for fun, I tried this (and it worked!):

1. Close DigiKam.
2. Open the file browser (Nautilus) and navigate to the folder that contains
the image files that DigiKam no longer displays thumbnails for.  Example:
/Pictures/1973Pix
3. Open another file browser window and navigate to the folder above the
problem folder. Example:  /Pictures
4. Create another folder and move the files from the problem folder to the
new folder.  
Example:  
$mkdir 1973Pix-a
$cd 1973Pix-a
$mv /Pictures/1973Pix/* .
(Of course, you can do this all in Nautilus)
5. Launch DigiKam and let it scan the target Collection for new or deleted
files.  Look (in DigiKam) under the new Album folder.  You should now see
the missing thumbnail images, complete with any tag info you've added
previously!
Optional:  I wanted to retain the original folder names.  So, I reversed the
process:
6. Close DigiKam.
7. Move the files from the new folder back to the original folder (Example:
1973Pix-a  > 1973Pix)
8. Delete the empty new folder (Example: $rm 1973Pix-a)
9. Launch DigiKam and wait for the rescan to complete.

I haven't tried this yet, but suspect that the entire target Collection root
folder could be renamed, the target Collection reset in the DigiKam
configuration settings, scanned by DigiKam, exit DigiKam, folder renamed
back, again reset the Collection target, launch DigiKam and allow it to
complete its scan.  I suspect this would catch any and all missing
thumbnails.

Caveat:
**Always keep a backup of your image files, just in case something goes
wrong!**


Hope this helps someone...DigiKam's a really great application!





--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Remco Viëtor]

On vendredi 1 décembre 2017 22:43:14 CET ShutterBug wrote:

> I recently encountered this problem, as well, using DigiKam 4.12 on Ubuntu
> 16.04.3.  Just for fun, I tried this (and it worked!):
>
> 1. Close DigiKam.
> 2. Open the file browser (Nautilus) and navigate to the folder that contains
> the image files that DigiKam no longer displays thumbnails for.  Example:
> /Pictures/1973Pix
> 3. Open another file browser window and navigate to the folder above the
> problem folder. Example:  /Pictures
> 4. Create another folder and move the files from the problem folder to the
> new folder.
> Example:
> $mkdir 1973Pix-a
> $cd 1973Pix-a
> $mv /Pictures/1973Pix/* .
> (Of course, you can do this all in Nautilus)
> 5. Launch DigiKam and let it scan the target Collection for new or deleted
> files.  Look (in DigiKam) under the new Album folder.  You should now see
> the missing thumbnail images, complete with any tag info you've added
> previously!
> Optional:  I wanted to retain the original folder names.  So, I reversed the
> process:
> 6. Close DigiKam.
> 7. Move the files from the new folder back to the original folder (Example:
> 1973Pix-a  > 1973Pix)
> 8. Delete the empty new folder (Example: $rm 1973Pix-a)
> 9. Launch DigiKam and wait for the rescan to complete.
>
> I haven't tried this yet, but suspect that the entire target Collection root
> folder could be renamed, the target Collection reset in the DigiKam
> configuration settings, scanned by DigiKam, exit DigiKam, folder renamed
> back, again reset the Collection target, launch DigiKam and allow it to
> complete its scan.  I suspect this would catch any and all missing
> thumbnails.
>
> Caveat:
> **Always keep a backup of your image files, just in case something goes
> wrong!**
>
>
> Hope this helps someone...DigiKam's a really great application!

Iirc, already for that version of Digikam (>2 years old), you could force
Digikam to refresh all thumbnails by deleting the thumbnails-digikam.db
(or safer, rename it/move it). Perhaps not worthwhile for a few missing
thumbs, but probably safer/faster than moving the whole collection around
(which implies *two* scans of the whole collection, tags included).

Also, if you go the way of moving files around outside Digikam, be careful with
sidecars: forget one, and the tagging for the corresponding image could be
lost (esp. for raw files, depending on config settings, other formats could also
be affected).

Remco

P.S. Is it known why some thumbnails disappear, or are not generated?

[ShutterBug]

Yes, I actually did try deleting thumbnails-digikam.db with a subsequent
re-scan, hoping that would solve the original problem of the missing
thumbnails.  That's why I tried the approach described above.  Since it did
work, where the other suggestions did not, I thought it useful to post here.

In your P.S., you ask if it's known why some thumbnails disappear...mine did
so when I was selecting (Ctrl-Click) a number of thumbnails for tagging.
I'd selected a dozen or more and was clicking on the next one when,
suddenly, the selected thumbnails all vanished from the DigiKam thumbnails
view.  After checking to see if the images had been removed from the actual
file system directory, they were still there.  That's when I went down this
rabbit hole and - voila - what I found worked.  The lengthy re-scan time was
worth the wait in order to get DigiKam back on track.  And, yes, caution is
well-advised...

DigiKam's a really great program and kudos to those who developed it...



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html