[Bug 283062] New: Limit the size of thumbnail database

https://bugs.kde.org/show_bug.cgi?id=283062

           Summary: Limit the size of thumbnail database
           Product: digikam
           Version: 1.9.0
          Platform: Fedora RPMs
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: wishlist
          Priority: NOR
         Component: Database
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]


Version:           1.9.0 (using KDE 4.6.5)
OS:                Linux

There should be an option to limit the size of the thumbnail database. Mine is
several hundred MB already, and this is a problematic issue e.g. for backups
because the database also changes all the time. It's also a problem on network
drives.

Alternatively give users the option to use KDE's .thumbnail folder instead,
which is there anyway.

Reproducible: Always



Expected Results:  
Saves disk space

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=283062


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
            Version|1.9.0                       |2.2.0




--- Comment #1 from Gilles Caulier <caulier gilles gmail com>  2011-09-30 08:29:53 ---
>There should be an option to limit the size of the thumbnail database. Mine is
>several hundred MB already, and this is a problematic issue e.g. for backups
>because the database also changes all the time.

About size this is not true. .thumbnail is bigger than digiKam DB...

>It's also a problem on network drives.

Why ? .thumbnail cannot manage removable media as network and usb drives.

>Alternatively give users the option to use KDE's .thumbnail folder instead,
>which is there anyway.

This option exist in cmake compilation setup :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/README#L118

But .thumbnail way is not a solution to fix size amount used by thumbs : It use
PNG image (LZW compression). digiKam DB use PGF which use wavelets compression.
There is a ratio of 2~3 between.

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=283062





--- Comment #2 from Marcel Wiesweg <marcel wiesweg gmx de>  2011-10-01 10:22:52 ---
There is need for a garbage collector if you have a troughput of images
(getting many new ones _and_ delete old ones). If would expect that most here
dont delete old images.

Apart from garbage collection, limiting the size would mean deleting some
(random) entries when new ones are added. So consequently: If you want a small
thumbnail db, delete the file and start with 0, all will be regenerated... As
for backup, the indication would be to preserve the processing time invested in
creating the thumbnails. There is no extra info to be backed up, backup is
needed for the main database and the image files.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=283062





--- Comment #3 from Stephan Matthiesen <stephan matthiesen gmx de>  2011-10-03 09:43:45 ---
Hi, perhaps I wasn't quite clear.

The problem with backups is that the thumbnail-db is one large file which
changes every time. So as soon as I do something in digikam, my backup needs to
upload the whole several hundred MB to my backup medium again.

This is not the case with the .thumbnails folder. There only a few files
change, so only the actual changes need to be backed up.

My comment about network drives: If home directory is on a network, the whole
db file has to move through the network. With .thumbnails the network traffic
would be much less, because again only the actual changes, not the whole
folder, need to be moved through the network.

I understand that .thumbnails is larger than the db. But the issue is that the
thumbnails are already there, if you use other programs like konqueror to look
at the images. So if you were able to re-use the .thumbnails folder in digikam,
there would be no *additional* space necessary.

It's nice that there is a compilation option, but most users don't compile...

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=283062





--- Comment #4 from Gilles Caulier <caulier gilles gmail com>  2011-10-03 09:54:25 ---
The reasons why we use a dedicated DB for thumbnails are :

- size of thumbs divided by 2~3 due to use wavelets compression (instead LZW
through PNG)
- The way to thumbnalize un-moutable media as DVD/CD/USB key. You can search
items through unavailable media and show thumbs of items use DB. It's not
possible to do with .thumbnails specification from free-desktop paper.

Anyway, you can turn off thumbs DB and use .thumbnails instead. Look Cmake
compilation option as i explain previously.

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=283062





--- Comment #5 from Marcel Wiesweg <marcel wiesweg gmx de>  2011-10-03 11:40:03 ---
Btw, face thumbnails (thumbnails for a detail of a picture) won't work with the
.thumbnails standard

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=283062





--- Comment #6 from Stephan Matthiesen <stephan matthiesen gmx de>  2011-10-04 05:38:22 ---
Hello again, let me please clarify something again. Using .thumbnail is not the
main point of my featuer request (just an alternative suggestion).

The main point is: please provide an option to limit the size of the thumbnails
database. There are many reasons why users may not want a file of several
hundred MBs.

Comment #2 says you can delete the db and it gets recreated in a smaller size.
Yes, I've figured that out and done that several times. Later I just symlinked
it to /tmp. But this is a power user perspective.

If the file is not strictly needed and the necessary info can be recreated
easily, why can't you make it easier for the casual user who does not have the
time to figure out what is taking up so much diskspace/backup bandwidth but
just assumes that programmes are efficient with their ressources?

It seems to me that this is an easy and fair request to ask that a programme
doesn't create huge databases that are not strictly needed, or at least give
the user more control over the process.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Peter Potrowl]

https://bugs.kde.org/show_bug.cgi?id=283062

Peter Potrowl <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #7 from Peter Potrowl <[hidden email]> ---
My question may be stupid but all my pictures have an embedded thumbnail. I
never use digiKam with unmountable media. So, why do I need those thumbnails to
be saved again (in a DB or in .thumbnails)?

Maybe we could save disk space by storing the thumbnails in the DB only when
the original files does not contain one?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=283062

--- Comment #8 from Gilles Caulier <[hidden email]> ---
To be able to display thumbnail from removable media which are been
disconnected from computer. The idea : searching items through DB with visible
thumbnails must be possible...

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=283062

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Limit the size of thumbnail |THUMBDB : Limit the size of
                   |database                    |thumbnail database

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=283062

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|THUMBDB : Limit the size of |THUMBDB : limit the size of
                   |thumbnail database          |thumbnail database

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=283062

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|Database                    |Database-Thumbs

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=283062

Mario Frank <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Latest Commit|                            |https://commits.kde.org/dig
                   |                            |ikam/2f8ddd42ef62d7aea9e490
                   |                            |cdb05ffcc644810c81
             Status|UNCONFIRMED                 |RESOLVED
   Version Fixed In|                            |5.5.0
         Resolution|---                         |FIXED

--- Comment #9 from Mario Frank <[hidden email]> ---
Git commit 2f8ddd42ef62d7aea9e490cdb05ffcc644810c81 by Mario Frank.
Committed on 22/02/2017 at 15:05.
Pushed by mfrank into branch 'master'.

Merged the current state of the garbage collection branch which improves the
database cleanup stage of the maintenance
and improves the reactiveness of the maintenance overall. We ported the way
items are processed to a queue based method
that can use the CPUs more effectively and does not create thousands of
threads.
Related: bug 216895, bug 374225, bug 351658, bug 362023, bug 329353
FIXED-IN: 5.5.0

M  +17   -12   NEWS

https://commits.kde.org/digikam/2f8ddd42ef62d7aea9e490cdb05ffcc644810c81

--
You are receiving this mail because:
You are the assignee for the bug.