[Bug 191494] New: Wish: search offline collections (e.g. removable medias)

[Raphaël Pinson]

https://bugs.kde.org/show_bug.cgi?id=191494

           Summary: Wish: search offline collections (e.g. removable
                    medias)
           Product: digikam
           Version: 0.10.0
          Platform: Ubuntu Packages
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: wishlist
          Priority: NOR
         Component: general
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]


Version:           0.10.0 (using KDE 4.2.2)
OS:                Linux
Installed from:    Ubuntu Packages

With the new multiple collections feature in Digikam 0.10.0, I was able to add
quite a few DVDs to my digikam collections.

However, I can only list the pictures on them if I insert the DVD _before_
starting digikam. This is not very useful, since I might not know where to find
the picture before inserting the DVD.

It would be great if all the search options were available for offline
collections. Maybe Digikam could store the metadata (including thumbnails) in
the local db so they can be used to perform searches, and then prompt the user
to insert the removable media to see the pictures (e.g. "Please insert
"DVD3-2009" to see this picture").

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=191494


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|general                     |Searches




--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=191494


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #1 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-05-03 21:05:36 ---
We have all info in the local db, but not the thumbnails. The freedesktop
standard that is used for storing thumbnails is not caring for removable media,
it stores by file path only. Though for now storing by filepath may be enough
for a wide range of use cases.
So yes this is possible.

When inserting a DVD digikam should recognize it even when running, but maybe
something is broken there. That is a bug.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Arnd Baecker]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #2 from Arnd Baecker <arnd baecker web de>  2009-05-03 21:12:05 ---
Maybe some visual indication for albums and/or individual images
would be helpful when they are not physically present (eg. some red border
etc.).
Also any action which would require the real image need to be
either disabled or recognized (eg, moving, renaming, editing etc.)
And yes, this would be a great feature....

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #3 from Mikolaj Machowski <mikmach wp pl>  2009-05-03 23:00:57 ---
Another solution could be creation of some .hidden hierarchy inside of main
root album where thumbnails for off-line images would be stored. Max sized
thumbnail in JPG format would be ca. 5kB so it wouldn't be hard on disk space.

#2 Arnd - red border is very strong signal. New Qt4 List Model View Marcel is
working on could use small overlayed CD icon (borrowed from FotoStation and
probably other software supporting off-line collections).

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #4 from Gilles Caulier <caulier gilles gmail com>  2009-05-04 06:50:10 ---
It will be interresting to know how other pro photo-management store
thumbnails, especially with remote media. there use database ?

gilles

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #5 from Mikolaj Machowski <mikmach wp pl>  2009-05-04 15:36:45 ---
FotoStation uses database with thumbnails embedded in it. Same for ACDSee as
far as I know.

The same also for non-pro apps like XnView.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #6 from Gilles Caulier <caulier gilles gmail com>  2009-05-04 22:19:14 ---
In these database, do you know which file format is use to store thumbs ?

JPEG, or PNG ?

Which DB file size ? I think if we store thumb in DB, it will become huge
speedly...

Marcel, what do you think about ? Perhaps we can create a second DB only
dedicated to store thumbs and forget now ~./thumbnails.

I think to have a better support of removable media is very important for the
future. This will solve backup issues...

Gilles

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #7 from Mikolaj Machowski <mikmach wp pl>  2009-05-04 23:47:33 ---
I suppose they use JPEG. FotoStation doesn't even supports PNG.

Don't remember FotoStation database size and related numbers but for my home
machine:
.thumbnails/large - 13000
almost all of them are coming from digiKam - total size: 1.1GB.
After compression to JPEG quality 80, subsampling 2x1 - 147MB. There is quality
difference, especially after upscaling, but there are gains. Note however that
my collection isn't very big. When we go into 50,000 - 100,000 collections db
could become monstrous.

This rings true for comparable size of FotoStation db - they recommend 2GB of
RAM...

Current digiKam database with for those images: 17MB

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #8 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-05-05 17:23:28 ---
If we choose this path, let's use a second database for this. There is no need
to bloat the main database, which contains valuable data and should be easy to
backup.
We should also critically reconsider what are the gains of a thumnail database
file versus simple storage as files in the file system (be it according to the
current spec or at a different place)

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #9 from Gilles Caulier <caulier gilles gmail com>  2009-05-05 17:52:34 ---
To Marcel,

I'm agree to separate DB : one for thumbs, another one for metadata.

The gain will be to use digiKam with remote media. Of course we lost
compatibility with opendesktop paper, but it's really important here ? Gwenview
and Dolphin/konqueror already support it. digiKam work out of simple desktop
image management scope now. For me it's not a problem, we are over :
photo-management.

With a dedicated thumbs DB, we can only save one thumbnail size by item :
256x256. No need to store another size as with OpenDesktop paper. As Mik said,
we can use JPEG with no compression value (to have the best image quality) or
better, another one based on wavelet compression to optimize DB size. Candidate
are :

JPEG2000 : GPL : slow (compression and decompression)
DJvu Photo : GPL : not yet tested.
OpenEXR : GPL : very slow (unadapted here)
HDPhoto : not GPL : Fast, but pattented by M$.
PGF : GPL : Not tested but very promissing :
http://en.wikipedia.org/wiki/Progressive_Graphics_File

The advantage to use wavelet instead JPEG : a better compression ratio than
JPEG : 5-10 !

Gilles

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #10 from Mikolaj Machowski <mikmach wp pl>  2009-05-05 20:25:37 ---
I gave examples for `convert -quality 80` which is quite lossy compression.
With transformation I really don't see differences between JPG and PNG `convert
-quality 90 -sampling-format 1x1` result is reduction of size from 1.1GB ->
247MB.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #11 from Gilles Caulier <caulier gilles gmail com>  2009-05-05 20:36:56 ---
Yes, JPEG with 80 compression is fine, but... using Wavelet compression will
decrease drastically thumbs data size. It's very important. Of course,
compression and decompression must be fast, at least as JPEG

I will make some tests with PGF format. It sound great...

Gilles

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #12 from Gilles Caulier <caulier gilles gmail com>  2009-05-29 07:54:15 ---
I found a comparison paper about PGF image format :

http://www.xeraina.ch/download/pgf_facts.pdf

Incredible no ? I think we need to use this format to host thumbs in digiKam :
space optimization a and fast load/saving.

Gilles

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #13 from Gilles Caulier <caulier gilles gmail com>  2009-05-29 08:21:58 ---
Marcel,

What's about to use a DB to host thumbnails cache ?

As we use multi-threading with current thumbs interface, if we use libsqlite to
host binary blob of thumbs will be compatible ? In other words, it safe to use
libsqlite with separate threads ? Look there :

http://www.sqlite.org/faq.html#q6

Gilles

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #14 from Gilles Caulier <caulier gilles gmail com>  2009-05-29 08:42:29 ---
For Informations:

PGF can host customized meta-data as byte-array. It's hosted just after header
of file and before image data. There are methods to set and get user data.
There is no no data size restriction. We can host what we want in this blob, as
Exif, IPTC or XMP (preferred), or all if necessary.

This is want mean that it's safe to use this format for photo purpose too...

Gilles

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #15 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-05-29 10:48:30 ---
The Qt SQL implementation requires that a database connection is not used
across threads; therefore we create one connection per thread. It requires (at
least I think so) not to access a database concurrently from two threads; so we
use a global mutex (by creating DatabaseAccess() each time). So we dont require
any special thread safety from sqlite.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #16 from Gilles Caulier <caulier gilles gmail com>  2009-05-29 11:01:46 ---
Created an attachment (id=34097)
 --> (http://bugs.kde.org/attachment.cgi?id=34097)
PGF compression tests

Some test results with PGF format using pgf command line tool:

* Original image:

-rwxr-xr-x 1 gilles gilles 17485555 2008-12-29 10:49 pict1999.png

* Original converted to PGF with wavelets lossless compression :
-rw-r--r-- 1 gilles gilles 13806596 2009-05-29 10:26 pict1999.pgf
Encoding time (encoding, writing PGF): 4.25 s
Total time (reading source, conversion, writing destination): 6.2 s
Compression ratio: 1.26646

* Original converted to PGF with wavelets compression level 10 (no visible
artifacts):
-rw-r--r-- 1 gilles gilles   949812 2009-05-29 10:51 pict1999-q10.pgf
Encoding time (encoding, writing PGF): 0.93 s
Total time (reading source, conversion, writing destination): 2.94 s
Compression ratio: 18.4095

* Thumbnail 256x256:
-rw-r--r-- 1 gilles gilles   366042 2009-05-29 10:28 pict1999-256.png

* Thumbnail converted to PGF with wavelets lossless compression :
-rw-r--r-- 1 gilles gilles   106942 2009-05-29 10:28 pict1999-256.pgf
Encoding time (encoding, writing PGF): 0.13 s
Total time (reading source, conversion, writing destination): 0.21 s
Compression ratio: 3.42281

* Thumbnail converted to PGF with wavelets compression level 10 (no visible
artifacts):

-rw-r--r-- 1 gilles gilles    22682 2009-05-29 10:45 pict1999-256-q10.pgf
Encoding time (encoding, writing PGF): 0.05 s
Total time (reading source, conversion, writing destination): 0.13 s
Compression ratio: 16.138

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #17 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-05-29 15:23:59 ---
Final output size and encoding time correlate with R^2=0.98 (though it's not
linear, the plot is exponential).
Is disk I/O the limiting factor? Then PGF is better because it creates so much
smaller files.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=191494





--- Comment #18 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-05-29 15:24:31 ---
It's logarithmic, I mean

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel