Some questions about digikam4.db

[Anders Stedtlund]

Hi,

My fault. Didn't mean to remove the thumbnails, just the db and have the thumbnails in the filesystem instead.

The pro to have thumbnails in the same db as everything else must be that it's possible to join over that table in a single run. Now I assume that there's one query to find images and another query to find the thumbnails.

/Anders

Den 10 nov 2011 11:55 skrev "Jean-François Rabasse" <[hidden email]>:

On Thu, 10 Nov 2011, Anders Stedtlund wrote:

Hi Jean-François,

digiKam used to have thumbnail images in the file system. Do you
suggest that the thumbnails db should be removed in the future? (If I
read between the lines...)

Hi Anders,

Oh no, I didn't meant "removed"; having thumbnaisl under hand is an
important feature for such an application.
My opinion is that that kind of data, thumbnails or fingerprints,
should probably not be part of the master DB file.
Because it's not part of the DB schema, it's only auxiliary data,
setup in advance for faster access upon need, and can be rebuilt at
any time, so don't need to be backuped even in case of disk disaster.

But saving that data "outside" the master DB, be it into another DB
file or in the file system, or whatever, is of little importance, the
application knows how to get it and it's only a development choice.

The inconvenient of "packing" all in one DB file is size (as was the
start point of this thread). When one looks at Digikam versions using
two files, digikam4.db and thumbnails-digikam.db, one sees the second
file is about 10 times bigger !

Jean-François

PS: another argument for the "splitted" philosophy is when an images
application runs on different machines. It's probably not a Digikam
issue, but it's a common configuration in web based images bank applications; the main application (CGI program) is on one machine
running a HTTP server, the database is on another machine running a
SQL server, Oracle, PorsgreSQL, MySQL, etc., protected on a local
network.
The design rule says "data should be where it's needed", so the
application will access master DB data via SQL requests, but will
load and send/display thumbnails images from it's local disk system.
If you put thumbnails in the main DB, you jam your local network
bandwidth.

So, splitting -> more flexibility and in some cases higher performances.


_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Jean-François Rabasse]

On Thu, 10 Nov 2011, Anders Stedtlund wrote:

> Hi,
>
> My fault. Didn't mean to remove the thumbnails, just the db and have the
> thumbnails in the filesystem instead.
>
> The pro to have thumbnails in the same db as everything else must be that
> it's possible to join over that table in a single run. Now I assume that
> there's one query to find images and another query to find the thumbnails.

You can issue joins with tables from different databases, if you
"attach" another DB to your current handle.
   attach another-base.db as nails;
then
   select .... from Images, nails.Thumbnails  where .. etc;

But this will be "single run" only if you process scalar tuples attributes.
With large size data, and thumbnails are, you could get a blob handle but
to get the effective data (the image bytes) you need to load via the blob
interface, sqlite3_blob_open(), sqlite3_blob_read(), sqlite3_blob_close()

It's nothing but "hidden" file access, and the management is close to
standard Unix I/O open(), read(), close().
So, probably not really a pro. IMHO

But it would be great if DK developers could just explain the reason why
the 2 DB, master and thumbnails, of previous versions have been merged.
What's the technical reason, what's the expected benefit ?

JF
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

2011/11/10 Jean-François Rabasse <[hidden email]>

On Thu, 10 Nov 2011, Anders Stedtlund wrote:

Hi,

My fault. Didn't mean to remove the thumbnails, just the db and have the
thumbnails in the filesystem instead.

The pro to have thumbnails in the same db as everything else must be that
it's possible to join over that table in a single run. Now I assume that
there's one query to find images and another query to find the thumbnails.

You can issue joins with tables from different databases, if you "attach" another DB to your current handle.
 attach another-base.db as nails;
then
 select .... from Images, nails.Thumbnails  where .. etc;

But this will be "single run" only if you process scalar tuples attributes.
With large size data, and thumbnails are, you could get a blob handle but
to get the effective data (the image bytes) you need to load via the blob
interface, sqlite3_blob_open(), sqlite3_blob_read(), sqlite3_blob_close()

It's nothing but "hidden" file access, and the management is close to standard Unix I/O open(), read(), close().
So, probably not really a pro. IMHO

But it would be great if DK developers could just explain the reason why
the 2 DB, master and thumbnails, of previous versions have been merged.
What's the technical reason, what's the expected benefit ?

In my archive, i found the commit in source code from Marcel which introduce the dysfunction about thumbnails archiving in main database :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/0ef1283f8952d2d7173b8c3b3e8895e50d3da4d5 

Gilles Caulier

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Jean-François Rabasse]

On Thu, 10 Nov 2011, Gilles Caulier wrote:

> In my archive, i found the commit in source code from Marcel which introduce
> the dysfunction about thumbnails archiving in main database :
>
> https://projects.kde.org/projects/extragear/graphics/digikam/repository/rev
> isions/0ef1283f8952d2d7173b8c3b3e8895e50d3da4d5 

Many thanks Gilles,
I've read. So, a code implementation issue rather than a design or
functional issue.
Well, no impact in normal use of DK because the total disk space used is
the same, one or two DB.
And if this implementation, single DB, is to stay as the standard design,
the size problem raised by users and starting this thread can be fixed
with a bit of cleanup before backup...

Jean-François
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users