New database schema

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

New database schema

Marcel Wiesweg
Hi,

during the last weeks I have been working on implementing the new database
schema that has been discussed for a long time.
The code will be merged into SVN tonight. It is a basic working version, with
a lot still to do (see below).

If you are a brave SVN user, this means for you:
- your old database will not be touched, it will be copied to digikam4.db and
updated to the new schema. The new database cannot be used with digikam 0.9.
- we do not guarantee yet that the final schema for 0.10 is be reached. Bugs
may still emerge. This means you may be required to delete your digikam4.db
file when problems arise. This state is (deliberately) set to last at most
one month from now (Nov 17th)
=> dont use digikam 0.10 yet for production work

Enough of that, what is new?
The new database schema offers room for much more information from the images.
For example, almost everything that you can read in the Image Properties
sidebar is now stored in the db. This means as well that it will be
searchable. It offers a wealth of new possibilities and I hope it will close
some long-standing bugs.
Secondly, we will have support for multiple album roots. There is no reason
why digikam should be restricted to one directory only. With hardware support
from Solid, we will also support CDs and other removable media. Network share
support is planned as well
Along with multiple paths, and albums on removable media, comes that
requirement that images are not removed from the DB only because their
storage is currently not accessible.
Connected with that as well is the idea to track images by their contents.
This idea I have to admit is impudently copied from Amarok. And it is not
implemented yet; but we have the necessary hash value in the database to
compare files.
We want as well to add support for similarity searching using a Haar
algorithm. (This is a completely different hash, build from the decoded image
data)

Now the backdraw:
You dont see anything from all this if you try out SVN right now. You will
even notice that support for comments is broken. With the db change, we have
the fundament and the structural work, but we need to build on top of that.

Immediate TODOs include:
- fix the listing by date
- add progress display to the collection scan during database upgrade
- change notification need to be restored
- a class for you to access the DownloadHistory table
- make use of the information in sidebar, tooltip etc.
- comments are currently disabled, at least a basic version need to be
restored (multiple language and comment type support needs to have UI, too)


Marcel
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Bugzilla from mikmach@wp.pl
Dnia Wednesday 17 of October 2007, Marcel Wiesweg napisał:
> Hi,
>
> Enough of that, what is new?
> The new database schema offers room for much more information from the
> images. For example, almost everything that you can read in the Image
> Properties sidebar is now stored in the db. This means as well that it
> will be searchable. It offers a wealth of new possibilities and I hope
> it will close some long-standing bugs.

Big kudos to all involved in database extension :)

m.

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

mick-21
Marcel,

The expanded database and search capabilities sound nice  but I will
dont think I will be a brave SVN user at this stage.

Now to slip in a comment on a possible enhancement.  You mentioned being
able to search on more and different fields.  At the moment you can
search the comments for a value (e.g. holiday or even a space character)
but you cannot search for an empty fields.

How about the ability to search for empty fields.  i.e. search the
comments field for any empty fields so you can find all photos you have
not commented yet.  (perhaps you can and I have just missed it)

Keep up the good work.

Mick.


On Wed, 2007-10-17 at 22:07 +0200, Mikolaj Machowski wrote:

> Dnia Wednesday 17 of October 2007, Marcel Wiesweg napisał:
> > Hi,
> >
> > Enough of that, what is new?
> > The new database schema offers room for much more information from the
> > images. For example, almost everything that you can read in the Image
> > Properties sidebar is now stored in the db. This means as well that it
> > will be searchable. It offers a wealth of new possibilities and I hope
> > it will close some long-standing bugs.
>
> Big kudos to all involved in database extension :)
>
> m.
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Arnd Baecker
On Thu, 18 Oct 2007, mick wrote:

> Marcel,
>
> The expanded database and search capabilities sound nice  but I will
> dont think I will be a brave SVN user at this stage.
>
> Now to slip in a comment on a possible enhancement.  You mentioned being
> able to search on more and different fields.  At the moment you can
> search the comments for a value (e.g. holiday or even a space character)
> but you cannot search for an empty fields.
>
> How about the ability to search for empty fields.  i.e. search the
> comments field for any empty fields so you can find all photos you have
> not commented yet.  (perhaps you can and I have just missed it)

I don't think this is possible right now.
Could you please file an entry in the bug tracker,
so that your wish is not lost.

Best, Arnd

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Marcel Wiesweg
In reply to this post by mick-21
> Marcel,
>
> The expanded database and search capabilities sound nice  but I will
> dont think I will be a brave SVN user at this stage.

Thats ok, it was meant as a small warning ;-)
But as soon as we enter the beta stage, all testers will be highly welcome!

>
> Now to slip in a comment on a possible enhancement.  You mentioned being
> able to search on more and different fields.  At the moment you can
> search the comments for a value (e.g. holiday or even a space character)
> but you cannot search for an empty fields.
>
> How about the ability to search for empty fields.  i.e. search the
> comments field for any empty fields so you can find all photos you have
> not commented yet.  (perhaps you can and I have just missed it)

No you cannot, I will try to keep this in mind.

>
> Keep up the good work.
>
> Mick.
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

mick-21
In reply to this post by Arnd Baecker
OK,

thanks.

On Thu, 2007-10-18 at 09:37 +0200, Arnd Baecker wrote:

> On Thu, 18 Oct 2007, mick wrote:
>
> > Marcel,
> >
> > The expanded database and search capabilities sound nice  but I will
> > dont think I will be a brave SVN user at this stage.
> >
> > Now to slip in a comment on a possible enhancement.  You mentioned being
> > able to search on more and different fields.  At the moment you can
> > search the comments for a value (e.g. holiday or even a space character)
> > but you cannot search for an empty fields.
> >
> > How about the ability to search for empty fields.  i.e. search the
> > comments field for any empty fields so you can find all photos you have
> > not commented yet.  (perhaps you can and I have just missed it)
>
> I don't think this is possible right now.
> Could you please file an entry in the bug tracker,
> so that your wish is not lost.
>
> Best, Arnd
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Eddie Armstrong
In reply to this post by Marcel Wiesweg
Do you publish your schema so people can use your db programmatically -
e.g.via python?
What type of DB do you use?
TIA
Eddie
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Gilles Caulier-4


2007/10/19, Eddie Armstrong <[hidden email]>:
Do you publish your schema so people can use your db programmatically -
e.g.via python?

A paper witch describe the new schema is in svn. Look here :

http://websvn.kde.org/trunk/extragear/graphics/digikam/DBSCHEMA.ODS?view=log
 

What type of DB do you use?

digiKam Database format still SQLite of course (at this moment)

Gilles

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Arnd Baecker


On Fri, 19 Oct 2007, Gilles Caulier wrote:

> 2007/10/19, Eddie Armstrong <[hidden email]>:
> >
> > Do you publish your schema so people can use your db programmatically -
> > e.g.via python?
>
>
> A paper witch describe the new schema is in svn. Look here :
>
> http://websvn.kde.org/trunk/extragear/graphics/digikam/DBSCHEMA.ODS?view=log
>
>
> What type of DB do you use?
>
>
> digiKam Database format still SQLite of course (at this moment)

For the "old" (i.e. current) database,
cdigi might be of interest, see
http://www.kde-apps.org/content/show.php?content=15103

BTW: if you come up with some simple (or more complex ;-) examples
of using the digikam database in python, could you post them
here? I (and presumably others as well) would find them
interesting ...

Best, Arnd
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Gilles Caulier-4


2007/10/19, Arnd Baecker <[hidden email]>:


On Fri, 19 Oct 2007, Gilles Caulier wrote:

> 2007/10/19, Eddie Armstrong <[hidden email]>:
> >
> > Do you publish your schema so people can use your db programmatically -
> > e.g.via python?
>
>
> A paper witch describe the new schema is in svn. Look here :
>
> http://websvn.kde.org/trunk/extragear/graphics/digikam/DBSCHEMA.ODS?view=log
>
>
> What type of DB do you use?
>
>
> digiKam Database format still SQLite of course (at this moment)

For the "old" (i.e. current) database,
cdigi might be of interest, see
http://www.kde-apps.org/content/show.php?content=15103

BTW: if you come up with some simple (or more complex ;-) examples
of using the digikam database in python, could you post them
here? I (and presumably others as well) would find them
interesting ...

...and it can be added to the list here :

http://www.digikam.org/?q=about/related

Gilles

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Eddie Armstrong
Thank you both for your replies (and quick too :-)
I will gladly contribute any useful scripts as I would consider it a
contribution to all the great OSS stuff around from which I've
benefitted - however I am just a Python beginner so you might have a
long wait :-)
I will remember though
Thanks again
Eddie

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Gilles Caulier-4
And unforget than digiKam has a place to host scripts and install it on the system. digiKam do not use it but just provide script as well for advanced users.

http://websvn.kde.org/branches/extragear/kde3/graphics/digikam/utilities/scripts/

Gilles

2007/10/19, Eddie Armstrong <[hidden email]>:
Thank you both for your replies (and quick too :-)
I will gladly contribute any useful scripts as I would consider it a
contribution to all the great OSS stuff around from which I've
benefitted - however I am just a Python beginner so you might have a
long wait :-)
I will remember though
Thanks again
Eddie

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users


_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: New database schema

Marcel Wiesweg
In reply to this post by Eddie Armstrong
> Do you publish your schema so people can use your db programmatically -
> e.g.via python?
> What type of DB do you use?

Perhaps in the future we can implement a scripting API with the KDE Kross
framework.
If you use the db file directly, do not forget to check the DBVersion setting
in the Settings table (currently "5", due to all historical versions)
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users