database on laptop and server/NAS

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

database on laptop and server/NAS

Manuel Bock
Using didkam for quite some years, my photo collection meanwhile takes up too
much disk space to alway fully synchronize it to a small laptop with a total
of 128GB. ;-((       Additionally I don't like the idea to  have that laptop
stolen out of a hotel room and have someone potentially getting access to all
the photos of my life.

Therefore I have my master collection on a local server, that I am accessing,
e.g. to run the diashow on the TV, make some final post processing at a PC
with a larger display having a color profile etc. Connecting the laptop to the
server's DB via VPN is slow, shows timeouts or is sometimes impossible at all,
depending on the internet connection.

The solution used so far was:
- - - - - - - - - - - - - - - - - - - - - - -

1. manage a separate database on the laptop, that writes all tags into the
images

2. work with all the photos taken for example on a holiday trip on that laptop

3. copy all subdirectories into the server's digikam library tree on return
and have a digikam instance in my home LAN import them on next start up into
the server's DB

Disadvantages so far
- - - - - - - - - - - - - - - - -

a. new tags created on the server have to be synchronized to the laptop (a
generic picture with all tags assigned that is synchronized into the laptops
is my manual workaround currently)

b. album properties, like comments or the icon assigned get lost (I usually
postpone that step for being back home)

Issue
- - - - -

With face detection (or manually marking faces), revisioning and
nondestructive editing there seems to be more and more information not coming
over to the server database with my workflow.

Is there a new way or best practice to "export/import to/from another digikam
instance" that I missed or what other way is the best practice to transfer as
much information as possible between two different digikam databases?

BTW: If some of the developers read this - GREAT SOFTWARE!!!!!!! - many thanks
for all the efforts going into it!


regards
Manuel
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Ben Staude
Am Freitag 07 September 2012 schrieb Manuel Bock:

[...]

> Is there a new way or best practice to "export/import to/from another
> digikam instance" that I missed or what other way is the best practice to
> transfer as much information as possible between two different digikam
> databases?

Generally speaking, I think this sort of issues will become more and more
important for digikam, as images move to NAS devices, people use tablets and
so on... The setup "one PC, one OS, one HDD" is not the typical one for the
future. We should plan for an (Android|iOS)-app for use with the digikam
database...!

I use an old version of digikam without any of the features that cause your
problems, but as long as this is a) stored into the digikam database and b)
you have unique file names or digikam generates the same uniqueHash for your
images on the server and the laptop, a sync between the two databases should
not be too hard to achieve with some script outside digikam (I implemented the
DB schema (or part of it?) in perl with DBIx::Class in the past), or (of
course preferable) from within digikam.
With the caveats you always have with such a sync (tagXY assigned on laptop,
but not on server -> assign tag also on server, or did you delete it on server
and want it to be deleted on laptop as well?)

For the uniqueHash it might not be a good idea to write metadata into the
images, but this depends on how this hash is generated. Anybody knows?

Regards
Ben

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Gilles Caulier-4
2012/9/10 Ben Staude <[hidden email]>:

> Am Freitag 07 September 2012 schrieb Manuel Bock:
>
> [...]
>
>> Is there a new way or best practice to "export/import to/from another
>> digikam instance" that I missed or what other way is the best practice to
>> transfer as much information as possible between two different digikam
>> databases?
>
> Generally speaking, I think this sort of issues will become more and more
> important for digikam, as images move to NAS devices, people use tablets and
> so on... The setup "one PC, one OS, one HDD" is not the typical one for the
> future. We should plan for an (Android|iOS)-app for use with the digikam
> database...!

With 3.0.0, a new DLNA export tool have been created. Run digiKam from
your host computer export desired album through DLNA, and that all.
image are now visible on your tablet (install right DLNA support
application from store of course), or on your connected TV.

i do it at home with my Sony Bravia TV which support DLNA by default,
or with my iPad with Media::Connect application.

http://itunes.apple.com/fr/app/media-connect-for-dlna-upnp/id335036887?mt=8

Gilles Caulier
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Ben Staude
Am Montag 10 September 2012 schrieb Gilles Caulier:

> 2012/9/10 Ben Staude <[hidden email]>:
> > Am Freitag 07 September 2012 schrieb Manuel Bock:
> >
> > [...]
> >
> >> Is there a new way or best practice to "export/import to/from another
> >> digikam instance" that I missed or what other way is the best practice
> >> to transfer as much information as possible between two different
> >> digikam databases?
> >
> > Generally speaking, I think this sort of issues will become more and more
> > important for digikam, as images move to NAS devices, people use tablets
> > and so on... The setup "one PC, one OS, one HDD" is not the typical one
> > for the future. We should plan for an (Android|iOS)-app for use with the
> > digikam database...!
>
> With 3.0.0, a new DLNA export tool have been created. Run digiKam from
> your host computer export desired album through DLNA, and that all.
> image are now visible on your tablet (install right DLNA support
> application from store of course), or on your connected TV.

That's very good to hear! But that creates the need for digikam running on my
NAS!? The images are there, the database is (i.e., will be) there, so it's no
fun to turn on the PC just to have digikam load the images from the NAS and
send it to the tablet. The NAS is right there to do such things:) Another
issue: can you do things like filter via tags (only stored in digikam's
database) via DLNA?
Very interesting subject, also slightly off topic in this thread...

Can you give some details about how the uniqueHash is created?

Thanks (for your reply, and for digikam!)
Ben
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Gilles Caulier-4
2012/9/10 Ben Staude <[hidden email]>:

> Am Montag 10 September 2012 schrieb Gilles Caulier:
>> 2012/9/10 Ben Staude <[hidden email]>:
>> > Am Freitag 07 September 2012 schrieb Manuel Bock:
>> >
>> > [...]
>> >
>> >> Is there a new way or best practice to "export/import to/from another
>> >> digikam instance" that I missed or what other way is the best practice
>> >> to transfer as much information as possible between two different
>> >> digikam databases?
>> >
>> > Generally speaking, I think this sort of issues will become more and more
>> > important for digikam, as images move to NAS devices, people use tablets
>> > and so on... The setup "one PC, one OS, one HDD" is not the typical one
>> > for the future. We should plan for an (Android|iOS)-app for use with the
>> > digikam database...!
>>
>> With 3.0.0, a new DLNA export tool have been created. Run digiKam from
>> your host computer export desired album through DLNA, and that all.
>> image are now visible on your tablet (install right DLNA support
>> application from store of course), or on your connected TV.
>
> That's very good to hear! But that creates the need for digikam running on my
> NAS!? The images are there, the database is (i.e., will be) there, so it's no
> fun to turn on the PC just to have digikam load the images from the NAS and
> send it to the tablet.

Well, it's simple. on your NAS, if it run under Linux, install
miniDLNA. that all.

On mine, an Excito B" NAS server, it's done automatically :

http://www.excito.com/node/9


> The NAS is right there to do such things:) Another
> issue: can you do things like filter via tags (only stored in digikam's
> database) via DLNA?

On digiKam DLNA tool, it's yet done, but planed.

> Very interesting subject, also slightly off topic in this thread...
>
> Can you give some details about how the uniqueHash is created?

uniqueHash of what ?

Gilles Caulier
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Manuel Bock
Gilles, Ben,

thanks for the quick answers! Let me summarize, what I understood, to make sure I got everything:

1. There seems to be no way currently to move additional meta data from one DB into another with methods provided out of digikam.

2. The (external) srcipt Ben proposes, needs a unique key to identify the pictures for synchronization between the two instances of digikam. He mentioned either a unique filename or a hash (not sure, probably stored within the DB and not visible to he normal user?)

3. Gilles mentioned a new DLNA export, that solves parts of my problem, showing (parts of) the collection on a TV or the iPad. What is a very promising outlook.

Unfortunately I am not able to write the script, Ben proposes, and the DNLA streaming does not help me to import the parts of the collection, I am working with, when on the road, coming back home. Remember, because of the screen size of the laptop, I would appreciate still being able to do some image fine tuning (plus geo tagging and so on) on a desktop.

Just an end user's thought: Would it be possible to expand the "Export/Import to/form another computer" function by a checkbox "include meta data", that writes/reads an "album side cart" for digikam-to-digikam transfer?

regards
Manuel

Am 10.09.2012 um 17:07 schrieb Gilles Caulier <[hidden email]>:

> 2012/9/10 Ben Staude <[hidden email]>:
>> Am Montag 10 September 2012 schrieb Gilles Caulier:
>>> 2012/9/10 Ben Staude <[hidden email]>:
>>>> Am Freitag 07 September 2012 schrieb Manuel Bock:
>>>>
>>>> [...]
>>>>
>>>>> Is there a new way or best practice to "export/import to/from another
>>>>> digikam instance" that I missed or what other way is the best practice
>>>>> to transfer as much information as possible between two different
>>>>> digikam databases?
>>>>
>>>> Generally speaking, I think this sort of issues will become more and more
>>>> important for digikam, as images move to NAS devices, people use tablets
>>>> and so on... The setup "one PC, one OS, one HDD" is not the typical one
>>>> for the future. We should plan for an (Android|iOS)-app for use with the
>>>> digikam database...!
>>>
>>> With 3.0.0, a new DLNA export tool have been created. Run digiKam from
>>> your host computer export desired album through DLNA, and that all.
>>> image are now visible on your tablet (install right DLNA support
>>> application from store of course), or on your connected TV.
>>
>> That's very good to hear! But that creates the need for digikam running on my
>> NAS!? The images are there, the database is (i.e., will be) there, so it's no
>> fun to turn on the PC just to have digikam load the images from the NAS and
>> send it to the tablet.
>
> Well, it's simple. on your NAS, if it run under Linux, install
> miniDLNA. that all.
>
> On mine, an Excito B" NAS server, it's done automatically :
>
> http://www.excito.com/node/9
>
>
>> The NAS is right there to do such things:) Another
>> issue: can you do things like filter via tags (only stored in digikam's
>> database) via DLNA?
>
> On digiKam DLNA tool, it's yet done, but planed.
>
>> Very interesting subject, also slightly off topic in this thread...
>>
>> Can you give some details about how the uniqueHash is created?
>
> uniqueHash of what ?
>
> Gilles Caulier
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

nianderson
On 09/10/2012 10:33 AM, Dr. Manuel Bock wrote:

> Just an end user's thought: Would it be possible to expand the
> "Export/Import to/form another computer" function by a checkbox
> "include meta data", that writes/reads an "album side cart"
> for digikam-to-digikam transfer?

Disconnected collection browsing would also be useful for this usecase.
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Ben Staude
In reply to this post by Gilles Caulier-4
Am Montag 10 September 2012 schrieb Gilles Caulier:

> 2012/9/10 Ben Staude <[hidden email]>:
> > Am Montag 10 September 2012 schrieb Gilles Caulier:
> >> 2012/9/10 Ben Staude <[hidden email]>:
> >> > Am Freitag 07 September 2012 schrieb Manuel Bock:
> >> >
> >> > [...]
> >> >
> >> >> Is there a new way or best practice to "export/import to/from another
> >> >> digikam instance" that I missed or what other way is the best
> >> >> practice to transfer as much information as possible between two
> >> >> different digikam databases?
> >> >
> >> > Generally speaking, I think this sort of issues will become more and
> >> > more important for digikam, as images move to NAS devices, people use
> >> > tablets and so on... The setup "one PC, one OS, one HDD" is not the
> >> > typical one for the future. We should plan for an (Android|iOS)-app
> >> > for use with the digikam database...!
> >>
> >> With 3.0.0, a new DLNA export tool have been created. Run digiKam from
> >> your host computer export desired album through DLNA, and that all.
> >> image are now visible on your tablet (install right DLNA support
> >> application from store of course), or on your connected TV.
> >
> > That's very good to hear! But that creates the need for digikam running
> > on my NAS!? The images are there, the database is (i.e., will be) there,
> > so it's no fun to turn on the PC just to have digikam load the images
> > from the NAS and send it to the tablet.
>
> Well, it's simple. on your NAS, if it run under Linux, install
> miniDLNA. that all.

No problem, right now I just use an nfs export. But - like miniDLNA - this is
not aware of all the data that's stored in the digikam DB:(

> On mine, an Excito B" NAS server, it's done automatically :
>
> http://www.excito.com/node/9
>
> > The NAS is right there to do such things:) Another
> > issue: can you do things like filter via tags (only stored in digikam's
> > database) via DLNA?
>
> On digiKam DLNA tool, it's yet done, but planed.

That's where digikam has to be run on my desktop pc again:( I wonder what the
best solution would be to use the data in the digikam DB without a full
digikam installation (like on a tablet). First step could be read-only
("filter by tag to show only certain images to friends on the tablet"), second
step "use tablet even to assign tags while browsing images on the couch")...
Android app? Web interface run on the NAS? Need to write everything into the
files and can't use any of the advantages of the (already existing) database
(like performance when filtering...)

> > Very interesting subject, also slightly off topic in this thread...
> >
> > Can you give some details about how the uniqueHash is created?
>
> uniqueHash of what ?

The uniqueHash that is stored in the "Images" table. I think this is used by
digikam to identify images that, for example, have moved to another path since
the last start of digikam (so that metadata won't be lost)?


_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Gilles Caulier-4
2012/9/11 Ben Staude <[hidden email]>:

> Am Montag 10 September 2012 schrieb Gilles Caulier:
>> 2012/9/10 Ben Staude <[hidden email]>:
>> > Am Montag 10 September 2012 schrieb Gilles Caulier:
>> >> 2012/9/10 Ben Staude <[hidden email]>:
>> >> > Am Freitag 07 September 2012 schrieb Manuel Bock:
>> >> >
>> >> > [...]
>> >> >
>> >> >> Is there a new way or best practice to "export/import to/from another
>> >> >> digikam instance" that I missed or what other way is the best
>> >> >> practice to transfer as much information as possible between two
>> >> >> different digikam databases?
>> >> >
>> >> > Generally speaking, I think this sort of issues will become more and
>> >> > more important for digikam, as images move to NAS devices, people use
>> >> > tablets and so on... The setup "one PC, one OS, one HDD" is not the
>> >> > typical one for the future. We should plan for an (Android|iOS)-app
>> >> > for use with the digikam database...!
>> >>
>> >> With 3.0.0, a new DLNA export tool have been created. Run digiKam from
>> >> your host computer export desired album through DLNA, and that all.
>> >> image are now visible on your tablet (install right DLNA support
>> >> application from store of course), or on your connected TV.
>> >
>> > That's very good to hear! But that creates the need for digikam running
>> > on my NAS!? The images are there, the database is (i.e., will be) there,
>> > so it's no fun to turn on the PC just to have digikam load the images
>> > from the NAS and send it to the tablet.
>>
>> Well, it's simple. on your NAS, if it run under Linux, install
>> miniDLNA. that all.
>
> No problem, right now I just use an nfs export. But - like miniDLNA - this is
> not aware of all the data that's stored in the digikam DB:(
>
>> On mine, an Excito B" NAS server, it's done automatically :
>>
>> http://www.excito.com/node/9
>>
>> > The NAS is right there to do such things:) Another
>> > issue: can you do things like filter via tags (only stored in digikam's
>> > database) via DLNA?
>>
>> On digiKam DLNA tool, it's yet done, but planed.
>
> That's where digikam has to be run on my desktop pc again:( I wonder what the
> best solution would be to use the data in the digikam DB without a full
> digikam installation (like on a tablet). First step could be read-only
> ("filter by tag to show only certain images to friends on the tablet"), second
> step "use tablet even to assign tags while browsing images on the couch")...
> Android app? Web interface run on the NAS? Need to write everything into the
> files and can't use any of the advantages of the (already existing) database
> (like performance when filtering...)
>
>> > Very interesting subject, also slightly off topic in this thread...
>> >
>> > Can you give some details about how the uniqueHash is created?
>>
>> uniqueHash of what ?
>
> The uniqueHash that is stored in the "Images" table. I think this is used by
> digikam to identify images that, for example, have moved to another path since
> the last start of digikam (so that metadata won't be lost)?

Look here :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/dimg/loaders/dimgloader.cpp#L308

Gilles Caulier
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Manuel Bock
For filtered dia shows I have a digikam instance running on a small PC
attached to the TV, that accesses the server's database. I wonder, if
accessing a central mysql db could also be the solution for other, read only
apps, accessing the digikam vault of pictures. But even then, there remains my
initial problem or question:

How do I work offline with digikam (out of the LAN, internet connection too
slow or simply not available) and how do I import the advanced meta data
afterwards, currently not  stored  in the picture file's meta data or it's
side cart?

Is there an easy way to dump the server's mysql db for import into the laptops
sqlite's and vice versa? How do I synchronize the corrsponding file structure,
when the laptop is not capable of holding all pictures?

Since there seems to be a cryptographic hash used for the uniqueHash, Ben
asked for, there might be chance of writing the script he proposes. Still
conflict resolution is to be determinded. I guess, this could also be used for
bringing over non-destructive editing or versioning information?

My proposal to extend the "Export to .." and "Import from another computer"
function to use a side cart for holding the currently missing meta data would
leave the conflict resolution transparently with the user and would help round
up  the user experience again:

* When I am importing files, the meta data contained in those pictures gets
added to my DB.

* Exporting and importing things form digikam, a user would expect, this
covers all the meta data. he entered (wearing my Mac glasses currently ;-))

* Surprisingly, parts of it gets lost, and I have to have at least a partly
understanding of the data structure or digikam's history to understand what
will get lost.

* For control freaks or data sharing with others, there could even be the a
granualar setup, which meta data to in/exclude ;-)

* a new person will be imported as a new tag, and a detected/marked face could
be added as well, the different image files are already imported, the
additional non-destructive editing steps would be imported and the revisioning
relation could be imported as well

* if I import pictures as duplicates, I can use find duplicates - I mean: it
was my fault, so I have to do cleaning up (I can even import picture from a
camera twice)

Would it be a big effort to implement that meta data export/import add-on?
Does it make sense to add this to the wish list? Is the proposed
external/internal script a better solution, or is yet another, better approach
already coming up?

regards
Manuel


Am Dienstag, 11. September 2012, 22:33:44 schrieb Gilles Caulier:

> 2012/9/11 Ben Staude <[hidden email]>:
> > Am Montag 10 September 2012 schrieb Gilles Caulier:
> >> 2012/9/10 Ben Staude <[hidden email]>:
> >> > Am Montag 10 September 2012 schrieb Gilles Caulier:
> >> >> 2012/9/10 Ben Staude <[hidden email]>:
> >> >> > Am Freitag 07 September 2012 schrieb Manuel Bock:
> >> >> >
> >> >> > [...]
> >> >> >
> >> >> >> Is there a new way or best practice to "export/import
> >> >> >> to/from another digikam instance" that I missed or what
> >> >> >> other way is the best practice to transfer as much
> >> >> >> information as possible between two different digikam
> >> >> >> databases?
> >> >> >
> >> >> > Generally speaking, I think this sort of issues will become
> >> >> > more and more important for digikam, as images move to NAS
> >> >> > devices, people use tablets and so on... The setup "one PC,
> >> >> > one OS, one HDD" is not the typical one for the future. We
> >> >> > should plan for an (Android|iOS)-app for use with the
> >> >> > digikam database...!
> >> >>
> >> >> With 3.0.0, a new DLNA export tool have been created. Run
> >> >> digiKam from your host computer export desired album through
> >> >> DLNA, and that all. image are now visible on your tablet
> >> >> (install right DLNA support application from store of course),
> >> >> or on your connected TV.
> >> >
> >> > That's very good to hear! But that creates the need for digikam
> >> > running on my NAS!? The images are there, the database is (i.e.,
> >> > will be) there, so it's no fun to turn on the PC just to have
> >> > digikam load the images from the NAS and send it to the tablet.
> >>
> >> Well, it's simple. on your NAS, if it run under Linux, install
> >> miniDLNA. that all.
> >
> > No problem, right now I just use an nfs export. But - like miniDLNA -
> > this is not aware of all the data that's stored in the digikam DB:(
> >
> >> On mine, an Excito B" NAS server, it's done automatically :
> >>
> >> http://www.excito.com/node/9
> >>
> >> > The NAS is right there to do such things:) Another
> >> > issue: can you do things like filter via tags (only stored in
> >> > digikam's database) via DLNA?
> >>
> >> On digiKam DLNA tool, it's yet done, but planed.
> >
> > That's where digikam has to be run on my desktop pc again:( I wonder
> > what the best solution would be to use the data in the digikam DB
> > without a full digikam installation (like on a tablet). First step
> > could be read-only ("filter by tag to show only certain images to
> > friends on the tablet"), second step "use tablet even to assign tags
> > while browsing images on the couch")... Android app? Web interface run
> > on the NAS? Need to write everything into the files and can't use any
> > of the advantages of the (already existing) database (like performance
> > when filtering...)
> >
> >> > Very interesting subject, also slightly off topic in this
> >> > thread...
> >> >
> >> > Can you give some details about how the uniqueHash is created?
> >>
> >> uniqueHash of what ?
> >
> > The uniqueHash that is stored in the "Images" table. I think this is
> > used by digikam to identify images that, for example, have moved to
> > another path since the last start of digikam (so that metadata won't be
> > lost)?
>
> Look here :
>
> https://projects.kde.org/projects/extragear/graphics/digikam/repository/revi
> sions/master/entry/libs/dimg/loaders/dimgloader.cpp#L308
>
> Gilles Caulier
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Jean-François Rabasse
In reply to this post by Manuel Bock

Hello,

Some - late - comments about this very interesting topic.

On Fri, 7 Sep 2012, Manuel Bock wrote:

> The solution used so far was:
> - - - - - - - - - - - - - - - - - - - - - - -
>
> 1. manage a separate database on the laptop, that writes all tags
> into the images
>
> 2. work with all the photos taken for example on a holiday trip on
> that laptop
>
> 3. copy all subdirectories into the server's digikam library tree on
> return and have a digikam instance in my home LAN import them on
> next start up into the server's DB
I have a very similar workflow, main home work done on a home desktop
and extra work, e.g. when in holidays, on a laptop. And, of course,
synchronisation issues once back home.


1. First, I fully agree with what Ben says :

On Mon, 10 Sep 2012, Ben Staude wrote:

> Generally speaking, I think this sort of issues will become more and
> more important for digikam, as images move to NAS devices,
> people use tablets and so on... The setup "one PC, one OS, one HDD"
> is not the typical one for the future. We should plan for an
> (Android|iOS)-app for use with the digikam database...!

Sure, this organisation model will probably reflect less and less
users way of working.
- more and more users have more than one computer, desktops, laptops,
tablets, etc.
- more and more users use non local storage devices, e.g. pluggable
USB drives or NAS.
- probably in a near future, more and more users will use
dematerialised storage, cloud drives et al.

So, rethinking the « one computer - one local drive - one online
collection » model will become necessary in the next few years.
At least, it's a bet.


2. I have also to face another issue : disconnected collection.
This has also been reminded on that thread :

On Mon, 10 Sep 2012, Nick Anderson wrote:

> Disconnected collection browsing would also be useful for this
> usecase.

... and has already been discussed on another thread
(Off-line Collection Browsing & DB cleanup, 10 to 13 May 2012)

This is a related issue because it's also a consequence of the
current « local drive collection » model. DK requires that all images
be online at any moment as the Scan for new images implementation
assumes that non present images have been removed, instead of being
temporary off-line.

My personal workaround is to keep new images on local drive as long as
I need them for edition tasks, then move new folders and images to an
external USB drive and replace each image file with a reduced size
version.

I agree it's not that great, it's a dirty trick, but as I haven't
enough disk space to hold my collections on a local drive, I still
can use DK to browse and search a reduced size images tree.
And, of course, when I want to access original images after searching,
to be used for a web album or paper prints, I need to plug my USB
storage (on my desktop or my laptop, at convenience).

I do think this off-line collection issue should be considered in a
future, along with support for remote data, NAS, pluggable devices,
cloud devices, etc. Strictly speaking, the physical presence of an
original image file is required (on a local drive) only in two
occasions :
- image edition, of course
- image use, paper print et al.

but for tagging, browsing, searching, only the existence of an image
has to be known, not the effective data. (NB: this is a very old
concept that has been used for centuries by books libraries keepers
and is called « book phantom ». The phantom is a cardboard sheet that
will remain on the shelves (local storage) when a book is borrowed
outside the library (off-line data). That sheet holds the book
informations, title, author, abstract, keywords (metadata) and is
enough for someone to search the library.)


3. Synchronisation issues

I work the same way as Manuel does, edit images metadata and write
into the images files. Synchronisation between computers is done on
the whole tree (that may contain full size new images and reduced size
off-line images). I use tools such as rsync.

This works well, as restarting DK on the target computer will find the
new added images or edited images, and reload metadata from files,
thus updating the database.

This works, but this is manual operations and must be careful
operations. Metadata synchronisation is ambiguous and not errors free,
as Ben says :

On Mon, 10 Sep 2012, Ben Staude wrote:

> With the caveats you always have with such a sync (tagXY assigned on
> laptop, but not on server -> assign tag also on server, or did you
> delete it on server and want it to be deleted on laptop as well?)

Right, and I believe there's no solution today. Having intelligent
synchronisation would require a kind of edition journaling in order to
keep time information along data. Things like « Tag added/modified on
<date-1> » , « Tag removed on <date-2> » , and let a synchronisation
tool rebuild the currently expected metadata state of the document.

As far as I know, there's no such implementation in metadata standards
that describe only data semantics and labels but no historical
information, so the only available solution is : be careful : )


4. Metadata synchronisation issues

On Mon, 10 Sep 2012, Ben Staude wrote:

> For the uniqueHash it might not be a good idea to write metadata
> into the images, but this depends on how this hash is generated.
> Anybody knows?

As appears from the DK source code, the image uniqueHash is a MD5
digest computed from the first and last tenths of kilobytes of the
images. So right, updating metadata in the image file itself,
or editing the image itself, will modify that signature and the
image may be thought as new after some migrations.

It's not a problem - in my opinion - if all metadata is stored into the
image file. DK may think it's a new image, reread metadata, and the DB
will be up to date.
It becomes a problem if extra image related information is stored in
the DB only, then the connection between information and image will be
lost.

My personal choice has been to :
- write metadata to images
- use only metadata writable informations, title, description, tags,
rating, etc. but avoid using features that may leave information in
the database only. This is a major usage restriction.

NB: at some time, I've used the sidecar files mechanism to avoid
writing into the images files. I've stopped using that after having
some disappointments and data losses; it seems (at least with my DK
2.2 version) that the sidecar files implementation is not fully
supported.

E.g. I happened to reorganise folders, search and select some images
sets, then move them via the drag/drop feature into another folder.
DK moves the images correctly, but not the related sidecar files. So,
when copying a collection tree to another computer and running DK on
the target collection, association between images and sidecar files is
broken. I don't know if this has been fixed, perhaps. If not it
probably should be and the « Move image » feature be replaced by
« Move image to target folder and if a sidecar file exists, move it
as well ».


5. Conclusion
I apologise for that long post which is a bit in idle mode; many
comments and questions, no answers nor solutions. But that kind of
topics, multi computers, offline data, appears regularly on this list
and this shows that users working habits evolve with time and are
certainly worth reflexions, discussions, hints sharing...

Regards,
Jean-François

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users
Reply | Threaded
Open this post in threaded view
|

Re: database on laptop and server/NAS

Ben Staude
In reply to this post by Manuel Bock
Am Mittwoch 12 September 2012 schrieb Manuel Bock:

[...]
> My proposal to extend the "Export to .." and "Import from another computer"
> function to use a side cart for holding the currently missing meta data
> would leave the conflict resolution transparently with the user and would >
> help round up  the user experience again:

I reread your current workflow. As far as I understand it, there should not be
any conflicts as you only import new images from the laptop (taken and edited
during vacation) into the master DB?

[...]

> Would it be a big effort to implement that meta data export/import add-on?

I'm not the one to really judge this. But as long as it's just some sort of
"advanced import" (see above), I think it's not too much effort.
I'd think of it as "Import subtree from another digikam instance" -> folder
subtree (@laptop) is added to master collection, metadata is taken from the
local digikam database. As you only import new images, no synchronisation
issues. New tags/faces/... appearing in the local DB are created in the master
as needed.

> Does it make sense to add this to the wish list?

My 2 cents: the wish list is there to add wishes and this is your wish -> go
for it:) Also it's a good place to collect ideas on how such a feature
could/should work.

> Is the proposed
> external/internal script a better solution, or is yet another, better
> approach already coming up?

An external script might be "better" as you can write it to perfectly fit your
personal workflow. In most (all?) other regards, it'd be better to have this
feature within digikam. For example, when the database schema changes with a
new version of digikam, your script might stop working, loose data or even
corrupt your database.

> regards
Ben
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users