Metadata

[Anders Lund]

I really like having meta-data written to files, but I badly dislikes digikam
freezing. It must be possible to do these things async, without freezing the
entire application!

--  
Anders
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

Freezing digiKam want mean a crash in internal process. It can be an
Exiv2 shared lib crash.

Do you have a GDB trace to investiguate ?

Also which metadata setting you use ?

Best

Gilles Caulier

2013/6/28 Anders Lund <[hidden email]>:

> I really like having meta-data written to files, but I badly dislikes digikam
> freezing. It must be possible to do these things async, without freezing the
> entire application!
>
> --
> Anders
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Simon Cropper-3]

Hi Gilles,

I think Anders means that the program either allows you to do normal
digikam type stuff OR write metadata to files.

What he would like is the ability to write metadata to his files while
still being able to do other stuff (e.g. edit another image, browse
albums, etc).

Usually if you save the metadata for a single file it takes less than a
second but if you are tagging hundreds of files it could take longer.
Personally the longest wait I have had is ~5-10 seconds to save data to
~40 files. I suppose it is how you work and how many files you are
manipulating at one time.

On 29/06/13 14:13, Gilles Caulier wrote:

> Freezing digiKam want mean a crash in internal process. It can be an
> Exiv2 shared lib crash.
>
> Do you have a GDB trace to investiguate ?
>
> Also which metadata setting you use ?
>
> Best
>
> Gilles Caulier
>
> 2013/6/28 Anders Lund <[hidden email]>:
>> I really like having meta-data written to files, but I badly dislikes digikam
>> freezing. It must be possible to do these things async, without freezing the
>> entire application!
>>
>> --
>> Anders
>> _______________________________________________
>> Digikam-users mailing list
>> [hidden email]
>> https://mail.kde.org/mailman/listinfo/digikam-users
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

--
Cheers Simon

    Simon Cropper - Open Content Creator

    Free and Open Source Software Workflow Guides
    ------------------------------------------------------------
    Introduction               http://www.fossworkflowguides.com
    GIS Packages           http://www.fossworkflowguides.com/gis
    bash / Python    http://www.fossworkflowguides.com/scripting
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Anders Lund]

On Lørdag den 29. juni 2013 15:13:40, Simon Cropper wrote:
> Hi Gilles,
>
> I think Anders means that the program either allows you to do normal
> digikam type stuff OR write metadata to files.

Exactly.

> What he would like is the ability to write metadata to his files while
> still being able to do other stuff (e.g. edit another image, browse
> albums, etc).
>
> Usually if you save the metadata for a single file it takes less than a
> second but if you are tagging hundreds of files it could take longer.
> Personally the longest wait I have had is ~5-10 seconds to save data to
> ~40 files. I suppose it is how you work and how many files you are
> manipulating at one time.

That is what I sometimes do, for example by tagging a whole album with some
tags.

What I thought of was something like an external process and some que
management.

> On 29/06/13 14:13, Gilles Caulier wrote:
> > Freezing digiKam want mean a crash in internal process. It can be an
> > Exiv2 shared lib crash.
> >
> > Do you have a GDB trace to investiguate ?
> >
> > Also which metadata setting you use ?
> >
> > Best
> >
> > Gilles Caulier
> >
> > 2013/6/28 Anders Lund <[hidden email]>:
> >> I really like having meta-data written to files, but I badly dislikes
> >> digikam freezing. It must be possible to do these things async, without
> >> freezing the entire application!
> >>
> >> --
> >> Anders
> >> _______________________________________________
> >> Digikam-users mailing list
> >> [hidden email]
> >> https://mail.kde.org/mailman/listinfo/digikam-users
> >
> > _______________________________________________
> > Digikam-users mailing list
> > [hidden email]
> > https://mail.kde.org/mailman/listinfo/digikam-users

--
Anders
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Le 29/06/2013 09:43, Anders Lund a écrit :

> That is what I sometimes do, for example by tagging a whole album with some
> tags.
>
> What I thought of was something like an external process and some que
> management.

then use exiftools...

I don't know if digikam batch allows tagging

jdd

--
http://www.dodin.org
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Anders Lund]

On Lørdag den 29. juni 2013 09:54:15, jdd wrote:

> Le 29/06/2013 09:43, Anders Lund a écrit :
> > That is what I sometimes do, for example by tagging a whole album with
> > some
> > tags.
> >
> > What I thought of was something like an external process and some que
> > management.
>
> then use exiftools...
>
> I don't know if digikam batch allows tagging

I really like using digikam :-)

--
Anders
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Christoph Siedentop-2]

Same problem here. One easy fix would be to simply wait with writing metadata to files for five minutes (the DB would be affected immediately though). Because for me this only causes problems when tagging a complete album. And what I like to do is start tagging big and then tag groups of files and finally individual files.

--
Christoph Siedentop

Sent from mobile.

On 29.06.2013, at 09:59, Anders Lund <[hidden email]> wrote:

> On Lørdag den 29. juni 2013 09:54:15, jdd wrote:
>> Le 29/06/2013 09:43, Anders Lund a écrit :
>>> That is what I sometimes do, for example by tagging a whole album with
>>> some
>>> tags.
>>>
>>> What I thought of was something like an external process and some que
>>> management.
>>
>> then use exiftools...
>>
>> I don't know if digikam batch allows tagging
>
> I really like using digikam :-)
>
> --
> Anders
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Remco Viëtor]

On Saturday 29 June 2013 09:59:42 Anders Lund wrote:
> On Lørdag den 29. juni 2013 09:54:15, jdd wrote:
> > Le 29/06/2013 09:43, Anders Lund a écrit :
> > > That is what I sometimes do, for example by tagging a whole album
with
> > > some
> > > tags.
> > >
> > > What I thought of was something like an external process and some que
> > > management.

Well, if Digikam would run a large tag update as a background job, you
might get in trouble if you do anything with one of the files concerned
while the queue is running.

For instance: all images in an album need a particular tag (e.g. a
location), and part of them need other tags. You start the queue for a
location tag, and then for the other tags. It's very easy to get in a
situation where a file gets read by the 1st job, then read and modified  by
the second, then modified by the first  => loss of tags added in second
queue. It's as easy to lose the tags from the first queue.

Another situation: you start a queue, then decide to edit an image rather
far ahead, so that the editor has opened the file when the tag queue has to
modify the tags. How do you want to handle that?

There are ways to handle this kind of things, but they are not easy to
implement 100% correct, and have an impact on speed.

Personally I prefer a conservative approach, even if it includes waiting a
bit every now and then, over a method that seems faster, but has way more
risk of data loss.

Remco
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Anders Lund]

On Lørdag den 29. juni 2013 10:51:03, Remco Viëtor wrote:

> On Saturday 29 June 2013 09:59:42 Anders Lund wrote:
> > On Lørdag den 29. juni 2013 09:54:15, jdd wrote:
> > > Le 29/06/2013 09:43, Anders Lund a écrit :
> > > > That is what I sometimes do, for example by tagging a whole album
>
> with
>
> > > > some
> > > > tags.
> > > >
> > > > What I thought of was something like an external process and some que
> > > > management.
>
> Well, if Digikam would run a large tag update as a background job, you
> might get in trouble if you do anything with one of the files concerned
> while the queue is running.
>
> For instance: all images in an album need a particular tag (e.g. a
> location), and part of them need other tags. You start the queue for a
> location tag, and then for the other tags. It's very easy to get in a
> situation where a file gets read by the 1st job, then read and modified  by
> the second, then modified by the first  => loss of tags added in second
> queue. It's as easy to lose the tags from the first queue.
>
> Another situation: you start a queue, then decide to edit an image rather
> far ahead, so that the editor has opened the file when the tag queue has to
> modify the tags. How do you want to handle that?
>
> There are ways to handle this kind of things, but they are not easy to
> implement 100% correct, and have an impact on speed.
>
> Personally I prefer a conservative approach, even if it includes waiting a
> bit every now and then, over a method that seems faster, but has way more
> risk of data loss.

In case that is the best choice, I think it would be good to make it more
obvious that digikam is busy. In my case, I try to move on becase it looks
like I can do so in the ui.

--
Anders
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Le 29/06/2013 10:51, Remco Viëtor a écrit :


> Personally I prefer a conservative approach, even if it includes waiting a
> bit every now and then, over a method that seems faster, but has way more
> risk of data loss.

sure.

for bulk work, I use small scripts used before digikam come

jdd


--
http://www.dodin.org
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Le 29/06/2013 10:55, Anders Lund a écrit :

> In case that is the best choice, I think it would be good to make it more
> obvious that digikam is busy. In my case, I try to move on becase it looks
> like I can do so in the ui.
>
usually there is a progress bar, but the location is not consitent
depending on digikam versions

jdd

--
http://www.dodin.org
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Anders Lund]

On Lørdag den 29. juni 2013 10:59:08, jdd wrote:
> Le 29/06/2013 10:55, Anders Lund a écrit :
> > In case that is the best choice, I think it would be good to make it more
> > obvious that digikam is busy. In my case, I try to move on becase it looks
> > like I can do so in the ui.
>
> usually there is a progress bar, but the location is not consitent
> depending on digikam versions

I have that, but it does not indicate that the program is unusable to me :)

--
Anders
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Le 29/06/2013 11:02, Anders Lund a écrit :
> On Lørdag den 29. juni 2013 10:59:08, jdd wrote:

>> usually there is a progress bar, but the location is not consitent
>> depending on digikam versions
>
> I have that, but it does not indicate that the program is unusable to me :)
>

yes.

When I make video, and I want a dvd it's the complete computer that is
unavailable for several hours.

buy a second one?

and when I have to clean the house, it's me that is not available for
the computer :-)

there are always trade offs to hard works!

I use to say than when you buy a twice more powerfull computer, you
wait finally twice more. Why? because you ask it fourth the load...

jdd
--
http://www.dodin.org
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Anders Lund]

On Lørdag den 29. juni 2013 11:06:16, jdd wrote:

> Le 29/06/2013 11:02, Anders Lund a écrit :
> > On Lørdag den 29. juni 2013 10:59:08, jdd wrote:
> >> usually there is a progress bar, but the location is not consitent
> >> depending on digikam versions
> >
> > I have that, but it does not indicate that the program is unusable to me
> > :)
>
> yes.
>
> When I make video, and I want a dvd it's the complete computer that is
> unavailable for several hours.

The video editing application does not leave you in doubt.

After applying tags, digikam for some reason replaces all the thumbnails, I
wonder why?

--
Anders
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Boudewijn]

On Saturday 29 June 2013 09:43:29 Anders Lund wrote:

> On Lørdag den 29. juni 2013 15:13:40, Simon Cropper wrote:
> > What he would like is the ability to write metadata to his files while
> > still being able to do other stuff (e.g. edit another image, browse
> > albums, etc).
> >
> > Usually if you save the metadata for a single file it takes less than a
> > second but if you are tagging hundreds of files it could take longer.
> > Personally the longest wait I have had is ~5-10 seconds to save data to
> > ~40 files. I suppose it is how you work and how many files you are
> > manipulating at one time.
>
> That is what I sometimes do, for example by tagging a whole album with some
> tags.
>
> What I thought of was something like an external process and some que
> management.

Thanks for bringing this up! I have seen the issue mentioned now and again on

the list, but it (still) is not completely clear to me and I hesitated to ask
(almost) the same question again.

Is Anders' goal reachable by turning off writing meta data to file while
actively tagging photos, then turn it on and sync the database to the files? I
have not yet dared to try it out, fearing to lose tags in the process.

What I had in mind is:
- write metadata to database only (leave the files as they are)
- when I'm done, tick "write meta data to file"
- chose the option to sync the database
- walk away from my computer

The advantages seem to me:
- I can work on tags without being interrupted
- writing of tags might be more efficient, because each file (might) only need
to be opened once (instead of for each tag I assign in a row)
- the actual writing of the tags is done at a time that I'm not waiting for
the results

Are my assumptions correct?

Best regards,

Boudewijn

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Elle Stone]

On 6/29/13, Boudewijn <[hidden email]> wrote:

> Is Anders' goal reachable by turning off writing meta data to file while
> actively tagging photos, then turn it on and sync the database to the files?
> I
> have not yet dared to try it out, fearing to lose tags in the process.
>
> What I had in mind is:
> - write metadata to database only (leave the files as they are)
> - when I'm done, tick "write meta data to file"
> - chose the option to sync the database
> - walk away from my computer
>
> The advantages seem to me:
> - I can work on tags without being interrupted
> - writing of tags might be more efficient, because each file (might) only
> need
> to be opened once (instead of for each tag I assign in a row)
> - the actual writing of the tags is done at a time that I'm not waiting for
>

I'm not sure exactly what Ander's goals are, but you just described
exactly what I do when I'm tagging. I go to the
settings/configure/metadata panel and *un*check write to the image, or
sidecar in my case. Then I tag. The tags are only written to the
database. When I'm done tagging I re-enable writing to the sidecar
files (the image files are all read-only) and sync the sidecars with
the database. It's cumbersome and I wish there was an option to make
it happen automatically, just as you described, so there was just one
"sync now" button to push instead of disabling/re-enabling writing to
the files.

Elle
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Anders Lund]

On Lørdag den 29. juni 2013 19:31:46, Elle Stone wrote:

> On 6/29/13, Boudewijn <[hidden email]> wrote:
> > Is Anders' goal reachable by turning off writing meta data to file while
> > actively tagging photos, then turn it on and sync the database to the
> > files? I
> > have not yet dared to try it out, fearing to lose tags in the process.
> >
> > What I had in mind is:
> > - write metadata to database only (leave the files as they are)
> > - when I'm done, tick "write meta data to file"
> > - chose the option to sync the database
> > - walk away from my computer
> >
> > The advantages seem to me:
> > - I can work on tags without being interrupted
> > - writing of tags might be more efficient, because each file (might) only
> > need
> > to be opened once (instead of for each tag I assign in a row)
> > - the actual writing of the tags is done at a time that I'm not waiting
> > for
>
> I'm not sure exactly what Ander's goals are, but you just described
> exactly what I do when I'm tagging. I go to the
> settings/configure/metadata panel and *un*check write to the image, or
> sidecar in my case. Then I tag. The tags are only written to the
> database. When I'm done tagging I re-enable writing to the sidecar
> files (the image files are all read-only) and sync the sidecars with
> the database. It's cumbersome and I wish there was an option to make
> it happen automatically, just as you described, so there was just one
> "sync now" button to push instead of disabling/re-enabling writing to
> the files.

Now that sounds perfect, the only missing thing is that it should be
automatic, or at least the switch should be available in the main ui.

--
Anders
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Jean-François Rabasse-2]

On Sat, 29 Jun 2013, Elle Stone wrote:

> I'm not sure exactly what Ander's goals are, but you just described
> exactly what I do when I'm tagging. I go to the
> settings/configure/metadata panel and *un*check write to the image, or
> sidecar in my case. Then I tag. The tags are only written to the
> database. When I'm done tagging I re-enable writing to the sidecar
> files (the image files are all read-only) and sync the sidecars with
> the database. It's cumbersome and I wish there was an option to make
> it happen automatically, just as you described, so there was just one
> "sync now" button to push instead of disabling/re-enabling writing to
> the files.

I think many users work more or less that way, for already stated
reasons. (The major reason beeing to avoid the overload raised by the
update process.)
And this is militant in favour of delayed update. Also it should be
kept in mind that Digikam doesn't use/need metadata from files (except
in some special cases, images import or new images scan at startup,
where metadata will be extracted once to setup an initial database state.)
So, clearly, files or sidecars update is never urgent, as long as the
database reflects the current state.

Remco pointed out the inconvenients of doing delayed asynchronous updates.
Yes, it's a difficult to implement stuff, if one wants to avoid data losses,
data overwrites, deadlocks, etc. So delayed update should probably take
place at end of session, not while working.

There's however a small inconvenient : when updating at once (i.e. the
current implementation), concerned images are known for it's the current
selection to which tags, titles, rating, where applied.
When delaying update to end of session, this information is lost and all
what could be done is to issue updates for all images.
But in case of a collection of, say, 10000 images, with 20 or 30 having
metadata modified, the whole process is far from being optimal.

A possible solution could be to associate with each image some kind of
state flag, a « dirty bit », that would be set when DK updates the database
with metadata (after the « Apply » click.)
So, when exiting Digikam, the user could be warned. « Warning: you have
modified metadata. Should the images/sidecars be updated now ? Y/N »
And only the concerned images could be processed for update.
(And, of course, have their flag reset.)


Jean-François
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Anders Lund]

On Søndag den 30. juni 2013 14:17:58, Jean-François Rabasse wrote:

> On Sat, 29 Jun 2013, Elle Stone wrote:
> > I'm not sure exactly what Ander's goals are, but you just described
> > exactly what I do when I'm tagging. I go to the
> > settings/configure/metadata panel and *un*check write to the image, or
> > sidecar in my case. Then I tag. The tags are only written to the
> > database. When I'm done tagging I re-enable writing to the sidecar
> > files (the image files are all read-only) and sync the sidecars with
> > the database. It's cumbersome and I wish there was an option to make
> > it happen automatically, just as you described, so there was just one
> > "sync now" button to push instead of disabling/re-enabling writing to
> > the files.
>
> I think many users work more or less that way, for already stated
> reasons. (The major reason beeing to avoid the overload raised by the
> update process.)
> And this is militant in favour of delayed update. Also it should be
> kept in mind that Digikam doesn't use/need metadata from files (except
> in some special cases, images import or new images scan at startup,
> where metadata will be extracted once to setup an initial database state.)
> So, clearly, files or sidecars update is never urgent, as long as the
> database reflects the current state.

I don't know if "delayed" is any good.

In my workflow, I typically import, select, tag, process and export images.
When exporting, having those metadata writte to the image is smart.

Writing meta data to images, I assume, is a question of copying data from the
digikam db into the image or sidecar file. So everytime an images data
changes, it should go into the queue, unless it is allready there. Now if the
actual data writing is done by a spearate process, what is the problem? That
process must then be throtteled, so that it does not take the system down.
 
When exporting, it could be checked if the image is in the metadata writing
queue, so it can be processed there prior to exporting.

--
Anders
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Jean-François Rabasse-2]

Hi Anders,

On Sun, 30 Jun 2013, Anders Lund wrote:

> I don't know if "delayed" is any good.
>
> In my workflow, I typically import, select, tag, process and export images.
> When exporting, having those metadata writte to the image is smart.

I said delayed, or postponed, in a general case because metadata in images
files is of no use for Digikam, so not urgent, except when exporting images.
This is right.

So, yes, if you do all in one session and want to have metadata in your
exported images, update should have been done before.

> Writing meta data to images, I assume, is a question of copying data from the
> digikam db into the image or sidecar file. So everytime an images data
> changes, it should go into the queue, unless it is allready there. Now if the
> actual data writing is done by a spearate process, what is the problem? That
> process must then be throtteled, so that it does not take the system down.

The problem would be that this leads to a very complicated software
architecture.
See the detailed message from Remco Viëtor, on June 29th, 10:51.

But anyway, it's certainly up to Digikam devel. team to comment those
different options, update at once, update at exit, update asynchronously.

Regards,
Jean-François
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users