digiKam ate my tags: bug, feature, or user-error?

[Elle Stone]

Hi All,

Awhile back I was merrily tagging along with digiKam 1.7 (Ubuntu 10.10). The next day I realized I'd lost most of the tags for around 350 images. Couldn't figure out what happened, forgot about it.

A couple days ago I lost most of the tags from 1,100 images. Fortunately, this time I had back-ups of the images "before tag loss". I'm not sure if I was using 1.7, 1.9, or 2.0, because in an effort to get the elusive exiv2 0.21 installed and the digikam libkexiv2 wrapper using 0.21, I've done a lot of compiling from svn in the last couple of days. At this moment I have digiKam 1.9 as "alongside" in /usr/local/bin, and then digiKam 2.0 beta in /usr/bin (alas the "alongside" part isn't working, so to switch versions means swapping out .kde folders).

Here's how the tags disappeared:

First, my tags are all arranged in hierarchies under the top categories "who, what, where, why, how" with also an assortment of other tags. The "who" tags have subtags "friends," "family," & "strangers", with sub-subtags for individuals. The "where" tag hierarchy has subtags by country/state/province/city/sublocations.

One of the "why" subtags is "family and friends". The 1,100 "family and friends" images had all of the people already tagged and also all of these images had the appropriate "where" tags.

I created a new tag with some sub-tags, selected all of the "friends and family" images, applied the new tag. digiKam was configured to write tags and captions to the images. Which it did. Immediately. Usually I disable the "write metadata" option until I'm ready to write a whole bunch of metadata at once, at the end of a tagging session, because otherwise digiKam slows down a bit when doing a lot of tagging. But this time when I started digiKam, I forgot to check the metadata writing settings before beginning to tag images. I wish digiKam had the option to "only write metadata to the images upon specific request". But it doesn't (unless I missed something).

Anyway, having applied the new tag to my 1,100 images, I shut down digiKam. The next day I started digiKam and created a new database, to my surprise none of the friends and family images had "who" or "where" tags. Why a new database? well, 2.0 can't use old databases, and also I'd already saved my tags to the image metadata, or so I thought.

So I started testing in 1.9 and then in 2.0. Create a test database and tag away. Then create a new tag and apply it to all your images (with write metadata enabled, of course). Any tag not shared in common with all the images (where, who) is lost from the image's own metadata. But it is still in the database. Check a single image's metadata in digiKam - you can see that the tags are not there, and exiftool confirms. Look at the tags on the thumbnail display - the tags are all there - in the database but not in the image itself.

Writing the new tag erased all the old tags. In IMatch, this behavior was an option. You could also choose to "add" tags rather than "replace" tags. I always assumed that digiKam always replaced ALL the tags in the image with ALL the tags in the database, old tags and tags-just-applied. But it seems that is not true.

Now leaving digiKam open, go do something else in digiKam. Just tag a single image. Now all the previously tagged images still don't have the old tags in their metadata, only the new tag and any in-common tags. But now select all the images and write metadata to the images from the Images drop-down menu. And presto, the tags are now in the image as well as in the database. At least in 1.9. I can't get 2.0 to write out the old tags back to the images, which seems to be a really bad bug, unless my installation of 2.0 is not quite right (very possible).

Hope that is not too confusing. I've checked several times. It is repeatable.

So - feature? bug? borked installation? user error? (in which case please, please tell me the right way to apply new tags)

Elle

[pshute]

Interesting top level tags, Elle. I'm just curious about what kind of photo qualifies to be tagged both "why" and "family and friends". In fact, I'm not sure I'd have many photos at all that could be tagged "why".

You seem to be finding a lot of good bugs.

> -----Original Message-----
> From: Elle Stone [mailto:[hidden email]]
> Sent: Thursday, 27 January 2011 8:46 AM
> To: [hidden email]
> Subject: [Digikam-users] digiKam ate my tags: bug, feature,
> or user-error?

> One of the "why" subtags is "family and friends".
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Elle Stone]

Hi Peter,

Only 4 subtags for "why", two of which are very "home life" oriented -
"family and friends", "vacations". The other reasons are "just
because" pictures where the goal is to satisfy my own artistic
inclinations, feeble as they may be; and pictures "in service of" some
other goal, like product photography or documenting some event or
another. Culling and rating images is much easier if I first think
"why did I take this picture?" Helps to sort them out into "why"
groups before getting started.

How do you tag?

I don't know if I found a bug or not - waiting for input from others
before making official bug report.

Regards,
Elle
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[pshute]

I didn't think to create top level tags like yours, but if I did then I would only have "who", "what" and "where". Most of my photos would be under the what tag, in a branch of tags called birds, which has the letters a to z as tags under it, and each species is under the appropriate letter, just to keep it managable.

Under people, I have a family tag, under which I have the family names, similar under friends, etc.

> -----Original Message-----
> From: Elle Stone [mailto:[hidden email]]
> Sent: Thursday, 27 January 2011 10:12 AM
> To: digiKam - Home Manage your photographs as a professional
> with the power of open source
> Subject: [Digikam-users] Re: digiKam ate my tags: bug,
> feature, or user-error?
>
> Hi Peter,
>
> Only 4 subtags for "why", two of which are very "home life" oriented -
> "family and friends", "vacations". The other reasons are "just
> because" pictures where the goal is to satisfy my own artistic
> inclinations, feeble as they may be; and pictures "in service of" some
> other goal, like product photography or documenting some event or
> another. Culling and rating images is much easier if I first think
> "why did I take this picture?" Helps to sort them out into "why"
> groups before getting started.
>
> How do you tag?
>
> I don't know if I found a bug or not - waiting for input from others
> before making official bug report.
>
> Regards,
> Elle
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Andrew Goodbody]

On 26/01/11 21:46, Elle Stone wrote:
> So - feature? bug? borked installation? user error? (in which case please,
> please tell me the right way to apply new tags)
>
> Elle
>

I just came across this behaviour in 1.8.0 (or close check out from SVN)
and I consider it to be a bug, so go ahead and raise a report. Looking
at my photos I would say that this is quite a long standing bug as it
seems to have affected photos I tagged with 1.3.0 or possibly an earlier
version, I'm not sure which.

In summary if you select a number of photos that have differing sets of
tags already applied and add a new tag, with metadata written out to
images by default, then only tags that are common to all photos selected
will be written out to the photos, tags not in common will be removed.
All tags are present, original and new as expected, in the database.

Andrew
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Martin (KDE)]

Am Samstag, 29. Januar 2011 schrieb Andrew Goodbody:

> On 26/01/11 21:46, Elle Stone wrote:
> > So - feature? bug? borked installation? user error? (in which
> > case please, please tell me the right way to apply new tags)
> >
> > Elle
>
> I just came across this behaviour in 1.8.0 (or close check out from
> SVN) and I consider it to be a bug, so go ahead and raise a
> report. Looking at my photos I would say that this is quite a long
> standing bug as it seems to have affected photos I tagged with
> 1.3.0 or possibly an earlier version, I'm not sure which.
>
> In summary if you select a number of photos that have differing
> sets of tags already applied and add a new tag, with metadata
> written out to images by default, then only tags that are common
> to all photos selected will be written out to the photos, tags not
> in common will be removed. All tags are present, original and new
> as expected, in the database.

I just tested this with my photos (digikam 1.7) and I have the same
behaviour. Tags not set for all selected photos are removed from XMP
data embedded in the files. Database is OK. I just synced all metadata
from DB to files (takes some time, but you never know).

Martin

>
> Andrew
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

Martin,

In XMP metadata, do you take a look in digiKam XMP namespace, where
whole tags path are registered ?

Gilles Caulier

2011/1/29 Martin (KDE) <[hidden email]>:

> Am Samstag, 29. Januar 2011 schrieb Andrew Goodbody:
>> On 26/01/11 21:46, Elle Stone wrote:
>> > So - feature? bug? borked installation? user error? (in which
>> > case please, please tell me the right way to apply new tags)
>> >
>> > Elle
>>
>> I just came across this behaviour in 1.8.0 (or close check out from
>> SVN) and I consider it to be a bug, so go ahead and raise a
>> report. Looking at my photos I would say that this is quite a long
>> standing bug as it seems to have affected photos I tagged with
>> 1.3.0 or possibly an earlier version, I'm not sure which.
>>
>> In summary if you select a number of photos that have differing
>> sets of tags already applied and add a new tag, with metadata
>> written out to images by default, then only tags that are common
>> to all photos selected will be written out to the photos, tags not
>> in common will be removed. All tags are present, original and new
>> as expected, in the database.
>
> I just tested this with my photos (digikam 1.7) and I have the same
> behaviour. Tags not set for all selected photos are removed from XMP
> data embedded in the files. Database is OK. I just synced all metadata
> from DB to files (takes some time, but you never know).
>
> Martin
>
>>
>> Andrew
>> _______________________________________________
>> Digikam-users mailing list
>> [hidden email]
>> https://mail.kde.org/mailman/listinfo/digikam-users
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Elle Stone]

Hi Andrew and Martin,

Thanks! for verifying.

I just posted a bug report, "Bug 264745 - When adding new tag to
multiple images, tags not in common are removed from metadata, though
still present in database"

Elle
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Elle Stone]

Hi Gilles,

While investigating this problem, not only did was I able to watch the
tags disappear from the metadata XMP hierarchicalSubject, etc in the
digiKam side window, then reappear after synching up the database with
the images, but also confirmed independently with exiftool that the
tags were being removed while tagging, and then rewritten when
synching.

Elle
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Martin (KDE)]

Am Samstag, 29. Januar 2011 schrieb Gilles Caulier:
> Martin,
>
> In XMP metadata, do you take a look in digiKam XMP namespace, where
> whole tags path are registered ?

Gilles,

I looked in the "Tags List" as well as in hierarchicalSubject, and
both were not correct (differ from the data in digikam DB and digikam
showed). After syncing metadata to file, they are correct now.

Martin

>
> Gilles Caulier
>
> 2011/1/29 Martin (KDE) <[hidden email]>:
> > Am Samstag, 29. Januar 2011 schrieb Andrew Goodbody:
> >> On 26/01/11 21:46, Elle Stone wrote:
> >> > So - feature? bug? borked installation? user error? (in which
> >> > case please, please tell me the right way to apply new tags)
> >> >
> >> > Elle
> >>
> >> I just came across this behaviour in 1.8.0 (or close check out
> >> from SVN) and I consider it to be a bug, so go ahead and raise
> >> a report. Looking at my photos I would say that this is quite a
> >> long standing bug as it seems to have affected photos I tagged
> >> with 1.3.0 or possibly an earlier version, I'm not sure which.
> >>
> >> In summary if you select a number of photos that have differing
> >> sets of tags already applied and add a new tag, with metadata
> >> written out to images by default, then only tags that are common
> >> to all photos selected will be written out to the photos, tags
> >> not in common will be removed. All tags are present, original
> >> and new as expected, in the database.
> >
> > I just tested this with my photos (digikam 1.7) and I have the
> > same behaviour. Tags not set for all selected photos are removed
> > from XMP data embedded in the files. Database is OK. I just
> > synced all metadata from DB to files (takes some time, but you
> > never know).
> >
> > Martin
> >
> >> Andrew
> >> _______________________________________________
> >> Digikam-users mailing list
> >> [hidden email]
> >> https://mail.kde.org/mailman/listinfo/digikam-users
> >
> > _______________________________________________
> > Digikam-users mailing list
> > [hidden email]
> > https://mail.kde.org/mailman/listinfo/digikam-users
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

ok, so, there is an error in metdatahub class in digiKam core. We must take look

Gilles

2011/1/29 Martin (KDE) <[hidden email]>:

> Am Samstag, 29. Januar 2011 schrieb Gilles Caulier:
>> Martin,
>>
>> In XMP metadata, do you take a look in digiKam XMP namespace, where
>> whole tags path are registered ?
>
> Gilles,
>
> I looked in the "Tags List" as well as in hierarchicalSubject, and
> both were not correct (differ from the data in digikam DB and digikam
> showed). After syncing metadata to file, they are correct now.
>
> Martin
>
>>
>> Gilles Caulier
>>
>> 2011/1/29 Martin (KDE) <[hidden email]>:
>> > Am Samstag, 29. Januar 2011 schrieb Andrew Goodbody:
>> >> On 26/01/11 21:46, Elle Stone wrote:
>> >> > So - feature? bug? borked installation? user error? (in which
>> >> > case please, please tell me the right way to apply new tags)
>> >> >
>> >> > Elle
>> >>
>> >> I just came across this behaviour in 1.8.0 (or close check out
>> >> from SVN) and I consider it to be a bug, so go ahead and raise
>> >> a report. Looking at my photos I would say that this is quite a
>> >> long standing bug as it seems to have affected photos I tagged
>> >> with 1.3.0 or possibly an earlier version, I'm not sure which.
>> >>
>> >> In summary if you select a number of photos that have differing
>> >> sets of tags already applied and add a new tag, with metadata
>> >> written out to images by default, then only tags that are common
>> >> to all photos selected will be written out to the photos, tags
>> >> not in common will be removed. All tags are present, original
>> >> and new as expected, in the database.
>> >
>> > I just tested this with my photos (digikam 1.7) and I have the
>> > same behaviour. Tags not set for all selected photos are removed
>> > from XMP data embedded in the files. Database is OK. I just
>> > synced all metadata from DB to files (takes some time, but you
>> > never know).
>> >
>> > Martin
>> >
>> >> Andrew
>> >> _______________________________________________
>> >> Digikam-users mailing list
>> >> [hidden email]
>> >> https://mail.kde.org/mailman/listinfo/digikam-users
>> >
>> > _______________________________________________
>> > Digikam-users mailing list
>> > [hidden email]
>> > https://mail.kde.org/mailman/listinfo/digikam-users
>>
>> _______________________________________________
>> Digikam-users mailing list
>> [hidden email]
>> https://mail.kde.org/mailman/listinfo/digikam-users
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Andrew Goodbody]

On 29/01/11 13:57, Elle Stone wrote:
> I just posted a bug report, "Bug 264745 - When adding new tag to
> multiple images, tags not in common are removed from metadata, though
> still present in database"
>
> Elle

Thanks for reporting this. It helped me to understand what was happening
to my photos. I added my votes to the bug you raised but I don't know if
Gilles et al take any notice of the votes or not.

Andrew
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

Andrew,

Probably the problem is somewhere here :

http://lxr.kde.org/source/extragear/graphics/digikam/digikam/metadatahub.cpp#410

But i cannot see what's wrong exactly...

Gilles Caulier

2011/1/29 Andrew Goodbody <[hidden email]>:

> On 29/01/11 13:57, Elle Stone wrote:
>> I just posted a bug report, "Bug 264745 - When adding new tag to
>> multiple images, tags not in common are removed from metadata, though
>> still present in database"
>>
>> Elle
>
> Thanks for reporting this. It helped me to understand what was happening
> to my photos. I added my votes to the bug you raised but I don't know if
> Gilles et al take any notice of the votes or not.
>
> Andrew
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Andrew Goodbody]

On 29/01/11 19:30, Gilles Caulier wrote:
> Andrew,
>
> Probably the problem is somewhere here :
>
> http://lxr.kde.org/source/extragear/graphics/digikam/digikam/metadatahub.cpp#410
>
> But i cannot see what's wrong exactly...

Were you talking to me? Sorry but I don't know the digikam source nor
even c++ particularly. However a quick glance suggests that line 498
must be going wrong. hasTag must, for each image, work on the merged
original tags of that image and any changes (additions or deletions)
that have been made. It seems that it is only taking notice of the
changes made and not the original tags.

I do wonder if the bug is somewhat deeper. In the right sidebar there
does not seem to be any indication for tags that are in some of the
images selected but not others. I would have expected along with the
selected and deselected tag states there would be a third in-between
state that indicated that only some of the selected images had this tag
applied. So for this third state, when the tags were written out, such a
tag would be unchanged in each image. However this may just be a
separate UI issue and fixing hasTag be what is needed to fix the bug.

I have not actually checked whether tags that are in common with all
images but are not changed remain intact or not in the metadata.

Andrew
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Martin (KDE)]

Am Samstag, 29. Januar 2011 schrieb Andrew Goodbody:

> On 29/01/11 19:30, Gilles Caulier wrote:
> > Andrew,
> >
> > Probably the problem is somewhere here :
> >
> > http://lxr.kde.org/source/extragear/graphics/digikam/digikam/meta
> > datahub.cpp#410
> >
> > But i cannot see what's wrong exactly...
>
> Were you talking to me? Sorry but I don't know the digikam source
> nor even c++ particularly. However a quick glance suggests that
> line 498 must be going wrong. hasTag must, for each image, work on
> the merged original tags of that image and any changes (additions
> or deletions) that have been made. It seems that it is only taking
> notice of the changes made and not the original tags.
>
> I do wonder if the bug is somewhat deeper. In the right sidebar
> there does not seem to be any indication for tags that are in some
> of the images selected but not others. I would have expected along
> with the selected and deselected tag states there would be a third
> in-between state that indicated that only some of the selected
> images had this tag applied. So for this third state, when the
> tags were written out, such a tag would be unchanged in each
> image. However this may just be a separate UI issue and fixing
> hasTag be what is needed to fix the bug.

I am not really sure, but I think in one of the older digikam version
(may be 1.4 or 1.5) I had tags not common in all sellected photos were
checked and greyed out.

Martin

>
> I have not actually checked whether tags that are in common with
> all images but are not changed remain intact or not in the
> metadata.
>
> Andrew
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users