[digiKam-users] Hierarchical tags, how are they stored?

[woenx]

I have been sorting some tags, as usual, and I noticed that some tags
containing hundreds of pictures were modified rather quickly (when I should
take a few minutes over the network). That let me to check the metadata, and
I observed that the full hierarchy of tags is not being written to the xmp
information, at least in some cases.

For instance, let's say I have the following hierarchy:

Canada/Ontario/Toronto

And many of the pictures in there only have the "Toronto" tag (despite being
part of the hierarchy). When scanning these pictures in a clean digikam
database, the hierarchy won't be reproduced. "Toronto" will just appear at
the root of the tag tree.

But in many cases, the xmp metadata indicates that the full hierarchy has
been saved into the metadata itself. I think it depends whether its parent
tag was explicitly used in that picture, but I am not sure.

However, I just tried right now, if I use 'Item/Write metadata to file" on
one of these pictures without the hierarchy, the metadata gets updated and
now it includes everything. So, is this a bug where the metadata is not
written to the picture when tags are reorganized (drag and drop to put them
as subtags)?



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[woenx]

Btw, I got suspicious when I dragged and dropped some tags into a hierarchy,
and digikam then warned that a large number of pictures were going to be
modified (that's fine), but the amount of pictures it mentioned was lower
than the total amount of pictures contained in the hierarchy I just moved.
So I suspect that only pictures that use the tag immediately next to the
parent tag are updated.

(e.g. Imagine I have an existing hierarchy Ontario/Toronto, and lots of
pictures in there are only tagged with "Toronto", but are somehow aware of
the hierarchy. Then I move "Ontario" over "Canada", so the hierarchy is now
Canada/Ontario/Toronto, but those pictures containing just "Toronto" are not
modified, only the few ones that contained "Ontario/Toronto" are updated).

I am sorry if it's a bit abstract, but I don't know if this a bug or normal
behavior.



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Noeck]

> For instance, let's say I have the following hierarchy:
>
> Canada/Ontario/Toronto

In my images this is saved in XMP metadata as

digiKam-Schema:
        Tags List = Canada/Ontario/Toronto

lr:
        hierarchicalSubject = Canada|Ontario|Toronto

mediapro:
        Catalog Sets = Canada|Ontario|Toronto

Microsoft Photo:
        Last Keyword XMP = Canada/Ontario/Toronto

Dublin Core:
        Subject = Toronto

So only the Subject does not store the hierarchy. "categories" even has
the hierarchy as an XML tree.

All these fields above can have multiple keywords as a comma separated
list. E.g. the value Canada, Canada/Ontario/Toronto means this tag
hierarchy:

[x] Canada
    [ ] Ontario
        [x] Toronto

I have never seen any inconsistency. If your tag trees are saved to XMP
inconsistently, it would make sense to describe what's wrong in more detail.

If you have "Toronto" in a separate root entry, you probably stored it
either in digikam on the root level additionally or another application
added a keyword "Toronto" which is interpreted by digikam as keyword
without parent (root level).

HTH
Joram

[Noeck]

Am 05.11.19 um 20:08 schrieb woenx:
> Btw, I got suspicious when I dragged and dropped some tags into a hierarchy,

I made some bad experience when moving tags by drag and drop some years
ago. I can't recall exactly but since then I do this to move tags:

1. Add a new tag (with the same name) in the hierarchy where I want it
2. Filter by the old tag
3. Add the new tag to all of those filtered images
4. Unselect the old tag on all those images
5. Check that no images show up if I select the old tag
6. Delete the old tag

It's a bit more complicated than drag and drop but it works reliably.

Best,
Joram

[woenx]

Thanks for the reply.

for the sake of simplicity, let's just focus on lighroom's XMP metadata:

lr:
        hierarchicalSubject = Canada|Ontario|Toronto

So I have moved pictures that had

        hierarchicalSubject = Ontario|Toronto

where some of them were:

    [ ] Ontario
        [x] Toronto

and some were:

    [x] Ontario
        [x] Toronto

(I don't mind, as long as they appear in the hierarchy).

Then, I created a new root tag called "Canada" (e.g. because I have pictures
from Quebec, and I want to incorporate everything into the "Canada" tag). So
on the tag tree I drag "Ontario" over "Canada". I get warned that xxx
pictures will be modified, etc, that's fine (but the number is lower than
the number of pictures that should be affected). Anyway, the new hierarchy
should look like this:

lr:
        hierarchicalSubject = Canada|Ontario|Toronto

Pictures that contained
   
 [x] Ontario
      [x] Toronto

have been successfully updated and incorporate the full hierarchy in the xmp
metadata ("hierarchicalSubject = Canada|Ontario|Toronto"). But pictures that
just contained

[ ] Ontario
    [x] Toronto

were not updated, and the metadata looks like this "lr: hierarchicalSubject
= Toronto". However, selecting any of these pictures and manually
"Item/Write metadata to file", writes the full metadata including lr:
hierarchicalSubject = Canada|Ontario|Toronto. So I personally think this is
a bug where the database and the xmp metadata are out of sync, and all the
pictures included in the hierarchy should have been automatically updated
with the new metadata.



Wait, but there is more. Let's place one of the pictures with just "lr:
hierarchicalSubject = Toronto" in a new folder, and tell digikam to start a
new database in that folder and just scan this picture. My common sense
tells me that there this hierarchy will be created:

[x] Toronto

Ok, makes sense. And now let's add a second picture, one with the full
metadata containing "lr: hierarchicalSubject = Canada|Ontario|Toronto". My
common sense tells me this will happen:

[x] Canada
    [x] Ontario
        [x] Toronto
[x] Toronto

But instead, I get this:

[x] Canada
    [x] Ontario
        [x] Toronto

So, the full hierarchy has been recreated correctly! How does digikam know
that the pictures just containing the "Toronto" tag, without hierarchy,
should be placed within Canada/Ontario?



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[woenx]

Ok, last post, I promise.

I noticed that these pictures that don't get update, still contain tags from
the previous hierarchy.

For instance, Let's say I have a set of pictures like this:

[ ] Kanada
    [ ] Ontario
        [x] Toronto

But I realize my mistake, create a new hierarchy with the correct spelling
"Canada", and I drag and drop "Ontario" into the new "Canada". So now it
should be:

[ ] Canada
    [ ] Ontario
        [x] Toronto

and appears as such in the database. But if you check any of the picture's
metadata, it still appears as:

lr:     hierarchicalSubject = Kanada|Ontario|Toronto

This is clearly a bug. If I manually write the metadata to file, it finally
gets updated to:

lr:     hierarchicalSubject = Canada|Ontario|Toronto

So apparently metadata is not written to files if the immediate parent tag
in the hierarchy has not been updated for any specific file?



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Noeck]

> For instance, Let's say I have a set of pictures like this:
>
> [ ] Kanada
>     [ ] Ontario
>         [x] Toronto
>
> But I realize my mistake

Which is not a mistake in German, btw. ☺

> So apparently metadata is not written to files if the immediate parent tag
> in the hierarchy has not been updated for any specific file?

That's probably what I encountered and mentioned in a previous mail.

If that is really the problem, it should be possible to solve it and
trigger a metadata sync. But I see one reason not to do it: if many
images were tagged with the old tag, all of them need to be written on
disk again. With potentially 100k or millions of images, this could be a
very time consuming operation triggered by a seemingly simple drag-and-drop.

Best,
Joram

[woenx]

Noeck wrote
>> For instance, Let's say I have a set of pictures like this:
>>
>> [ ] Kanada
>>     [ ] Ontario
>>         [x] Toronto
>>
>> But I realize my mistake
>
> Which is not a mistake in German, btw. ☺

Haha, true.

Yes, now I am making sure to run a "write metadata" sync after each
movement. And yes, that may imply thousands of pictures, and take a very
long time, but the user is already warned by digikam (if it changes more
than 100 pictures or so), so is doing that under his own responsibility.
Alternatively, if you commonly do these kind hierarchy changes, using XMP
sidecars instead of directly writing to files would be much faster and
safer. I have not had any picture corrupted by digikam yet, though, so I
trust it (I also have regular backups, so it's fine).

Anyway, the changes I am doing today (changing the root of some tags to the
country name) is something I hope to do just once. Unless someday I want to
group countries under continents or something ¯\_(ツ)_/¯



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Noeck]

You are right about the warning and the user’s responsibility IMHO.

> Unless someday I want to group countries under continents or something ¯\_(ツ)_/¯

I group them under "places" to have a separate tree for other tagging
criteria :)

Best,
Joram

[woenx]

I still dream about the day that places are a separate panel in digikam,
automatically populated by reverse geocoding pictures with GPS coordinates
during import and stored in a local database, so I don't have to manually
tag places in pictures anymore.

(This is how Google Photos actually works, although you cannot browse a
location tree, just search locations)



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html