[Bug 220545] New: Cannot write XMP sidecar files

https://bugs.kde.org/show_bug.cgi?id=220545

           Summary: Cannot write XMP sidecar files
           Product: digikam
           Version: unspecified
          Platform: Ubuntu Packages
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: wishlist
          Priority: NOR
         Component: general
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]


Version:           1.0.0-rc (using KDE 4.3.2)
OS:                Linux
Installed from:    Ubuntu Packages

I would very much appreciate the possibility to write XMP data to a sidecar
file for import into Bibble. For me, I feel safer not touching my original
NEFs.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545


Jakob Malm <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |usability




--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=220545


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
          Component|general                     |Metadata
            Version|unspecified                 |1.0.0




--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=220545


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #1 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-01-29 18:19:20 ---
*** Bug 224784 has been marked as a duplicate of this bug. ***

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #2 from Jakob Malm <jakob malm gmail com>  2010-03-30 20:03:39 ---
If I understood Caulier, this should be implemented through libkexiv2, without
any change in digiKam. If someone could give me a hint as to where to start in
libkexiv2, I would be interested in getting my hands dirty...

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #3 from Jakob Malm <jakob malm gmail com>  2010-03-30 22:51:46 ---
Hmm... Looking through the KExiv2::load and save functions I think the support
for XMP sidecar files might already be there, because Exiv2::XmpSidecar
inherits from Exiv2::Image. So, by supplying a filename like photo.xmp, should
read from, or write to, an XMP sidecar file.

I think what needs to be done is rather provide a means to tell digiKam to read
from or write to XMP sidecar files, i.e. by supplying a filename that ends with
'.xmp'.

The following are ways to do this (both should be made available, in my
opinion):
1) An option in the metadata settings tab to have digikam always write XMP
sidecar files, or perhaps only for certain image formats (e.g. raw formats).
Not sure about reading...
2) Two commands in the menu (and contextual menu) "Read (write) metadata from
(to) XMP sidecar file". This approach would be the easiest to implement, and
the approach I would like to try first. The command would then just issue
KExiv2::load(xmpSidecarFilename) and use the loaded metadata for the photo, or
vice versa get the metadata from the photo and issue
KExiv2::save(xmpSidecarFilename). If there is already some metadata associated
with the photo, only the metadata that was loaded should be replaced.

What are your thoughts on this?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=220545


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #4 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-03-30 23:10:03 ---
First, watch the little 'k' ;-)
You need to implement sidecar support in libkexiv2, using libexiv2. It's just
our Qt/KDE layer on top of exiv2.

In digikam I see a number of different workflows, from ignoring all sidecars to
using them exclusively. We'd need user input about the workflow needs here.
Also we should keep in mind that Sidecars carry only XMP; how to handle
concurrently present Exif and IPTC in the picture itself? There are a few
situations to think of.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #5 from Jakob Malm <jakob malm gmail com>  2010-03-30 23:32:11 ---
Yeah, the little k is easy to miss... But I think in this case I have double
checked: KExiv2::load and save use instances of Exiv2::Image obtained from
Exiv2::ImageFactory::open(filename) for reading and writing. This is why it
seems to me that it will also read and write XMP sidecar files. Have you
experienced that this is not the case?

I think you're absolutely right about many different workflows with different
uses of XMP sidecars. This should definitely be checked somehow with users.

As I understand XMP, everything in EXIF and IPTC can be put in XMP, using the
corresponding namespaces. So this may not be a problem after all. But, as you
say, metadata can be found both in the image file itself, and in an
accompanying XMP sidecar file.

This is one reason to first implement a menu command solution, like the one I
suggested in method 2), above.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #6 from Gilles Caulier <caulier gilles gmail com>  2010-03-31 11:00:31 ---
>As I understand XMP, everything in EXIF and IPTC can be put in XMP, using the
>corresponding namespaces. So this may not be a problem after all. But, as you
>say, metadata can be found both in the image file itself, and in an
>accompanying XMP sidecar file.

Warning, one important part cannot be found in XMP : Exif makernotes. This is a
huge problem there, because these private data cannot be translate to user
readable xmp tags.

My proposal with Andreas Huggel (Exiv2 author, available in this room), is to
set a single XMP tag to host whole binary Exif makernote tag content as well.
He is not very convinced by this way, but at least, we not lost any import
camera maker data in XMP.

Look like ExifTool do something like that :

http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/XMP.html

... search "Markernote" section...

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=220545


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #7 from Jakob Malm <jakob malm gmail com>  2010-03-31 16:31:11 ---
> Warning, one important part cannot be found in XMP : Exif makernotes. This is a
> huge problem there, because these private data cannot be translate to user
> readable xmp tags.

Ah, yes, I remember having read something about that at some point. What's in
these makernotes?

Anyway, my main interest in getting XMP sidecar support (yes, I'm perhaps
selfish) is to make it easy to share metadata with Bibble 5 (both ways), so the
makernotes are of less importance to me.

I do realize that a perfect solution from the start would be the best, but
maybe I could first get reading and writing XMP sidecar files supported from a
menu command, and then we can see where we want to go from there?

I'm not sure where in digiKam to start coding, though...

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #8 from Jakob Malm <jakob malm gmail com>  2010-03-31 21:45:46 ---
Reading XMP sidecar files works in libkexiv2! I just did a quick test:

1. Select any photo.
2. From the menu: Image -> Metadata -> Import XMP...
3. Select a .xmp file with some tags

I imported a file with some exif:GPS*, lr:HierarchicalSubject, and dc:subject
tags, and they were all imported into the XMP metadata of the selected photo.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #9 from Andreas Huggel <ahuggel gmx net>  2010-04-01 03:33:09 ---
Exiv2 (that without the little 'k' ;) converts a number of Exif tags and IPTC
datasets to the XMP sidecar and back. Try it out with the exiv2 command line
utility.

Gilles is right about the makernote, that is lost. It's also true that I'm not
keen to dump the makernote as a large binary blob into an XMP property as
exiftool does. It is unlikely that any other application would ever be able to
deal with such a blob properly. Instead I suggested to write the _known_
makernote tags as readable properties in new custom namespaces (e.g., like
Xmp.Nikon3.WhiteBalance). But that's just an idea at this point, no such code
exists yet.

> maybe I could first get reading and writing XMP sidecar files supported from a
> menu command, and then we can see where we want to go from there?

I think that should be possible. In digiKam the main concern is about the
workflow and the options to control the handling of XMP sidecars. When it comes
to the contents of the sidecar files, you'll need to turn to libexiv2.

Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #10 from Jakob Malm <jakob malm gmail com>  2010-04-01 07:32:00 ---
Andreas, thanks for your reply!

(In reply to comment #9)
> Exiv2 (that without the little 'k' ;) converts a number of Exif tags and IPTC
> datasets to the XMP sidecar and back. Try it out with the exiv2 command line
> utility.
Yes, it's a very handy tool. I guess you mean "... IPTC datasets _in image
files_ ..."? In my simple test above, the XMP encoded Exif and IPTC stuff was
returned in the XMP 'field' of the Exiv2::Image - not in the Exif and IPTC
fields, respectively, as far as I could tell. How can this be accomplished? The
case is different from what the command line utility does with the "-i X"
option in that it _only_ reads the contents of the XMP sidecar, and doesn't
have an image file to put write it to. Is it in the writing that the conversion
happens?

I suppose that at least when I would do a "save" from libkexiv2 (which puts
Exif, IPTC and XMP metadata in the Exiv2::Image using setExifData, setIPTCData,
and setXMPData, respectively, and then calls writeMetaData) the Exif and IPTC
data would get converted to XMP tags, and written to the XMP sidecar? Can you
confirm this behaviour?

> Gilles is right about the makernote, that is lost. It's also true that I'm not
> keen to dump the makernote as a large binary blob into an XMP property as
> exiftool does. It is unlikely that any other application would ever be able to
> deal with such a blob properly. Instead I suggested to write the _known_
> makernote tags as readable properties in new custom namespaces (e.g., like
> Xmp.Nikon3.WhiteBalance). But that's just an idea at this point, no such code
> exists yet.

I must say that, personally, I have no opinion about these makernotes, as I do
not know anything about them...

> > maybe I could first get reading and writing XMP sidecar files supported from a
> > menu command, and then we can see where we want to go from there?
>
> I think that should be possible. In digiKam the main concern is about the
> workflow and the options to control the handling of XMP sidecars.

Agreed.

> When it comes
> to the contents of the sidecar files, you'll need to turn to libexiv2.

I'm not following... Are you talking about the issue with getting the XMP
encoded Exif and IPTC tags converted into proper Exif and IPTC, i.e. the issue
discussed at the top of this reply?

>
> Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #11 from Gilles Caulier <caulier gilles gmail com>  2010-04-01 09:17:36 ---
about XMP sidecar support, see below my proposal :

1/ read XMP sidecar :
 - must be used when metadata must be imported to database (scan at startup for
ex.).
 - in first image metadata must be check, including makernotes. if nothing is
find, XMP sidecar is the alternative.
 - this include all file format as images and video files.

2/ Write XMP sidecar :
 - must be used to sync image metadata with database contents.
 - in first metadata must be written in image file. If image format is read
only or not supported by Exiv2, XMP sidecar must be created. This is the case
of RAW files or video files for ex.

To manage these rules, 2 implementation must be fixed :

1/ libkexiv2, especially read and write method.
2/ digiKam MetadataHub which is the core metadata management from digiKam.
3/ digiKam metadata settings pannel where option must be added to play with XMP
sidecar.

links to code :

1/ =>
http://websvn.kde.org/trunk/KDE/kdegraphics/libs/libkexiv2/libkexiv2/kexiv2.cpp?revision=1070289&view=markup
2/ =>
http://websvn.kde.org/trunk/extragear/graphics/digikam/digikam/metadatahub.cpp?revision=1050241&view=markup
3/ =>
http://websvn.kde.org/trunk/extragear/graphics/digikam/utilities/setup/setupmetadata.cpp?revision=1075046&view=markup

Important : only libkexiv2 and digiKam from trunk must be fixed in this way.

Marcel, i forget something there ?

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #12 from Gilles Caulier <caulier gilles gmail com>  2010-04-01 09:29:17 ---
I forget a very important features : manage image file and xmp sidecar file as
well :

1/ if image is removed, remove xmp sidecar too.
2/ if image is renamed, rename xmp sidecar too.
3/ if image is moved, move xmp sidecar too.

typically, xmp file name is image file name without image format extension,
this one is replaced by xmp. ex : DSC12568.cr2 => DSC12568.xmp

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #13 from Andreas Huggel <ahuggel gmx net>  2010-04-01 16:27:20 ---
Re comment #10:

> I suppose that at least when I would do a "save" from libkexiv2 (which puts
> Exif, IPTC and XMP metadata in the Exiv2::Image using setExifData, setIPTCData,
> and setXMPData, respectively, and then calls writeMetaData) the Exif and IPTC
> data would get converted to XMP tags, and written to the XMP sidecar? Can you
> confirm this behaviour?

Yes, XmpSidecar::writeMetadata() converts certain Exif and IPTC tags from their
respective Exiv2 containers as set in the XmpSidecar to XMP properties in the
resulting file. So you'll find properties like exif:FNumber in the *.xmp file.

XmpSidecar::readMetadata converts (copies) these properties back into the Exiv2
Exif and IPTC containers. So you'll find the Exif tag Exif.Photo.FNumber in the
ExifData container and the XMP property Xmp.exif.FNumber in the XmpData
container after reading the sidecar.

>> When it comes
>> to the contents of the sidecar files, you'll need to turn to libexiv2.
>
> I'm not following... Are you talking about the issue with getting the XMP
> encoded Exif and IPTC tags converted into proper Exif and IPTC, i.e. the issue
> discussed at the top of this reply?

Yes. I was referring to the conversion described above. That code in Exiv2 will
be the place to tweak to tweak what Exif and IPTC metadata are converted and
how they are stored in the sidecar file.

Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #14 from Jakob Malm <jakob malm gmail com>  2010-04-01 16:59:05 ---
(In reply to comment #11)
> about XMP sidecar support, see below my proposal :
>
> 1/ read XMP sidecar :
>  - must be used when metadata must be imported to database (scan at startup for
> ex.).
>  - in first image metadata must be check, including makernotes. if nothing is
> find, XMP sidecar is the alternative.

My opinion on this is that tags present in an associated XMP sidecar file
should take precedence over the tags found in the image. In any case, an
existing sidecar file would need to be read even if there is metadata in the
image/video file itself, because it may contain other tags than what's found in
the image/video.

>  - this include all file format as images and video files.
>
> 2/ Write XMP sidecar :
>  - must be used to sync image metadata with database contents.
>  - in first metadata must be written in image file. If image format is read
> only or not supported by Exiv2, XMP sidecar must be created. This is the case
> of RAW files or video files for ex.

I think this should be set in preferences. I would rather _not_ write to the
image, even if it is writable, but rather use the XMP sidecar.

>
> To manage these rules, 2 implementation must be fixed :
>
> 1/ libkexiv2, especially read and write method.
> 2/ digiKam MetadataHub which is the core metadata management from digiKam.
> 3/ digiKam metadata settings pannel where option must be added to play with XMP
> sidecar.
>
> links to code :
>
> 1/ =>
> http://websvn.kde.org/trunk/KDE/kdegraphics/libs/libkexiv2/libkexiv2/kexiv2.cpp?revision=1070289&view=markup
> 2/ =>
> http://websvn.kde.org/trunk/extragear/graphics/digikam/digikam/metadatahub.cpp?revision=1050241&view=markup
> 3/ =>
> http://websvn.kde.org/trunk/extragear/graphics/digikam/utilities/setup/setupmetadata.cpp?revision=1075046&view=markup
>
> Important : only libkexiv2 and digiKam from trunk must be fixed in this way.

Can I use 1.2.0 as a base, though?

>
> Marcel, i forget something there ?
>
> Gilles Caulier

Jakob Malm

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #15 from Jakob Malm <jakob malm gmail com>  2010-04-01 17:01:20 ---
(In reply to comment #12)

> I forget a very important features : manage image file and xmp sidecar file as
> well :
>
> 1/ if image is removed, remove xmp sidecar too.
> 2/ if image is renamed, rename xmp sidecar too.
> 3/ if image is moved, move xmp sidecar too.
>
> typically, xmp file name is image file name without image format extension,
> this one is replaced by xmp. ex : DSC12568.cr2 => DSC12568.xmp
>
> Gilles Caulier

Absolutely! This must be the final intention, but cannot be my first goal.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=220545





--- Comment #16 from Jakob Malm <jakob malm gmail com>  2010-05-05 08:09:14 ---
Just to let you know that I'm still working on this. I have some working code,
but am right now thinking of how to make Exif and IPTC tags be written to the
XMP sidecar. In the KExiv2::save function, before these tags are put in the
Image object, it is checked whether the file has support for them. For XMP
sidecar files, this is not the case and these fields are never set, and hence,
information only found in Exif and IPTC tags are not written to the XMP
sidecar. Perhaps I should just add a check to see if we're using XMP sidecar
and write the fields anyway.

But it may be a question that actually is worth considering more generally.
Can't we let exiv2 handle the issue of whether Exif and IPTC fields can be
written to the file? In the XMP sidecar case, exiv2 will do the conversion.
What about other cases? Will it convert, skip, or produce an error?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel