[digikam] [Bug 347737] New: Tagged RAW files do not show tags after round trip to RawTherapee

[Eric Mesa]

https://bugs.kde.org/show_bug.cgi?id=347737

            Bug ID: 347737
           Summary: Tagged RAW files do not show tags after round trip to
                    RawTherapee
           Product: digikam
           Version: 4.9.0
          Platform: Fedora RPMs
                OS: Linux
            Status: UNCONFIRMED
          Severity: major
          Priority: NOR
         Component: Tags
          Assignee: [hidden email]
          Reporter: [hidden email]

I take a RAW file from my Canon 400D - it doesn't matter if it's the CR2 file
or if it's been converted to a DNG. I add a tag, title, and caption in Digikam.
Then I edit the file in RawTherapee. Then I load up the newly created JPEG in
Digikam. It only can read the caption. The title and tag do not show in
Digikam. Looking with exiftool confirms that the data has been properly written
to the JPEG.

However, when I look at the file using exiv2 - which Digikam uses, I get the
following warnings:
exiv2 -p a _MG_3694.jpg
Warning: Ignoring IPTC information encoded in the Exif data.
Warning: Ignoring XMP information encoded in the Exif data.

Which I feel is responsible for the problem.

Reproducible: Always

Steps to Reproduce:
1. Tag RAW files in Digikam
2. Edit in RawTherapee
3. Produce a JPEG
4. The JPEG is missing tags.

Actual Results:  
No tags (or title) on resulting JPEGs

Expected Results:  
Tags and titles on resulting JPEGs.

You can see the CR2 here:
https://www.dropbox.com/s/jvqiinpsaapzxf4/_MG_3694.CR2?dl=0
You can see the resulting JPEG here;
https://www.dropbox.com/s/ip503k3tom9ciew/_MG_3694.jpg?dl=0

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Eric Mesa]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #1 from Eric Mesa <[hidden email]> ---
Oh yeah, the version of exiv2 on Fedora is 0.24-4, but I had to install that
separately from Digikam so I imagine that you guys have exiv2 embedded and the
version I have on my computer doesn't matter.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=347737

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |UPSTREAM
                 CC|                            |[hidden email]
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #2 from Gilles Caulier <[hidden email]> ---
>I take a RAW file from my Canon 400D - it doesn't matter if it's the CR2 file or if it's been converted to a DNG. I add a tag, title, and caption in Digikam. >Then I edit the file in RawTherapee. Then I load up the newly created JPEG in Digikam. It only can read the caption.

=> Caption is stored in JPEG comment section, outside XMP and IPTC

>The title and tag do not show in Digikam. Looking with exiftool confirms that the data has been properly written to the JPEG.

Stored in XMP

>However, when I look at the file using exiv2 - which Digikam uses, I get the following warnings:
>exiv2 -p a _MG_3694.jpg
>Warning: Ignoring IPTC information encoded in the Exif data.
>Warning: Ignoring XMP information encoded in the Exif data.

Sound like a corruption or somthing like that performed by RawTherapee. Exiv2
do not like it.

>Which I feel is responsible for the problem.

It's clear. Metadata have been corrupted by RawTherapee. There is nothing to do
in digiKam/libkexiv2 interface

1/ Report this problem to Exiv2 bugzilla. Explain well all operation done in
RawTherapee to reproduce the problem. Provide original images, images tagged in
digiKam, and images fixed in RawTherapee.
2/ When problem is identified by Exiv2 team, the question will be : there is
something to fix in Exiv2 ? If not report this problem to RawTherapee team.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #3 from Gilles Caulier <[hidden email]> ---
>Oh yeah, the version of exiv2 on Fedora is 0.24-4, but I had to install that separately from >Digikam so I imagine that you guys have exiv2 embedded and the version I have on my >computer doesn't matter.

No. digiKam do not include Exiv2 library code. It use system shared library as
well.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Alan Pater]

https://bugs.kde.org/show_bug.cgi?id=347737

Alan Pater <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #4 from Alan Pater <[hidden email]> ---
Gilles, what does "If possible, write Metadata to RAW files (experimental)" do?

In the samples, the XMP has landed up inside of Exif. That is the experimental
feature?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #5 from Gilles Caulier <[hidden email]> ---
Well, Raw metadata writing mode delegate Raw writing support to Exiv2 as it's
done with others file format.

Raw is considerated by people as read only image (as negative photo). So the
option is disabled by default.

The option is annotated experimental because not all format are supported by
Exiv2 (and also tested). Typically, only TIFF/EP file based are supported.

I think we must remove this option in Exiv2, since XMP sidecar do the stuff
instead, if end user turn on right option in this settings panel.

Note : It's libkexiv2 which wrap these workflow settings with Exiv2 in
background.

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Eric Mesa]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #6 from Eric Mesa <[hidden email]> ---
Just so  you know, this also happens with DNG file, which is considered "safe"
for writing metadata to without the need for XMP sidecar. Also, very important
to keep from having to keep track of two different files when backing up, etc

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Alan Pater]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #7 from Alan Pater <[hidden email]> ---
The XMP is inside of Exif ApplicationNotes. Is that valid according to the
specs?

According to exiftool:

  | 16) ApplicationNotes (SubDirectory) -->
  | + [XMP directory, 4868 bytes]
  | | XMPToolkit = XMP Core 4.4.0-Exiv2
  | | Software = digiKam-4.9.0
  | | DateTime = 2015-05-13T17:08:52
  | | CreatorTool = digiKam-4.9.0
  | | CreateDate = 2015-05-13T17:08:52
  | | MetadataDate = 2015-05-13T17:08:52
  | | ModifyDate = 2015-05-13T17:08:52
  | | DateTimeOriginal = 2015-05-13T17:08:52
  | | DateCreated = 2015-05-13T17:08:52
  | | Urgency = 0
  | | PickLabel = 0
  | | ColorLabel = 0
  | | ImageDescription = Leaves and Scarlett
  | | UserComment = Leaves and Scarlett
  | | TagsList = Scarlett
  | | CaptionsAuthorNames =
  | | CaptionsDateTimeStamps = 2015-05-13T20:33:02
  | | RegionList =
  | | RegionInfoRegions =
  | | LastKeywordXMP = Scarlett
  | | HierarchicalSubject = Scarlett
  | | Title = Scarlett Playing with Leaves
  | | Description = Leaves and Scarlett
  | | Subject = Scarlett

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #8 from Gilles Caulier <[hidden email]> ---
The XMP in which file ? JPEG generated by RawTherapee ?

If yes, the Exif spec permit to do it but it's not recommended with JPEG (and
in general), as JPEG Exif chunk is limited to 64Kb. JPEG IPTC and XMP chunk are
also limited in size to 64Kb, but as separated than Exif. To resume :

Exif = 64Kb
IPTC = 64Kb
XMP = 64Kb

If you host IPTC and XMP to Exif chunk as well, there is a risk to overload
this chunk quickly and to break JPEG structure. This is why, if i remember
well, Exiv2 drop this Exif entries.

Note : i know that Adode has proposed a way to extend each JPEG chunk to use
more than one 64Kb blocks, but i'm not sure if this is already standardized.

In all case, RawTherapee must store IPTC and XMP in dedicated chunks.

Gillesq

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Alan Pater]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #9 from Alan Pater <[hidden email]> ---
No, the XMP is in the cr2 file generated by digikam.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Alan Pater]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #10 from Alan Pater <[hidden email]> ---
Ok, it looks like what exiftool calls ApplicationNotes, exiv2 calls
Exif.Image.XMLPacket.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Eric Mesa]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #11 from Eric Mesa <[hidden email]> ---
Does that mean exiv2 needs a translation between the two so it's able to read
it?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #12 from Gilles Caulier <[hidden email]> ---
To comment #11 : no, if i remember, there is already a file in Exiv2 bugzilla
about to ignore ITPC and XMP from Exif tags

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #13 from Gilles Caulier <[hidden email]> ---
Alan,

Just a terminology adjustement : I used JPEG "chunk" previously, where in JPEG
we use JPEG "section". Chunk is used for PNG container.

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #14 from Gilles Caulier <[hidden email]> ---
Exiv2 issues more and less relevant :

http://dev.exiv2.org/issues/699
http://dev.exiv2.org/issues/533

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Eric Mesa]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #15 from Eric Mesa <[hidden email]> ---
(In reply to Gilles Caulier from comment #12)
> To comment #11 : no, if i remember, there is already a file in Exiv2
> bugzilla about to ignore ITPC and XMP from Exif tags
>
> Gilles

So at this point it's still an exiv2 but, then?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Eric Mesa]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #16 from Eric Mesa <[hidden email]> ---
(In reply to Eric Mesa from comment #15)
> (In reply to Gilles Caulier from comment #12)
> > To comment #11 : no, if i remember, there is already a file in Exiv2
> > bugzilla about to ignore ITPC and XMP from Exif tags
> >
> > Gilles
>
> So at this point it's still an exiv2 but, then?

That should say "bug" not "but", couldn't find how to edit my comment.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #17 from Gilles Caulier <[hidden email]> ---
Not sure if it's an Exiv2 bug. But in all case, in digiKam, there is nothing to
do...

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Eric Mesa]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #18 from Eric Mesa <[hidden email]> ---
(In reply to Gilles Caulier from comment #17)
> Not sure if it's an Exiv2 bug. But in all case, in digiKam, there is nothing
> to do...
>
> Gilles

Gotcha. Thanks.

At this point it appears either something in Exiv2 or something in RawTherapee
- I guess whoever's right depends on who's adhering to the spec the most, I
imagine. Worst case scenario I could always write a bash script to read the
data out of the DNGs with Exiftool and into the JPEGs with Exiv2 until it's
resolved.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Alan Pater]

https://bugs.kde.org/show_bug.cgi?id=347737

--- Comment #19 from Alan Pater <[hidden email]> ---
1st: It has nothing to do with RawTherapee as the XMP data was written to the
Exif section of the CR2 image by libkexiv2 via digikam. RawTherapee carried
over that data unchanged to the JPG.

2nd: This is why it is labelled experimental. For CR2 images, the correct
workflow is to allow digikam to only write to XMP sidecar files. Then once you
have converted to JPG, import the XMP metadata from the sidecar file.

3rd: There has been some work on improving RAW file support in exiv2, but it is
not yet complete. This includes reading from Exif.Image.XMLPacket.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel