[digiKam-users] Importing xmp data from Lightroom 4.4 to Digikam 5.5.0

[jonjt]

Hello Everyone,
I need some help migrating from Lightroom 4.4 to Digikam 5.5.0.  I've added
my collection to Digikam and loaded a few images that I know I've edited.
However, it seems to me that the sidecar files (*.xmp where the * is the
.cr2's file name) are being ignored.  I believe this is happening because
Lightroom 4.4 doesn't use a standard method of populating those sidecar
files.  As such, I suspect that the only way to preserve the post processing
I've done for the ~10,000 raw files is to convert them all to .dngs or
.tiffs with the current changes applied.  I could do this by installing
Lightroom 4.4 on this same PC, as I have Windows 10 also installed
(although, I've had no need to use it, heretofore!).  Is there any other
method I could use to apply previous changes?  I don't know much about
Digikam so, I thought it prudent to ask the forum, beforehand.  I also
noticed that all of the EXIF data is gone, as well (aside from my name).  Is
there a way to import the keywords from Lightroom 4.4?

Note that I have searched past posts and come to find that other people have
asked similar questions, before.  However, the versions of Digikam used were
different and the challenges weren't exactly the same.  I suspect that I
will have to use the aforementioned solution but, I just wanted to check
before I do the work.

Thanks!



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Gilles Caulier-4]

Hi,

Normally, LR metadata must be backported to DK. The rules are there, i implemented myself the code. When i said LR metadata, i want mean all that DK can understand from this application.

Can you share some files (CR2 + XMP sidecars) to see if i can reproduce the problem.

Thanks in advance

Gilles Caulier

2018-03-15 13:57 GMT+01:00 jonjt <[hidden email]>:

Hello Everyone,
I need some help migrating from Lightroom 4.4 to Digikam 5.5.0.  I've added
my collection to Digikam and loaded a few images that I know I've edited.
However, it seems to me that the sidecar files (*.xmp where the * is the
.cr2's file name) are being ignored.  I believe this is happening because
Lightroom 4.4 doesn't use a standard method of populating those sidecar
files.  As such, I suspect that the only way to preserve the post processing
I've done for the ~10,000 raw files is to convert them all to .dngs or
.tiffs with the current changes applied.  I could do this by installing
Lightroom 4.4 on this same PC, as I have Windows 10 also installed
(although, I've had no need to use it, heretofore!).  Is there any other
method I could use to apply previous changes?  I don't know much about
Digikam so, I thought it prudent to ask the forum, beforehand.  I also
noticed that all of the EXIF data is gone, as well (aside from my name).  Is
there a way to import the keywords from Lightroom 4.4?

Note that I have searched past posts and come to find that other people have
asked similar questions, before.  However, the versions of Digikam used were
different and the challenges weren't exactly the same.  I suspect that I
will have to use the aforementioned solution but, I just wanted to check
before I do the work.

Thanks!



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Remco Viëtor]

On vendredi 16 mars 2018 16:45:23 CET Gilles Caulier wrote:
> Hi,
>
> Normally, LR metadata must be backported to DK. The rules are there, i
> implemented myself the code. When i said LR metadata, i want mean all that
> DK can understand from this application.
>
> Can you share some files (CR2 + XMP sidecars) to see if i can reproduce the
> problem
> .

I understood that part of OP's problem is in conserving the edits done in
Lightroom. For that, I think he'll have to export all his images to TIFF (or
PNG if no layers present, png will be smaller), DNG won't help much as it is
still a raw format.

Remco

[meku]

I've found that Lightroom saves CR2 sidecars as FILENAME.XMP whereas Digikam uses FILENAME.CR2.XMP, this could be the reason why keywords and other user-created metadata are not being imported.

On 17 March 2018 at 03:52, Remco Viëtor <[hidden email]> wrote:

On vendredi 16 mars 2018 16:45:23 CET Gilles Caulier wrote:
> Hi,
>
> Normally, LR metadata must be backported to DK. The rules are there, i
> implemented myself the code. When i said LR metadata, i want mean all that
> DK can understand from this application.
>
> Can you share some files (CR2 + XMP sidecars) to see if i can reproduce the
> problem
> .

I understood that part of OP's problem is in conserving the edits done in
Lightroom. For that, I think he'll have to export all his images to TIFF (or
PNG if no layers present, png will be smaller), DNG won't help much as it is
still a raw format.

Remco

[jonjt]

Hello Everyone,
Thanks for the replies.

You will find an example image here:
https://drive.google.com/open?id=1GaokEvrZ3MEkIco_WmGHUXORZ4p30sQc
This example is a little complicated so, forgive me if my explanation is a
bit confusing.

This tar.gz file has a few files in it.  First, you will find the .cr2 file.
Of course, that is the original file I shot with my 7D.  Next, there is
"IMG_7109-Edit-2.tif".  This file was the file that I performed noise
reduction on with Topaz Denoise.  It's also one of three total .tiff files
that I've had in my Lightroom library for some time.  The other two .tif
files I created for post processing experimentation.  However, I have not
tried editing those files in some time.  As such, I decided not to upload
those two files.  However, I thought it prudent to mention these additional
files, because I do believe the changes made to those additional files are
tracked in the .xmp file that is shown here.
Additionally, I've included two other files.  The first file in that last is
"IMG_7109-Edit-2 DNG-TEST3.dng".  That was a .dng that I recently exported,
to see how previews and edits .dngs.  Previewing this .dng shows the edits,
even after I've selected the "Preview shows the full image" option in the
configuration widget.  However, when I open this image in the image editor,
the changes shown in preview disappear.
Finally, "IMG_7109-Edit-2 TIFF-Test.tif" is also included.  The previewed
image and that shown in the image editor are both the same; both images show
the postprocessing changes I've made in Lightroom 4.4.  Of course, this is
different from the other files located in the linked tar.gz file above.

I'm guessing that the process for importing changes made in Lightroom 4.4
would require me to create a second, .tif, copy of all files edited in
lightroom.  Doing so would allow me to preserve the post processing I've
done while also saving the .cr2 that has the unmodified raw data.  I wanted
to run this by you just to see if there is a better way to migrate.  While I
do have enough storage space to keep multiple lossless copies of these
original images, I'd prefer to be a bit more efficient with the space that I
use.

Thanks for the help and please let me know if I should provide anything else
(including another example).



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Gilles Caulier-4]

Hi,

I just tested your test photos, and i think i found the reason why the LR metadata are not backported to DK :

digikam.metaengine: Exiv2 ( 3 ) :  XMP Toolkit error 101: Schema namespace URI and prefix mismatch
digikam.metaengine: Exiv2 ( 2 ) :  Failed to decode XMP metadata.

These lines are written on the console by Exiv2 library when files are parsed to be registered in database.

An XMP namespace is badly defined in Exiv2 (the URI use to identify the namespace). Probably Adobe change the URI as well (which can be a mess). I'm not yet sure. I continue to search.

Gilles Caulier

2018-03-17 22:48 GMT+01:00 jonjt <[hidden email]>:

Hello Everyone,
Thanks for the replies.

You will find an example image here:
https://drive.google.com/open?id=1GaokEvrZ3MEkIco_WmGHUXORZ4p30sQc
This example is a little complicated so, forgive me if my explanation is a
bit confusing.

This tar.gz file has a few files in it.  First, you will find the .cr2 file.
Of course, that is the original file I shot with my 7D.  Next, there is
"IMG_7109-Edit-2.tif".  This file was the file that I performed noise
reduction on with Topaz Denoise.  It's also one of three total .tiff files
that I've had in my Lightroom library for some time.  The other two .tif
files I created for post processing experimentation.  However, I have not
tried editing those files in some time.  As such, I decided not to upload
those two files.  However, I thought it prudent to mention these additional
files, because I do believe the changes made to those additional files are
tracked in the .xmp file that is shown here.
Additionally, I've included two other files.  The first file in that last is
"IMG_7109-Edit-2 DNG-TEST3.dng".  That was a .dng that I recently exported,
to see how previews and edits .dngs.  Previewing this .dng shows the edits,
even after I've selected the "Preview shows the full image" option in the
configuration widget.  However, when I open this image in the image editor,
the changes shown in preview disappear.
Finally, "IMG_7109-Edit-2 TIFF-Test.tif" is also included.  The previewed
image and that shown in the image editor are both the same; both images show
the postprocessing changes I've made in Lightroom 4.4.  Of course, this is
different from the other files located in the linked tar.gz file above.

I'm guessing that the process for importing changes made in Lightroom 4.4
would require me to create a second, .tif, copy of all files edited in
lightroom.  Doing so would allow me to preserve the post processing I've
done while also saving the .cr2 that has the unmodified raw data.  I wanted
to run this by you just to see if there is a better way to migrate.  While I
do have enough storage space to keep multiple lossless copies of these
original images, I'd prefer to be a bit more efficient with the space that I
use.

Thanks for the help and please let me know if I should provide anything else
(including another example).

--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Gilles Caulier-4]

Ok, i found. 

The problem do not come Exiv2, even if this URI problem need to be fixed (Please report the URI problem in Exiv2 project with your test image link).

For digiKam the problem is know and we must fix it for next 6.0.0 release.

https://www.flickr.com/photos/digikam/40934505211/in/dateposted/

Look well my screenshot : i renamed the sidecar from IMG_7109.xmp to IMG_7109.CR2.xmp.

And now the metadata import to database work as expected (look tags list set in CR2 files). The problem have been already report in bugzilla :

https://bugs.kde.org/show_bug.cgi?id=278935

Best

Gilles Caulier

2018-03-21 8:44 GMT+01:00 Gilles Caulier <[hidden email]>:

Hi,

I just tested your test photos, and i think i found the reason why the LR metadata are not backported to DK :

digikam.metaengine: Exiv2 ( 3 ) :  XMP Toolkit error 101: Schema namespace URI and prefix mismatch
digikam.metaengine: Exiv2 ( 2 ) :  Failed to decode XMP metadata.

These lines are written on the console by Exiv2 library when files are parsed to be registered in database.

An XMP namespace is badly defined in Exiv2 (the URI use to identify the namespace). Probably Adobe change the URI as well (which can be a mess). I'm not yet sure. I continue to search.

Gilles Caulier

2018-03-17 22:48 GMT+01:00 jonjt <[hidden email]>:

Hello Everyone,
Thanks for the replies.

You will find an example image here:
https://drive.google.com/open?id=1GaokEvrZ3MEkIco_WmGHUXORZ4p30sQc
This example is a little complicated so, forgive me if my explanation is a
bit confusing.

This tar.gz file has a few files in it.  First, you will find the .cr2 file.
Of course, that is the original file I shot with my 7D.  Next, there is
"IMG_7109-Edit-2.tif".  This file was the file that I performed noise
reduction on with Topaz Denoise.  It's also one of three total .tiff files
that I've had in my Lightroom library for some time.  The other two .tif
files I created for post processing experimentation.  However, I have not
tried editing those files in some time.  As such, I decided not to upload
those two files.  However, I thought it prudent to mention these additional
files, because I do believe the changes made to those additional files are
tracked in the .xmp file that is shown here.
Additionally, I've included two other files.  The first file in that last is
"IMG_7109-Edit-2 DNG-TEST3.dng".  That was a .dng that I recently exported,
to see how previews and edits .dngs.  Previewing this .dng shows the edits,
even after I've selected the "Preview shows the full image" option in the
configuration widget.  However, when I open this image in the image editor,
the changes shown in preview disappear.
Finally, "IMG_7109-Edit-2 TIFF-Test.tif" is also included.  The previewed
image and that shown in the image editor are both the same; both images show
the postprocessing changes I've made in Lightroom 4.4.  Of course, this is
different from the other files located in the linked tar.gz file above.

I'm guessing that the process for importing changes made in Lightroom 4.4
would require me to create a second, .tif, copy of all files edited in
lightroom.  Doing so would allow me to preserve the post processing I've
done while also saving the .cr2 that has the unmodified raw data.  I wanted
to run this by you just to see if there is a better way to migrate.  While I
do have enough storage space to keep multiple lossless copies of these
original images, I'd prefer to be a bit more efficient with the space that I
use.

Thanks for the help and please let me know if I should provide anything else
(including another example).

--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Remco Viëtor]

On mercredi 21 mars 2018 08:44:05 CET Gilles Caulier wrote:

> Hi,
>
> I just tested your test photos, and i think i found the reason why the LR
> metadata are not backported to DK :
>
> digikam.metaengine: Exiv2 ( 3 ) :  XMP Toolkit error 101: Schema namespace
> URI and prefix mismatch
> digikam.metaengine: Exiv2 ( 2 ) :  Failed to decode XMP metadata.
>
> These lines are written on the console by Exiv2 library when files are
> parsed to be registered in database.
>
> An XMP namespace is badly defined in Exiv2 (the URI use to identify the
> namespace). Probably Adobe change the URI as well (which can be a mess).
> I'm not yet sure. I continue to search.

Are those namespaces defined within Exiv2, or within the XMP files? I.e. are the
URI's that play a role here defined within Exiv2 or by the program that wrote
the XMP (in this case lightroom)? As I understand namespaces, having them
defined within the library that reads the data would defeat portability of XML
data.

And when I look at a Digikam sidecar, I see a bunch of lines of the form
xmlns:<prefix>="<URL>" with the prefixes then used later in the different data
items.

In this case, if LR indicates a non-existing or wrong namespace, of course
Exiv2/Digikam will have a problem...

Remco

[Gilles Caulier-4]

2018-03-21 9:44 GMT+01:00 Remco Viëtor <[hidden email]>:

On mercredi 21 mars 2018 08:44:05 CET Gilles Caulier wrote:
> Hi,
>
> I just tested your test photos, and i think i found the reason why the LR
> metadata are not backported to DK :
>
> digikam.metaengine: Exiv2 ( 3 ) :  XMP Toolkit error 101: Schema namespace
> URI and prefix mismatch
> digikam.metaengine: Exiv2 ( 2 ) :  Failed to decode XMP metadata.
>
> These lines are written on the console by Exiv2 library when files are
> parsed to be registered in database.
>
> An XMP namespace is badly defined in Exiv2 (the URI use to identify the
> namespace). Probably Adobe change the URI as well (which can be a mess).
> I'm not yet sure. I continue to search.

Are those namespaces defined within Exiv2, or within the XMP files?

Both. Exiv2 know the namespaces :

https://github.com/Exiv2/exiv2/blob/master/src/properties.cpp#L106

And a XMP sidecar data header indicate which namespace will be used. Both must be identical :

[gilles@localhost Example 1]$ cat IMG_7109.xmp |more
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.5-c002 1.148022, 2012/07/15-18:06:45        ">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
 <rdf:Description rdf:about=""
   xmlns:xmp="http://ns.adobe.com/xap/1.0/"                                <<<< !!!!!!
   xmlns:tiff="http://ns.adobe.com/tiff/1.0/"                              <<<< !!!!!!
   xmlns:exif="http://ns.adobe.com/exif/1.0/"                              <<<< !!!!!!
   xmlns:dc="http://purl.org/dc/elements/1.1/"                             <<<< !!!!!!
   xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"                           <<<< !!!!!!
   xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"                    <<<< !!!!!!
   xmlns:xmpMM="http://ns.adobe.com/xap/1.0/mm/"                           <<<< !!!!!!
   xmlns:stEvt="http://ns.adobe.com/xap/1.0/sType/ResourceEvent#"          <<<< !!!!!!
   xmlns:crs="http://ns.adobe.com/camera-raw-settings/1.0/"                <<<< !!!!!!
   xmlns:lr="http://ns.adobe.com/lightroom/1.0/"                           <<<< !!!!!!
  xmp:ModifyDate="2015-02-21T19:34:28.00"
  xmp:CreateDate="2015-02-21T19:34:28.00"
  xmp:MetadataDate="2016-11-28T10:15:25-05:00"
  tiff:Make="Canon"
  tiff:Model="Canon EOS 7D"
...

Gilles

[Remco Viëtor]

On mercredi 21 mars 2018 09:57:43 CET Gilles Caulier wrote:

> 2018-03-21 9:44 GMT+01:00 Remco Viëtor <[hidden email]>:
> > On mercredi 21 mars 2018 08:44:05 CET Gilles Caulier wrote:
> > > Hi,
> > >
> > > I just tested your test photos, and i think i found the reason why the
> > > LR
> > > metadata are not backported to DK :
> > >
> > > digikam.metaengine: Exiv2 ( 3 ) :  XMP Toolkit error 101: Schema
> >
> > namespace
> >
> > > URI and prefix mismatch
> > > digikam.metaengine: Exiv2 ( 2 ) :  Failed to decode XMP metadata.
> > >
> > > These lines are written on the console by Exiv2 library when files are
> > > parsed to be registered in database.
> > >
> > > An XMP namespace is badly defined in Exiv2 (the URI use to identify the
> > > namespace). Probably Adobe change the URI as well (which can be a mess).
> > > I'm not yet sure. I continue to search.
> >
> > Are those namespaces defined within Exiv2, or within the XMP files?
>
> Both. Exiv2 know the namespaces :
>
> https://github.com/Exiv2/exiv2/blob/master/src/properties.cpp#L106
>
> And a XMP sidecar data header indicate which namespace will be used. Both
> must be identical :
>
> [gilles@localhost Example 1]$ cat IMG_7109.xmp |more
> <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.5-c002
> 1.148022, 2012/07/15-18:06:45        ">
> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
>  <rdf:Description rdf:about=""
>    xmlns:xmp="http://ns.adobe.com/xap/1.0/"
> <<<< !!!!!!
>    xmlns:tiff="http://ns.adobe.com/tiff/1.0/"
>  <<<< !!!!!!
>    xmlns:exif="http://ns.adobe.com/exif/1.0/"
>  <<<< !!!!!!
>    xmlns:dc="http://purl.org/dc/elements/1.1/"
> <<<< !!!!!!
>    xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
> <<<< !!!!!!
>    xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"
>  <<<< !!!!!!
>    xmlns:xmpMM="http://ns.adobe.com/xap/1.0/mm/"
> <<<< !!!!!!
>    xmlns:stEvt="http://ns.adobe.com/xap/1.0/sType/ResourceEvent#"
>  <<<< !!!!!!
>    xmlns:crs="http://ns.adobe.com/camera-raw-settings/1.0/"
>  <<<< !!!!!!
>    xmlns:lr="http://ns.adobe.com/lightroom/1.0/"
> <<<< !!!!!!
>   xmp:ModifyDate="2015-02-21T19:34:28.00"
>   xmp:CreateDate="2015-02-21T19:34:28.00"
>   xmp:MetadataDate="2016-11-28T10:15:25-05:00"
>   tiff:Make="Canon"
>   tiff:Model="Canon EOS 7D"
> ...
>
> Gilles

Thank you for the clarification.

Funny detail: there's no more server on "ns.adobe.com" (no idea if that's
important in practice)

Remco

[jonjt]

Hello Gilles Caulier,
Thank you for the information.  Can you tell me when digiKam version 6.0.0
is supposed to be released?

Thank you



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Gilles Caulier-4]

6.0.0 will be released later this summer, but we will release beta version before, including linux appimage to test easily.

The first beta will be published in few weeks, when code will be ready.

Gilles Caulier

2018-03-26 4:12 GMT+02:00 jonjt <[hidden email]>:

Hello Gilles Caulier,
Thank you for the information.  Can you tell me when digiKam version 6.0.0
is supposed to be released?

Thank you

--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[jonjt]

Thank you!



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html