[Bug 309058] New: The digiKam database can't be synchronized with XMP sidecars.

[Kevin Dalley]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #19 from Kevin Dalley <[hidden email]> ---
I pushed another patch into 'bug309058', which solves the problem of moving a
tag within the tree. When this patch is placed into master, then the bug can be
closed.

 When tags are moved to a different position in the tree, synchronize
 tag changes by writing to files, using WriteFromDatabaseToFile

I repeated Test 3 from above.

Test 3, move a tag, passes: After recreating the tag "Sidecar", move it to
"Testing/Sidecar" Examine the jpeg and the xmp sidecar exiftool -a file |grep
Sidecar Note that "Testing/Sidecar" does appear now.  Note that Sidecar-new
shows up under Subject. This seems to be the result of early testing.
The following does not fix the problem. I consider this either a different bug,
or possibly a feature, where Subject does not get updated to match the tags.
Click on Image=>Write Metadata to Image


kevin@nereocystis:~/src/digikam-git/digikam$ exiftool -a
/home/kevin/Nikon/DCIM/106ND700/DSC_3148.NEF.xmp|grep Sidecar
Tags List                       : Testing/Sidecar, Testing
Last Keyword XMP                : Testing/Sidecar, Testing
Hierarchical Subject            : Testing|Sidecar, Testing
Subject                         : Sidecar, Testing

kevin@nereocystis:~/src/digikam-git/digikam$ exiftool -a
/home/kevin/Nikon/DCIM/Android/newstuff/2013-11-03/IMG_20131103_102820.jpg
|grep Sidecar
Tags List                       : Testing/Sidecar, Testing
Last Keyword XMP                : Testing/Sidecar, Testing
Hierarchical Subject            : Testing|Sidecar, Testing
Subject                         : Sidecar, Testing
Keywords                        : Sidecar-new, Sidecar, Testing

kevin@nereocystis:~/src/digikam-git/digikam$ exiftool -a
/home/kevin/Nikon/DCIM/Android/newstuff/2013-11-03/IMG_20131103_102820.jpg.xmp
|grep Sidecar
Tags List                       : Testing/Sidecar, Testing
Last Keyword XMP                : Testing/Sidecar, Testing
Hierarchical Subject            : Testing|Sidecar, Testing
Subject                         : Sidecar-new, Sidecar, Testing

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #20 from Gilles Caulier <[hidden email]> ---
Hi Kevin,

What's happen if a lots of images relevant of tags hierarchy changes
needs to be updated ? This can take a while to process. To update
files as well can decrease digiKAm performance a lots until files
update is done. Do you tried in production ?

I know that Aperture or Lightroom put all changes in a list to play
when users can do it (for ex when he don't use application in
workflow). This is the goal of Maintenance tool in digiKam. End users
run it at the most adapted moment...

Best

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kevin Dalley]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #21 from Kevin Dalley <[hidden email]> ---
That's a good question.

I have tried changing the name of a tag which was used by about 100
files. It does take a little while at that level. Of course, a user
could modify the tags for every file. That could be slow.

Here are the options I see:

1. Use the code changes I suggested.
    Advantages:
        Much lower risk that database and files are out of sync.
    Disadvantages:
        Could significantly slow digikam.
        Repeated operations are even worse.

2. Use Maintenance Tool. Perhaps user could be warned when operation is
performed.
    Advantages:
        Users control when files are updated.
    Disadvantages:
        User has to remember to run Maintenance.
        User has to know what to run, and which direction. A
            very big disadvantage, I think.
        If database is read from out of date files, old tags    
            return, creating a lot of work for the user.

3. Use digikam Preferences to determine behavior from 1 and 2.

4. Allow users to specify behavior when operations are performed.

5. Modify MetadataSynchronizer so that synchronization is slower and
less painful for the user. This is easy to say, but more challenging to
implement.

6. A variation on 5. Keep a list of images which are out of sync, stored
in the database. Occasionally process a few. This method avoids having
the same image listed twice. Occasionally process some files.

I somewhat like 6, but I am not sure.

Any of these methods allow for the possibility of unsynchronized data.
Completely removing this risk requires a lot of work.

On 12/09/2013 02:49 PM, Gilles Caulier wrote:

> https://bugs.kde.org/show_bug.cgi?id=309058
>
> --- Comment #20 from Gilles Caulier <[hidden email]> ---
> Hi Kevin,
>
> What's happen if a lots of images relevant of tags hierarchy changes
> needs to be updated ? This can take a while to process. To update
> files as well can decrease digiKAm performance a lots until files
> update is done. Do you tried in production ?
>
> I know that Aperture or Lightroom put all changes in a list to play
> when users can do it (for ex when he don't use application in
> workflow). This is the goal of Maintenance tool in digiKam. End users
> run it at the most adapted moment...
>
> Best
>
> Gilles Caulier
>

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kevin Dalley]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #22 from Kevin Dalley <[hidden email]> ---
I performed some additional tests, changing the tag name for a tag which
had over 1000 images. Then I changed it back again.

During the change back, I hit a deadlock, at least that is my best
guess. I'm investigating it now, attaching gdb to the running process.
Digikam hung up. I did try performing other operations while the writing
was finishing.

I suspect that writing that many images tickled a bug which had existed
before.

In addition to the deadlock, the intial change was slow, perhaps 5
minutes. Performing other operations was slow, but they did complete.

On 12/09/2013 02:49 PM, Gilles Caulier wrote:

> https://bugs.kde.org/show_bug.cgi?id=309058
>
> --- Comment #20 from Gilles Caulier <[hidden email]> ---
> Hi Kevin,
>
> What's happen if a lots of images relevant of tags hierarchy changes
> needs to be updated ? This can take a while to process. To update
> files as well can decrease digiKAm performance a lots until files
> update is done. Do you tried in production ?
>
> I know that Aperture or Lightroom put all changes in a list to play
> when users can do it (for ex when he don't use application in
> workflow). This is the goal of Maintenance tool in digiKam. End users
> run it at the most adapted moment...
>
> Best
>
> Gilles Caulier
>

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #23 from Marcel Wiesweg <[hidden email]> ---
"thr appl all bt" gives you backtraces of all threads. If there's a deadlock,
it can often be identified with this snapshot.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|Tags                        |Sidecar Management
            Summary|The digiKam database can't  |The database can't be
                   |be synchronized with XMP    |synchronized with XMP
                   |sidecars.                   |sidecars

--- Comment #24 from Gilles Caulier <[hidden email]> ---
Elle,

This file still valid using last digiKam 4.2.0 ?

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|The database can't be       |Database can't be
                   |synchronized with XMP       |synchronized with XMP
                   |sidecars                    |sidecars

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kevin Dalley]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #25 from Kevin Dalley <[hidden email]> ---
Gilles,

Thanks for checking.

I'll see whether I can test this.

I am still actively using digikam, but I'm not carefully tracking my bugs.

It will probably take a few days to a week before I have an answer.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |WAITINGFORINFO
             Status|ASSIGNED                    |RESOLVED

--- Comment #26 from Gilles Caulier <[hidden email]> ---
This file still valid using last digiKam 4.10.0 ?

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|                            |4.12.0

--- Comment #27 from Gilles Caulier <[hidden email]> ---
digiKam 4.11.0 is out. Still reproducible ?

Thanks in advance

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|4.12.0                      |

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kevin Dalley]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #28 from Kevin Dalley <[hidden email]> ---
Did my changes ever get merged into the main code base?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #29 from Gilles Caulier <[hidden email]> ---
yes, this must be.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Elle Stone-2]

https://bugs.kde.org/show_bug.cgi?id=309058

Elle Stone <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ellestone@ninedegreesbelow.
                   |                            |com

--- Comment #30 from Elle Stone <[hidden email]> ---
I set up a test folder with 3 images, and a test digiKam database. I told
digiKam to only write to the XMP files and only read from the XMP files. I
don't see any XMP files being created. So I am unable to test to see whether
the original bug has been fixed. Maybe I'm doing something wrong. How does one
create the XMP side car files?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #31 from Gilles Caulier <[hidden email]> ---
Change/update something metadata to create XMP sidecar. Ex : open file in
editor, rotate image and save it.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Elle Stone-2]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #32 from Elle Stone <[hidden email]> ---
I already made a whole lot of changes, rearranging the tag tree and renaming
some tags. No XMP files were created. I tried reading and writing the metadata.
The old metadata got pulled in from the images to the database, which is not
desirable behavior because now the database the "prechange" information and
also the "postchange" information. But no XMP sidecars were created.

Per your suggestion, I rotated an image, which created a sidecar called
"JpegRotator-M20394.digikamtempfile.jpg.XML". That XML file doesn't contain any
informationl except the line "<?xml version="1.0" encoding="UTF-8"?>".

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #33 from Gilles Caulier <[hidden email]> ---
Go to Help/Components Info dialog and look in XMP is supported by Exiv2...

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Elle Stone-2]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #34 from Elle Stone <[hidden email]> ---
digiKam version 4.10.0
Exiv2 supports XMP metadata: Yes

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #35 from Gilles Caulier <[hidden email]> ---
To rotate image use image editor as well.

Run digiKam form a console while this operation. Did you seen some debug traces
?

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Elle Stone-2]

https://bugs.kde.org/show_bug.cgi?id=309058

--- Comment #36 from Elle Stone <[hidden email]> ---
After rotating an image with showFoto, nothing is written to the terminal.
While asking digiKam to write to the image (which should mean write to the
XMPfile) and then read from the image (which should mean read from the XMP
file), the following (doesn't seem relevant) is printed to screen:

digikam(20825)/digikam (core) Digikam::DNotificationWrapper: parent is null
Thread::requestAbort: not running.
Thread::requestAbort: not running.  
digikam(20825)/KEXIV2: Cannot find Xmp key 'Xmp.acdsee.caption' into image
using Exiv2   (Error # 35 :  No namespace info available for XMP prefix
`acdsee'  
digikam(20825)/KEXIV2: Cannot find Xmp key 'Xmp.acdsee.author' into image using
Exiv2   (Error # 35 :  No namespace info available for XMP prefix `acdsee'  
digikam(20825)/KEXIV2: Cannot find Xmp key 'Xmp.acdsee.notes' into image using
Exiv2   (Error # 35 :  No namespace info available for XMP prefix `acdsee'  
Thread::requestAbort: not running.  
Thread::requestAbort: not running.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel