[digikam] [Bug 341772] New: Re-read metadata when image file timestamp has changed

[Kristian Karl]

https://bugs.kde.org/show_bug.cgi?id=341772

            Bug ID: 341772
           Summary: Re-read metadata when image file timestamp has changed
           Product: digikam
           Version: unspecified
          Platform: Other
                OS: Linux
            Status: UNCONFIRMED
          Severity: wishlist
          Priority: NOR
         Component: Metadata
          Assignee: [hidden email]
          Reporter: [hidden email]

I would like to have a feature that automatically makes digikam re-read all
image files that has a different time stamp.

If this feature already exists, I apologize that I have not yet found it, and
would gladly take any pointer in how to do this. Otherwise, I'm prepare to
write a path that implements such feature.

This is my use case for this feature.
1) I have a Desktop with all my 88K+ images stored on SSD disks locally. All
metadata are saved to the images.
2) I have a fast home wired/wifi network, and I use sshfs to share and mount
those pics on my laptops.
3) I use a laptop to browse and occasionally update an image here and there
using a local Digikam. I add new face tags, and normal tags.
4) Now, if I start Digikam on the Desktop, these changes will not appear in
Digikam (on the Desktop machine), unless I do a complete re-read of metadata of
all 88K+ images.

My suggestion is to save the time stamps of each image file into the digikam
database. When Digikam starts, it will notice which files has changed (if they
have gotten a changed file time stamp), and initiate a re-read of those
specific image files.

Does this sound like a feature that somebody else than me would like to have?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=341772

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
            Version|unspecified                 |4.5.0

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Alan Pater]

https://bugs.kde.org/show_bug.cgi?id=341772

Alan Pater <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #1 from Alan Pater <[hidden email]> ---
In theory, applications should update the Xmp.xmp.MetadataDate key. That way
other apps would know to re-read metadata.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kristian Karl]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #2 from Kristian Karl <[hidden email]> ---
Interesting. I guess though, that the performance penalty might be much higher,
reading the  Xmp.xmp.MetadataDate key for the image, than reading the image
files time stamp from the operating systems file system. Would do you think?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #3 from Gilles Caulier <[hidden email]> ---
1/ I don't know this XMP tag. Interesting.
2/ I'm not sure that all application will update this tag automatically. In
digiKam case, this must be delegate to Exiv2, as finally library do the
background job.
3/ about time latency to read XMP tag : yes, file system time stamp is a less
time consuming operation. For XMP, file must be open, XMP container found, tag
read, file closed, etc. This will take more time if a XMPS sidecar is present
or not because a check must be operate before to see if original file or
sidecar must be used.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kristian Karl]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #4 from Kristian Karl <[hidden email]> ---
I would be willing to try to implement this feature.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kristian Karl]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #5 from Kristian Karl <[hidden email]> ---
Created attachment 91802
  --> https://bugs.kde.org/attachment.cgi?id=91802&action=edit
Patch with proposed fix and unit test

I realized that Digikam already detectes a change in an image file, using
either file size and/or date time modification.

However, in CollectionScanner, scanModifiedFile(fi, scanInfo) is used to
re-scan the file. That does not re-read metadata changes.
I tested using rescanFile(fi, scanInfo) instead, and that worked much better.

See:
https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/database/collectionscanner.cpp#L1202

The patch contains the proposed change, and a unit test to verify that metadata
changes 'outside' digikam is detected and re-read into the database.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=341772

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Re-read metadata when image |Re-read metadata when image
                   |file timestamp has changed  |file timestamp has changed
                   |                            |[patch]

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=341772

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #6 from Gilles Caulier <[hidden email]> ---
Kristian,

Thanks a lots for your patch.

Marcel, do you want this patch included in Qt4 or Qt5 version of digiKam (or
both) ?

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kristian Karl]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #7 from Kristian Karl <[hidden email]> ---
I'm using KDE 4, so definitely Qt4. But why not both.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #8 from Gilles Caulier <[hidden email]> ---
yes Kristian, this can be applied to both, but i would to see Marcel viewpoint.

As i see, your patch is not intrusive. So for me both is fine. But Marcel know
better than me this implementation, so we will wait feedback from him.

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #9 from Gilles Caulier <[hidden email]> ---
Kristian,

For the moment, digiKam core is ported to Qt5, but it still some task to
achieve.

About scan image and FS detection changes, there is a port to pure Qt5 to do.

See my wiki page for all technical details :

https://techbase.kde.org/Projects/Digikam/CodingSprint2014

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #10 from Marcel Wiesweg <[hidden email]> ---
This is a very sensitive area. The current behavior has been conservative all
these years:
You are not forced to have your db in sync with your metadata. You can update
the metadata in the database and not write the change to the image. You can
switch off writing to RAW images. There may be image formats which contain
metadata but for which writing is not supported or experimental. Plus
constellations that we did not think of.
Against this background the full re-scan has been a manual operation.

If now any change in the file's modification date, which is done so easily,
will cause a full re-scan, we change behavior which was constant over many
years. I see many bugs reports and annoyed users.

The change is completely valid for your setup, but for me it must come with a
setup option to enable it, and must be disabled by default.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kristian Karl]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #11 from Kristian Karl <[hidden email]> ---
Ah, I agree.
I'll rewrite the patch with a suggestion for setup option, which is disabled by
default.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kristian Karl]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #12 from Kristian Karl <[hidden email]> ---
Created attachment 91806
  --> https://bugs.kde.org/attachment.cgi?id=91806&action=edit
Improved patch with UI setting

An improved patch with UI setting and an additional unit test.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Kristian Karl]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #13 from Kristian Karl <[hidden email]> ---
Created attachment 91807
  --> https://bugs.kde.org/attachment.cgi?id=91807&action=edit
New UI setting

The UI setting shown in the Configure dialog.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Maik Qualmann]

https://bugs.kde.org/show_bug.cgi?id=341772

Maik Qualmann <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
      Latest Commit|                            |http://commits.kde.org/digi
                   |                            |kam/937109fb0f3219230825843
                   |                            |7ccad7f8dac536acd
             Status|UNCONFIRMED                 |RESOLVED
   Version Fixed In|                            |4.9.0

--- Comment #14 from Maik Qualmann <[hidden email]> ---
Git commit 937109fb0f32192308258437ccad7f8dac536acd by Maik Qualmann.
Committed on 01/04/2015 at 16:40.
Pushed by mqualmann into branch 'master'.

apply patch #91806 from Kristian Karl to make an optional rescan of files when
files are modified
FIXED-IN: 4.9.0
GUI:

M  +2    -1    NEWS
M  +9    -1    libs/database/collectionscanner.cpp
M  +3    -0    libs/dmetadata/metadatasettingscontainer.cpp
M  +1    -0    libs/dmetadata/metadatasettingscontainer.h
M  +17   -0    tests/CMakeLists.txt
A  +232  -0    tests/timestampupdatetest.cpp     [License: GPL (v2+)]
A  +48   -0    tests/timestampupdatetest.h     [License: GPL (v2+)]
A  +-    --    tests/timestampupdatetestimages/1.jpg
M  +19   -7    utilities/setup/setupmetadata.cpp

http://commits.kde.org/digikam/937109fb0f32192308258437ccad7f8dac536acd

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #15 from Gilles Caulier <[hidden email]> ---
Git commit ca46de929504d17f57c45d8d4e5ff01dcbcef838 by Gilles Caulier.
Committed on 02/04/2015 at 06:53.
Pushed by cgilles into branch 'master'.

use Qt macro
use moc file
fix coding style
code polish

M  +18   -25   tests/timestampupdatetest.cpp
M  +6    -2    tests/timestampupdatetest.h

http://commits.kde.org/digikam/ca46de929504d17f57c45d8d4e5ff01dcbcef838

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #16 from Gilles Caulier <[hidden email]> ---
Git commit b29881591ec06bb1574941686c40f1492ef82b57 by Gilles Caulier.
Committed on 27/03/2015 at 12:51.
Pushed by cgilles into branch 'frameworks'.

backport commit #937109fb0f32192308258437ccad7f8dac536acd from git/master to
frameworks branch

M  +12   -4    libs/database/collection/collectionscanner.cpp
M  +3    -0    libs/dmetadata/metadatasettingscontainer.cpp
M  +1    -0    libs/dmetadata/metadatasettingscontainer.h
M  +34   -14   tests/CMakeLists.txt
A  +225  -0    tests/timestampupdatetest.cpp     [License: GPL (v2+)]
A  +52   -0    tests/timestampupdatetest.h     [License: GPL (v2+)]
A  +-    --    tests/timestampupdatetestimages/1.jpg
M  +19   -7    utilities/setup/setupmetadata.cpp

http://commits.kde.org/digikam/b29881591ec06bb1574941686c40f1492ef82b57

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=341772

--- Comment #17 from Gilles Caulier <[hidden email]> ---
Kristian,

Your patch is applied to next 4.9.0 and backported to future 5.0.0 (Qt5)

did you have another entries in bugzilla where you want to propose another
patch ?

Note : git/master where KDE4 version of digiKam will only accept bugfixes or
not too intrusive patches, because KDE and Qt5 implementation start to be
seriously different. Backporting must still a simple task to do.

There are also few serious tasks to do on Qt5 port, as for ex the migration of
code from KDirWatch to QFileSystemWatcher, to reduce KDE dependencies.

https://techbase.kde.org/Projects/Digikam/CodingSprint2014#digiKam_Core

Best

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel