[digiKam-users] Migrating from Digikam 4 to 6

[J D]

Hi,

I lately installed a new Linux distribution, and along with that change, got new version of Digikam (4 -> 6.4). How can I import the meta-data created using the old one to the new one?

The meta-data is in a database file. I don't like the idea of embedding it into the picture files because that way, changes in meta-data would mean changes in picture files, and that would complicate making back ups.

I found a dialog for migration in the new Digikam, but got immediately confused, as it seems to require me to move (or copy, that was unclear) the pictures from a directory to another along with the migration. There are roughly 35000 pictures taking 475 GB of space so proceeding with that dialog is not something I will try before I know better what is going happen and why.

BR,

angrdtsk

[Gilles Caulier-4]

Le sam. 16 janv. 2021 à 14:01, J D <[hidden email]> a écrit :
>
> Hi,
>
> I lately installed a new Linux distribution, and along with that change, got new version of Digikam (4 -> 6.4). How can I import the meta-data created using the old one to the new one?
>
> The meta-data is in a database file. I don't like the idea of embedding it into the picture files because that way, changes in meta-data would mean changes in picture files, and that would complicate making back ups.

If you use XMP sidecar files (See option in DK/Setup/Metadata panel),
you will not touch image files as well.

>
> I found a dialog for migration in the new Digikam, but got immediately confused, as it seems to require me to move (or copy, that was unclear) the pictures from a directory to another along with the migration. There are roughly 35000 pictures taking 475 GB of space so proceeding with that dialog is not something I will try before I know better what is going happen and why.

In all cases, if you use database migration tool, do not migrate to
6.4 release, but to 7.2. release candidate available here :

https://files.kde.org/digikam/

Best

Gilles Caulier

[J D]

Hi, and thanks for quick reply,

I got Digikam 6.4 along with a Linux distro, so updating it may not be that simple.

Do I have any options in addition to using the migration tool?

What would be the preferred (hopefully future-proof) solution for this case?

BR,

angrdtsk

la 16. tammik. 2021 klo 15.13 Gilles Caulier ([hidden email]) kirjoitti:

Le sam. 16 janv. 2021 à 14:01, J D <[hidden email]> a écrit :
>
> Hi,
>
> I lately installed a new Linux distribution, and along with that change, got new version of Digikam (4 -> 6.4). How can I import the meta-data created using the old one to the new one?
>
> The meta-data is in a database file. I don't like the idea of embedding it into the picture files because that way, changes in meta-data would mean changes in picture files, and that would complicate making back ups.

If you use XMP sidecar files (See option in DK/Setup/Metadata panel),
you will not touch image files as well.

>
> I found a dialog for migration in the new Digikam, but got immediately confused, as it seems to require me to move (or copy, that was unclear) the pictures from a directory to another along with the migration. There are roughly 35000 pictures taking 475 GB of space so proceeding with that dialog is not something I will try before I know better what is going happen and why.

In all cases, if you use database migration tool, do not migrate to
6.4 release, but to 7.2. release candidate available here :

https://files.kde.org/digikam/

Best

Gilles Caulier

[Gilles Caulier-4]

Le sam. 16 janv. 2021 à 14:25, J D <[hidden email]> a écrit :
>
> Hi, and thanks for quick reply,
>
> I got Digikam 6.4 along with a Linux distro, so updating it may not be that simple.

We provide an universal Linux AppImage bundle. There is nothing to
install, it does not install anything. AppImage is just a simple
executable which uncompresses the whole dependencies in a virtual disk
in memory and runs the application using chroot.

It's simple to use.

Just to be sure, backup all your database files previously, in case of...

>
> Do I have any options in addition to using the migration tool?

No, because we have fixed more than 1300 bugs since the 6.x release,
so it will be complicated to resume all problems from previous
releases.

>
> What would be the preferred (hopefully future-proof) solution for this case?
>

This kind of huge migration must be performed carefully. This is my
recommandations:

1/ Backup databases in all cases.
2/ Change Setup/metadata/xmp sidecar option and create a new XMP
sidecar file for all images. Use Maintenance tool.
3/ Stop digiKam and remove your previous database.
4/ start the new 7.2.0 appimage. It will ask to recreate the database
from scratch scanning collection (especially XMP sidecar).
5/ Let's start the scan and take a coffee.

Alternative, more faster of course :

1/ Backup databases in all cases.
2/ run migration database tool
3/ setup original DB and target DB.
4/ process migration and take a coffee.

I always processed the first solution with my huge familly collection
(250Mb on SSD with sqlite).

Best

Gilles Caulier

[Sveinn í Felli-2]

Þann 16.1.2021 13:39, skrifaði Gilles Caulier:
> my huge familly collection
> (250Mb on SSD with sqlite).

Guess you mean 250Gb, not 250Mb?
I've got several images much larger than 250Mb each... ;-)

best,
Sveinn í Felli

[Gilles Caulier-4]

yes (:=))) 250Gb... with plenty of large panorama

Gilles

Le sam. 16 janv. 2021 à 15:27, Sveinn í Felli <[hidden email]> a écrit :

>
> Þann 16.1.2021 13:39, skrifaði Gilles Caulier:
> > my huge familly collection
> > (250Mb on SSD with sqlite).
>
> Guess you mean 250Gb, not 250Mb?
> I've got several images much larger than 250Mb each... ;-)
>
> best,
> Sveinn í Felli

[J D]

Does the migration tool move the picture files or make copies of them? Is it possible to do the migration in-place?

BR,

angrdtsk

la 16. tammik. 2021 klo 15.40 Gilles Caulier ([hidden email]) kirjoitti:

Le sam. 16 janv. 2021 à 14:25, J D <[hidden email]> a écrit :
>
> Hi, and thanks for quick reply,
>
> I got Digikam 6.4 along with a Linux distro, so updating it may not be that simple.

We provide an universal Linux AppImage bundle. There is nothing to
install, it does not install anything. AppImage is just a simple
executable which uncompresses the whole dependencies in a virtual disk
in memory and runs the application using chroot.

It's simple to use.

Just to be sure, backup all your database files previously, in case of...

>
> Do I have any options in addition to using the migration tool?

No, because we have fixed more than 1300 bugs since the 6.x release,
so it will be complicated to resume all problems from previous
releases.

>
> What would be the preferred (hopefully future-proof) solution for this case?
>

This kind of huge migration must be performed carefully. This is my
recommandations:

1/ Backup databases in all cases.
2/ Change Setup/metadata/xmp sidecar option and create a new XMP
sidecar file for all images. Use Maintenance tool.
3/ Stop digiKam and remove your previous database.
4/ start the new 7.2.0 appimage. It will ask to recreate the database
from scratch scanning collection (especially XMP sidecar).
5/ Let's start the scan and take a coffee.

Alternative, more faster of course :

1/ Backup databases in all cases.
2/ run migration database tool
3/ setup original DB and target DB.
4/ process migration and take a coffee.

I always processed the first solution with my huge familly collection
(250Mb on SSD with sqlite).

Best

Gilles Caulier

[Andrew Goodbody]

No, the migration tool will not move or copy your picture files. At
least not unless you are doing something unusual that you have not told
us about.

I suspect that you may be suffering from the change in location of where
the digikamrc file was stored, so the new version of digikam is not
aware of where the old version kept its settings. That may not be well
dealt with as you are leaping over a major release, ie digikam 5. But
then I do not remember precisely when that happened, so maybe not.

But you are moving from a very old and obsolete version of digikam to an
old and obsolete version of digikam. You would do better to follow
Giles' advice and download the latest release appimage for digikam 7.

If there is anything you are not sure about please ask a specific
question with as much detail as possible. Explain what you see and why
it concerns you. I do not remember ever needing to run any sort of
migration so I cannot guide you on it with the question you have asked.
You are assuming a level of familiarity with a once in a lifetime task
that we simply do not have nor should we be expected to have.

Andrew

On 16/01/2021 15:18, J D wrote:

> Does the migration tool move the picture files or make copies of them?
> Is it possible to do the migration in-place?
>
> BR,
> angrdtsk
>
> la 16. tammik. 2021 klo 15.40 Gilles Caulier ([hidden email]
> <mailto:[hidden email]>) kirjoitti:
>
>     Le sam. 16 janv. 2021 à 14:25, J D <[hidden email]
>     <mailto:[hidden email]>> a écrit :
>      >
>      > Hi, and thanks for quick reply,
>      >
>      > I got Digikam 6.4 along with a Linux distro, so updating it may
>     not be that simple.
>
>     We provide an universal Linux AppImage bundle. There is nothing to
>     install, it does not install anything. AppImage is just a simple
>     executable which uncompresses the whole dependencies in a virtual disk
>     in memory and runs the application using chroot.
>
>     It's simple to use.
>
>     Just to be sure, backup all your database files previously, in case
>     of...
>
>      >
>      > Do I have any options in addition to using the migration tool?
>
>     No, because we have fixed more than 1300 bugs since the 6.x release,
>     so it will be complicated to resume all problems from previous
>     releases.
>
>      >
>      > What would be the preferred (hopefully future-proof) solution for
>     this case?
>      >
>
>     This kind of huge migration must be performed carefully. This is my
>     recommandations:
>
>     1/ Backup databases in all cases.
>     2/ Change Setup/metadata/xmp sidecar option and create a new XMP
>     sidecar file for all images. Use Maintenance tool.
>     3/ Stop digiKam and remove your previous database.
>     4/ start the new 7.2.0 appimage. It will ask to recreate the database
>     from scratch scanning collection (especially XMP sidecar).
>     5/ Let's start the scan and take a coffee.
>
>     Alternative, more faster of course :
>
>     1/ Backup databases in all cases.
>     2/ run migration database tool
>     3/ setup original DB and target DB.
>     4/ process migration and take a coffee.
>
>     I always processed the first solution with my huge familly collection
>     (250Mb on SSD with sqlite).
>
>     Best
>
>     Gilles Caulier
>

[Rob D]

There are many ways to skin a cat when it comes to upgrading your copy of Digikam.

As you are upgrading from a pretty old version, these are the steps >I< would do things in:

1) make sure you have a current backup of your photos.  From your earlier messages, it appears you don't like to write the metadata to your photos as it messes with your backups, so I imagine you are doing incremental backups.  Just make sure everything is up do date with your backup first

2)  configure Digikam to write all metadata into sidecar files (.xmp)  Once done, sync the metadata for all your photos to disk.  This will take a bit of time, and will write a .xmp file for every photo

3)  MOVE your digikam database files and digikam settings folder to a safe place.  You want these to be re-created when you run digikam the next time

4) download the appimage for the current 7.2 release candidate for Digikam and mark it executable (either command line or viewing the properties from the desktop) 

     You can find the latest release candidate at   https://files.kde.org/digikam/    You will want the X86-64 file as you are running a current linux distribution

5) Run the appimage (double click on the appimage icon from your file browser on the desktop) and perform the first run configuration pointing digikam at your photo library.  Once it starts scanning photos, take a break. It might take a while

6) if everything looks good, uninstall your old copy of digikam, and remove the old database files and settings folder.

7) In case things don't work, restore your original database files and digikam settings folder and appimage file  to revert back to the old version.  

8) configure Digikam settings and update for your preferred metadata settings.  As .xmp files are quite small, it might be acceptable to write to .xmp files in addition to the database in the future as it won't affect your backup size too dramatically.

9) for bonus points and if you're bored enduring a COVID lockdown, scan your entire collecton for faces and be prepared to name a lot of people.

Again, this is how I would upgrade my digikam as it gives the ability to roll back at any point until you remove the old database and configuration files. 

Rob

On Sat, 2021-01-16 at 15:25 +0200, J D wrote:

Hi, and thanks for quick reply,

I got Digikam 6.4 along with a Linux distro, so updating it may not be that simple.

Do I have any options in addition to using the migration tool?

What would be the preferred (hopefully future-proof) solution for this case?

BR,

angrdtsk

la 16. tammik. 2021 klo 15.13 Gilles Caulier ([hidden email]) kirjoitti:

Le sam. 16 janv. 2021 à 14:01, J D <[hidden email]> a écrit :
>
> Hi,
>
> I lately installed a new Linux distribution, and along with that change, got new version of Digikam (4 -> 6.4). How can I import the meta-data created using the old one to the new one?
>
> The meta-data is in a database file. I don't like the idea of embedding it into the picture files because that way, changes in meta-data would mean changes in picture files, and that would complicate making back ups.

If you use XMP sidecar files (See option in DK/Setup/Metadata panel),
you will not touch image files as well.

>
> I found a dialog for migration in the new Digikam, but got immediately confused, as it seems to require me to move (or copy, that was unclear) the pictures from a directory to another along with the migration. There are roughly 35000 pictures taking 475 GB of space so proceeding with that dialog is not something I will try before I know better what is going happen and why.

In all cases, if you use database migration tool, do not migrate to
6.4 release, but to 7.2. release candidate available here :

https://files.kde.org/digikam/

Best

Gilles Caulier

[J D]

Hi,

Thanks for all your suggestions!

It is interesting and also good to see that so much has happened with Digikam. I mean, my upgrade of Linux was from year 2017 to year 2020, and I don't remember that my previous such upgrade would have brought such changes regarding Digikam.

I have the old installation and one copy of the photos on an old-style hard disk and the old installation and another copy of photos on a brand new SSD. It seems that I need to go back to the old disk and the old system to do the move to sidecar files and then re-sync the old and the new disk. I may actually end up creating a small test album and try the migration with that first.

And yes, I do have incremental back-up system.

BR,

angrdtsk

la 16. tammik. 2021 klo 22.17 Rob Dueckman ([hidden email]) kirjoitti:

There are many ways to skin a cat when it comes to upgrading your copy of Digikam.

As you are upgrading from a pretty old version, these are the steps >I< would do things in:

1) make sure you have a current backup of your photos.  From your earlier messages, it appears you don't like to write the metadata to your photos as it messes with your backups, so I imagine you are doing incremental backups.  Just make sure everything is up do date with your backup first

2)  configure Digikam to write all metadata into sidecar files (.xmp)  Once done, sync the metadata for all your photos to disk.  This will take a bit of time, and will write a .xmp file for every photo

3)  MOVE your digikam database files and digikam settings folder to a safe place.  You want these to be re-created when you run digikam the next time

4) download the appimage for the current 7.2 release candidate for Digikam and mark it executable (either command line or viewing the properties from the desktop) 

     You can find the latest release candidate at   https://files.kde.org/digikam/    You will want the X86-64 file as you are running a current linux distribution

5) Run the appimage (double click on the appimage icon from your file browser on the desktop) and perform the first run configuration pointing digikam at your photo library.  Once it starts scanning photos, take a break. It might take a while

6) if everything looks good, uninstall your old copy of digikam, and remove the old database files and settings folder.

7) In case things don't work, restore your original database files and digikam settings folder and appimage file  to revert back to the old version.  

8) configure Digikam settings and update for your preferred metadata settings.  As .xmp files are quite small, it might be acceptable to write to .xmp files in addition to the database in the future as it won't affect your backup size too dramatically.

9) for bonus points and if you're bored enduring a COVID lockdown, scan your entire collecton for faces and be prepared to name a lot of people.

Again, this is how I would upgrade my digikam as it gives the ability to roll back at any point until you remove the old database and configuration files. 

Rob

On Sat, 2021-01-16 at 15:25 +0200, J D wrote:

Hi, and thanks for quick reply,

I got Digikam 6.4 along with a Linux distro, so updating it may not be that simple.

Do I have any options in addition to using the migration tool?

What would be the preferred (hopefully future-proof) solution for this case?

BR,

angrdtsk

la 16. tammik. 2021 klo 15.13 Gilles Caulier ([hidden email]) kirjoitti:

Le sam. 16 janv. 2021 à 14:01, J D <[hidden email]> a écrit :
>
> Hi,
>
> I lately installed a new Linux distribution, and along with that change, got new version of Digikam (4 -> 6.4). How can I import the meta-data created using the old one to the new one?
>
> The meta-data is in a database file. I don't like the idea of embedding it into the picture files because that way, changes in meta-data would mean changes in picture files, and that would complicate making back ups.

If you use XMP sidecar files (See option in DK/Setup/Metadata panel),
you will not touch image files as well.

>
> I found a dialog for migration in the new Digikam, but got immediately confused, as it seems to require me to move (or copy, that was unclear) the pictures from a directory to another along with the migration. There are roughly 35000 pictures taking 475 GB of space so proceeding with that dialog is not something I will try before I know better what is going happen and why.

In all cases, if you use database migration tool, do not migrate to
6.4 release, but to 7.2. release candidate available here :

https://files.kde.org/digikam/

Best

Gilles Caulier

[Jyk]

I am in a similar case, many thanks for the procedures shared by Gilles and
Rob.

I am facing some trouble in step 2 "Change Setup/metadata/xmp sidecar option
and create a new XMP
sidecar file for all images. Use Maintenance tool"

I chose "From database to image metada" and "Write to XML Only" and selected
all the choices in "Write This information to Metadata".

And unfortunately all my images been impacted in their unix file status,
very surprising after carefuly selected  "Write to XML Only".

I compared md5sum of files with backup, they are not modified, and modified
time also not modified, only status time modified, so I will continue
procedure to migrate from 4 to 6, but I still don understand why unix file
status be modified.

Jyk



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Jyk]

At the end of the Gilles and Rob procedures I was without any tags, quite
disappointing.

I reapplied everything for the same results, as my PC is not so fast or I
have too many pictures, a full run requires multiples days for "reboot in
Ubuntu 16, restore from backups,  export with Digikam 4 tags in XML sidecar
files via Maintenance tool synchronization from database to image metadata,
reboot in Ubuntu 20, start Digikam v7 from an appimage, choose fresh install
configuration, let Digikam analyze pictures, start Maintenance tool
synchronization from image metadata to database"

After two runs, I built a fresh new Ubuntu user, and tried again with
hundreds pictures copied from my main account, with their sidecar xml files.

And I found a change from Digikam 4 to Digikam 7 that puzzled me on the main
tries.

*The sidecar options are now in a separate tab*, and I missed them, due to
new dialog for DNG format, I wrongly thought it was somewhere a replacement
for XML files.

So I reran on my small subset of files the Maintenance tool synchronization
from image metadata to database, and fortunately it retrieved my tags.

I have now to try on the real set of images.




--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html