[digikam] [Bug 364258] New: Upgrade to 5.0 (beta6) and "Migrate Database" destroys tags and location data of ALL photos

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

            Bug ID: 364258
           Summary: Upgrade to 5.0  (beta6) and "Migrate Database"
                    destroys tags and location data of ALL photos
           Product: digikam
           Version: 5.0.0
          Platform: Other
                OS: Linux
            Status: UNCONFIRMED
          Severity: critical
          Priority: NOR
         Component: Database-Migration
          Assignee: [hidden email]
          Reporter: [hidden email]

So I upgraded to Ubuntu 16.04 to be able to test Digikam 5beta6. After the
upgrade, I started Digikam 5 and was a little surprised that it did not offer
to import / migrate / update the Digikam 4 database I already had, but behaved
like a new installation.

(Is this intentional during the beta phase so that production (Digikam 4.14)
and testing (Digikam 5) can be kept separate?)

Since digikam did not detect my Digikam4 library I went to Settings > Database
Migration and clicked "Migrate" with the default settings (SQlite format and
the same path on both sides, after all, Digikam 5 did not find my library, so
it probably uses another file name!) But it didn't work, even afterwards
Digikam did not find my library.

What's more, after closing and reopening Digikam 4.14, it did not display any
photos - and all my locations, tags, album comments and so on were gone!!! It
seems they were deleted from digikam4.db; the "Tags" table was empty, so was
the "Settings" table and some others.

IMHO there should be no way of doing this. Digikam should warn before
overwriting data or modifying it so that an old version cannot read it any
more. (Was this even the right procedure to get Digikam 5 set up?)



Reproducible: Always

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Upgrade to 5.0  (beta6) and |Upgrade to 5.0  (beta6) :
                   |"Migrate Database" destroys |tags and location data of
                   |tags and location data of   |photos are lost
                   |ALL photos                  |
                 CC|                            |[hidden email]
          Component|Database-Migration          |Database

--- Comment #1 from [hidden email] ---
No.

digiKam 5 ans Sqlite DB still exactly the same. Nothing has changed. DB file is
the same that digiKam 4.x. SQL schema for Sqlite still untouched.

The update from a 4.x to a 5.x do not require a migration.

The problem is in your computer settings.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|critical                    |normal

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #2 from Jens B. Benecke <[hidden email]> ---
"My Computer Settings" - a.k.a. Digikam 5 - permit overwriting digikam4.db with
the exact same file (digikam4.db) in the migration dialog. I think this is the
problem. Can you please verify and if so, add a check that the migration
database files must be different.

Thanks!

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #3 from [hidden email] ---
Why do you talk about "migration" to import DK sqlite database from version 4
to 5 ?

As i said before, it'z the same file !

There is nothing to migrate. Just configure DK 5 as DK 4. that all...

Migration tool is to switch from one DB type to another and different one. Not
to the same DB type from original to target.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #4 from Jens B. Benecke <[hidden email]> ---
Let me explain.

Digikam 5 (beta6), when started first, did not recognize my existing albums and
collections from Digikam 4.14 (on Ubuntu Linux) and just showed me a welcome
and setup screen. So I logically assumed there must be a migration procedure.
(This I would consider a bug - if there are previous existing collections, a
new version of Digikam should just open them or offer to import them.)

I found the migration option and selected my digikam4.db (the target was
already preselected) and started the migration. This destroyed the digikam4.db
tables as described below so subsequent starts of Digikam 4.14 wouldn't show
tags etc. any more.
Starting the migration might have been my mistake but a user friendly
application should prevent users from doing something stupid (like starting a
migration when it isn't required or possible) so I consider this a bug too.

The solution is simple: upon start, just load the digikam4 config (if
available) and - if required - copy it to a digikam5 config. And when a user
starts the Migration task from the maintenance menu and selects database files
or formats which cannot be migrated, show an appropriate error message and
refuse the migration.

Do you agree?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #5 from [hidden email] ---
>Digikam 5 (beta6), when started first, did not recognize my existing albums and >collections from Digikam 4.14 (on Ubuntu Linux) and just showed me a welcome and >setup screen. So I logically assumed there must be a migration procedure.

No. The reason why the old collection is not recognized (and first run
assistant launched) is the location of  digikamrc file including all DK
configuration which have been moved from ~/.kde4/share/config to ~/.config
which is the standardized place under Linux. If you copy this file at this
place all will start as expected.

In all cases, the migration option is not the right way to process.

It's right than no rule to copy old digikamrc file from old place to new one is
not implemented.

But, starting with a new fresh DK setup and configure the collections and the
place where the database is stored will normally not touch the DB content and
all will be available as with 4.x release.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #6 from Jens B. Benecke <[hidden email]> ---
> If you copy this file at this place all will start as expected.

Do you expect a normal user (= non-geek, non-programmer) to know that?
Then this is a usability issue, not a technical one. But an important one.
Or is Digikam not meant to be used by non-geeks?

> In all cases, the migration option is not the right way to process.

How should a user know? He can't. So prevent the migration from being abused in
this case. Users should not be able to perform tasks that destroy data,
otherwise they lose their trust in the application.

> But, starting with a new fresh DK setup and configure the collections and the place where the database is stored will normally not touch the DB content and all will be available as with 4.x release.

OK, but - again - (how) do you expect a normal user (= non-geek,
non-programmer) to know that? From a usability perspective, Digikam5 should
proactively offer to import all Digikam4 settings, collections and so on - no
matter where they were saved previously. Nobody wants to research how to do
this manually, even if it is possible.

Please don't get me wrong, I like Digikam a lot and I want it to be successful.
This is why these things are so important to me.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |CONFIRMED
     Ever confirmed|0                           |1

--- Comment #7 from [hidden email] ---
I confirm that older place of digikamrc file from .kde4 is not backported in
new .config at first run of 5.0.0.

There is a copy method in KDE AP :

https://api.kde.org/frameworks/kconfig/html/classKConfig.html#adff2a11187952db0c4ad5fbff0c11bad

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #8 from [hidden email] ---
Files to backport under Linux :

- ~/.kde4/share/config/digikamrc
- ~/.kde4/share/config/digikam_tagsmanagerrc
- ~/.kde4/share/config/kipipluginsrc
- ~/.kde4/share/config/kipirc
- ~/.kde4/share/config/showfotorc
- ~/.kde4/share/apps/digikam/

New place under Linux :

- ~/.config/digikamrc
- ~/.config/digikam_tagsmanagerrc
- ~/.config/kipipluginsrc
- ~/.config/kipirc
- ~/.config/showfotorc
- ~/.local/share/apps/digikam/*

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #9 from [hidden email] ---
Files to backport under OSX :

- ~/Library/Preferences/KDE/share/config/digikamrc
- ~/Library/Preferences/KDE/share/config/digikam_tagsmanagerrc
- ~/Library/Preferences/KDE/share/config/kipipluginsrc
- ~/Library/Preferences/KDE/share/config/kipirc
- ~/Library/Preferences/KDE/share/config/showfotorc
- ~/Library/Preferences/KDE/share/apps/digikam/

New place under OSX :

- ~/Library/Preferences/digikamrc
- ~/Library/Preferences/digikam_tagsmanagerrc
- ~/Library/Preferences/kipipluginsrc
- ~/Library/Preferences/kipirc
- ~/Library/Preferences/showfotorc
- ~/Library/Preferences/share/apps/digikam/*

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #10 from [hidden email] ---
Files to backport under WINDOWS :

- ~/AppData/Local/digikamrc
- ~/AppData/Local/digikam_tagsmanagerrc
- ~/AppData/Local/kipipluginsrc
- ~/AppData/Local/kipirc
- ~/AppData/Local/showfotorc
- ~/AppData/Local/digikam/

New place under WINDOWS :

- ~/Local Settings/digikamrc
- ~/Local Settings/digikam_tagsmanagerrc
- ~/Local Settings/kipipluginsrc
- ~/Local Settings/kipirc
- ~/Local Settings/showfotorc
- ~/Local Settings/digikam/*

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|Database                    |setup
            Summary|Upgrade to 5.0  (beta6) :   |Upgrade to 5.0  (beta6)
                   |tags and location data of   |:KDE4 application
                   |photos are lost             |configuration rc files
                   |                            |ignored

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

Alexey Stukalov <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #11 from Alexey Stukalov <[hidden email]> ---
That's amazing! The bug is there for a month and I guess the developers
understand its consequences and the fix is not very complex. Guess what? I've
just overwritten my old digikam4.db in the migration dialog without digikam
asking for any confirmation. Years of tags are lost. Well done digikam devs!

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #12 from [hidden email] ---
Alexey,

This is not implemented because we seen that old digikam 4.x settings from
digikamrc file crash violently the application and we haven't yet identified
which one exactly.

You must always make a backup of your database before a major update.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #13 from Jens B. Benecke <[hidden email]> ---
Backups are a must anyway. (IMHO: Just install deja-dup and setup some remote
storage (a NAS or a cloud service) and stop thinking about it.)

Specifically, did you narrow down your search about 4.x settings digikam 5 so
that I can help doing some bisect testing?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #14 from Jens B. Benecke <[hidden email]> ---
Also which bug report is about these crashes?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

--- Comment #15 from [hidden email] ---
This one for ex :

https://bugs.kde.org/show_bug.cgi?id=365079

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #16 from [hidden email] ---
(In reply to caulier.gilles from comment #8)

> Files to backport under Linux :
>
> - ~/.kde4/share/config/digikamrc
> ...
> - ~/.kde4/share/apps/digikam/
>
> New place under Linux :
>
> - ~/.config/digikamrc
> ...

Here the migration was failed. It has removed all the databases entries :(
Here is a sample of messages when it scan the databases the 1st run :

digikam.database: Creating new Location  "/"  uuid
"volumeid:?path=%2Fhome%2Fdomi%2FPhotos"
digikam.database: location for  "%2Fhome%2Fdomi%2FPhotos"  is available  true
KMemoryInfo: Platform identified :  "LINUX"
KMemoryInfo: TotalRam:  8376160256
digikam.general: Allowing a cache size of 200 MB
digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret =  0
digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret =  0
digikam.thumbsdb: Thumbs database: have a structure version  "3"
digikam.general: Thumbnails database ready for use
digikam.database: Removed items: (1, 2, 3, 4, 5, 6) related items: ()
digikam.database: Removed items: (9) related items: ()
digikam.database: Removed items: (15747, 15945, 16033) related items: ()
digikam.database: Removed items: (2410, 2411, 2412, 2413, 2414, 2415, 2416,
2417, 2418, 2419, 2420, 2421, 2422, 2423, 2424, 2425, 2426, 2427, 2428, 2429,
2430, 2431, 2432, 2433, 2434, 2435, 2436, 2437, 2438, 2439, 2440, 2441, 2442,
2443, 2444, 2445, 2446, 2447, 2448, 2449, 2450, 2451, 2452, 2453, 2454, 2455,
2456) related items: ()
...
digikam.database: Removed items: (16304, 16305, 16306, 16307, 16308, 16309,
16310, 16311, 16312, 16313) related items: ()
digikam.database: Folder does not exist or is not readable:
"%2Fhome%2Fdomi%2FPhotos"

Now, when it starts, I have this message :
digikam.database: Folder does not exist or is not readable:
"%2Fhome%2Fdomi%2FPhotos"

Why %2F in place of slash ? it should be /home/domi/Photos" ??!!!
in digikamrc file, the path is correct :
Album Path=/home/domi/Photos
Database File Path=/home/domi/Photos

I have migrated another user successfully, and his folder appears correctly
with slash in path not %2F...

I think it's a clue.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=364258

Kristian <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #17 from Kristian <[hidden email]> ---
I think as well that the current non-existing migration process from digikam 4
to 5 is pretty user unfriendly. I wondered also why I had to reconfigure
everything and my templates were gone. After some searching, I found this bug.
At least, my database wasn't fucked up. Since several bad experiences with
digikam breaking its db, I'm very cautious in this regard.

I would have expected at least a piece of information somewhere that the config
paths changed. It shouldn't be so hard to implement a notice in the first-run
wizard. Alternatively, one could ask the user if he did use digikam 4 before
and if so, display some warnings. You could explain that the config path did
change and that the files can be copied by the user at own risk.
Furthermore, is there any reason why you do not make a copy of digikam4.db and
save it as digikam5.db (although the db didn't change)?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel