[digikam] [Bug 373003] New: wrongly calculated uniqueHash

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[digikam] [Bug 373003] New: wrongly calculated uniqueHash

bugzilla_noreply
https://bugs.kde.org/show_bug.cgi?id=373003

            Bug ID: 373003
           Summary: wrongly calculated uniqueHash
           Product: digikam
           Version: 5.2.0
          Platform: openSUSE RPMs
                OS: Linux
            Status: UNCONFIRMED
          Severity: major
          Priority: NOR
         Component: Database-Mysql
          Assignee: [hidden email]
          Reporter: [hidden email]
  Target Milestone: ---

A new uniqueHash algorithm was implemented a few years ago. See
https://bugs.kde.org/show_bug.cgi?id=210353. The new algorithm in uniqueHashV2
was only made available for newly created databases. An already existing
database was (and is till today) still using the old uniqueHash algorithm. My
10 years old digikam db still uses the old uniqueHash algo. The old algo worked
fine till digikam 4.14. During the porting to DK 5, a bug was introduced for
the old algo:

In dimgloaded.cpp, the KDE4 command for the old v1 uniqueHash algo

   KMD5 md5;
   ...
   hash = md5.hexDigest();

was replaced in the KDE5 porting with ( still uniqueHash v1)

   QCryptographicHash md5(QCryptographicHash::Md5);
   ...
   hash = md5.result();

which is wrong. Instead, it should have been replaced by

   QCryptographicHash md5(QCryptographicHash::Md5);
   ...
   hash = md5.result().toHex();

See the migration document
https://api.kde.org/frameworks/kdelibs4support/html/classKMD5.html#a49b31c3e5eb180fa8be8c8f576266221

As a consequence, a database created before 2010 calculates different
uniqueHash values in DK5 compared to DK4. Furthermore, UTF8 cannot be used
since some of the now calculated uniqueHash values are not valid UTF8
characters, the SQL server erorrs with

  "REPLACE INTO Images  ( album, name, status, category, modificationDate,
  fileSize, uniqueHash )  VALUES (?,?,?,?,?,?,?);"
  Error messages: "QMYSQL3: Unable to execute statement" "Incorrect string
  value: '\\xF1\\x9E\\xA6\\x87' for column 'uniqueHash' at row 1" 1366 2
  Bound values:  (QVariant(int, 118053), QVariant(QString, "P4240656.JPG"),
  QVariant(int, 1), QVariant(int, 1), QVariant(QString,
  "2016-10-30T19:16:42"), QVariant(qlonglong, 6951936), QVariant(QString,
  "m�7�\U0005E987"))

There is uniqueHashVersion in the System table, but the value "2" in DK4 and
DK5 return different hash values.

The fix would be easy, just add the missing ".toHex()". However, all images
added in DK5 already contain the invalid hash values. They need legacy
treatment.

The most clean solution is to get rid of hashValue version 1 and recalculate
all hash values in the next DK release

As as sidenote: hashValueV2 is not affected by this bug. This means a db
created after 2010 is not affected

--
You are receiving this mail because:
You are the assignee for the bug.
Reply | Threaded
Open this post in threaded view
|

[digikam] [Bug 373003] wrongly calculated uniqueHash

bugzilla_noreply
https://bugs.kde.org/show_bug.cgi?id=373003

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #1 from [hidden email] ---
You are right.

A patch can be easy to make against git/master...

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
Reply | Threaded
Open this post in threaded view
|

[digikam] [Bug 373003] wrongly calculated uniqueHash

bugzilla_noreply
In reply to this post by bugzilla_noreply
https://bugs.kde.org/show_bug.cgi?id=373003

Maik Qualmann <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|                            |5.4.0
         Resolution|---                         |FIXED
             Status|UNCONFIRMED                 |RESOLVED
      Latest Commit|                            |https://commits.kde.org/dig
                   |                            |ikam/2b8265ab1c785fae668dec
                   |                            |4b468242cc898acc83

--- Comment #2 from Maik Qualmann <[hidden email]> ---
Git commit 2b8265ab1c785fae668dec4b468242cc898acc83 by Maik Qualmann.
Committed on 27/11/2016 at 21:39.
Pushed by mqualmann into branch 'master'.

fix wrongly calculated uniqueHash v1
FIXED-IN: 5.4.0

M  +2    -1    NEWS
M  +1    -1    libs/dimg/loaders/dimgloader.cpp
M  +2    -1    utilities/setup/setupdatabase.cpp

https://commits.kde.org/digikam/2b8265ab1c785fae668dec4b468242cc898acc83

--
You are receiving this mail because:
You are the assignee for the bug.
Reply | Threaded
Open this post in threaded view
|

[digikam] [Bug 373003] wrongly calculated uniqueHash

bugzilla_noreply
In reply to this post by bugzilla_noreply
https://bugs.kde.org/show_bug.cgi?id=373003

Maik Qualmann <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #3 from Maik Qualmann <[hidden email]> ---
If you are still at uniqueHash v1 version, see you in setup->database the
update button for the uniqueHash v2 version.

Maik

--
You are receiving this mail because:
You are the assignee for the bug.
Reply | Threaded
Open this post in threaded view
|

[digikam] [Bug 373003] wrongly calculated uniqueHash

bugzilla_noreply
In reply to this post by bugzilla_noreply
https://bugs.kde.org/show_bug.cgi?id=373003

--- Comment #4 from Kusi <[hidden email]> ---
Maik, where exactly can I update to v2? I don't see any option in the settings
of DK5.2

Kusi

PS: thanks for the fast fix!

--
You are receiving this mail because:
You are the assignee for the bug.
Reply | Threaded
Open this post in threaded view
|

[digikam] [Bug 373003] wrongly calculated uniqueHash

bugzilla_noreply
In reply to this post by bugzilla_noreply
https://bugs.kde.org/show_bug.cgi?id=373003

--- Comment #5 from Maik Qualmann <[hidden email]> ---
Created attachment 102491
  --> https://bugs.kde.org/attachment.cgi?id=102491&action=edit
database.png

See screenshot, if the database has not yet been converted to V2, this entry
appears. Otherwise, the entry in the Settings table should be reset external.

SQL Command:
UPDATE digikam.Settings SET value=1 WHERE keyword=uniqueHashVersion;

Maik

--
You are receiving this mail because:
You are the assignee for the bug.
Reply | Threaded
Open this post in threaded view
|

[digikam] [Bug 373003] wrongly calculated uniqueHash

bugzilla_noreply
In reply to this post by bugzilla_noreply
https://bugs.kde.org/show_bug.cgi?id=373003

--- Comment #6 from Kusi <[hidden email]> ---
thanks, that solved my problem! Just for reference, before your fix, the
"update hash" GUI did not work.

For me, the issue is solved, but I'm sure there are thousands of installations
out there with invalid uniqueHash values. I strongly recommend to force a
uniqueHash update to v2 in DK 5.4.

Furthermore, most of the SQL errors are not sent to the GUI, they are only
visible on the command line. The user just don't see images or other weird
behavior and has no clue why. It would be really helpful to be more strict and
let the user know in case of errors.

--
You are receiving this mail because:
You are the assignee for the bug.
Reply | Threaded
Open this post in threaded view
|

[digikam] [Bug 373003] wrongly calculated uniqueHash

bugzilla_noreply
In reply to this post by bugzilla_noreply
https://bugs.kde.org/show_bug.cgi?id=373003

Wolfgang Scheffner <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #7 from Wolfgang Scheffner <[hidden email]> ---
>Furthermore, most of the SQL errors are not sent to the GUI, they are only visible on the command line. The user just don't see images or other weird behavior and has no clue why. It would be really helpful to be more strict and let the user know in case of errors.< (Kusi)

As one of the most silly users (don't even know what a unique hash algorithm
is) I can only support this as a matter of principle.

--
You are receiving this mail because:
You are the assignee for the bug.