[digiKam-users] Corrupted digikam database repair - easy solution?

[zwyklydigikam]

Hello,
My database with 170 000 photos is corrupted (something went wrong with face
recognition or even before that, thats the time i found error after crash).
When DK starts, housands of errors in windows event viewer, two of them loop
with different numbers:
"
InnoDB: Your database may be corrupt or you may have copied the InnoDB
tablespace but not the InnoDB log files.
InnoDB: Page [page id: space=9, page number=3136] log sequence number
114156703 is in the future! Current system log sequence number 1649127.
"

DK is working fine i think (you wont see any error info, without event
viewer), but cannot save mysql_install_db.exe and mysqld.exe. When i click
OK and go back, still missing.
I don't know if database is corrupt because of missing .exe files or exe
files missing because of corrupted database.

I was trying to repair databse, but cannot find a good simple tutorial
without studing whole mysql books. All say, use the innodb_force_recovery =
1,2,3,4,5,6, but where, how, which ini file, i tried all of them, nothing
changes, maybe digikam is a bit different, they use different config files,
but i cannot find them on my HDD.

Maybe i'm doing something wrong, or everything, but is there any one/two
click solution to fix those errors and repair database? If its just changing
some files in confing files, why not make a button in digikam "Reapair
problems" or sth like that, aby selfheal? ;)
I could try to find a fix, probably can fix it by myself with google (i know
some IT) but lots of users, photographers have no idea how to open .ini
file, not to mention understanding windows event logs etc.

I've installed digikam on so many computers (friends, familly...) and they
will probably lose everything one day with 1 error, without IT support
(months of tagging pictures).

BTW. I got another digikam on linux, and its ok.

Windows 10 64x
Digikam 6.0.0 beta2 64x
Mariadb 10.3.10
Mysql (experimantal)
Face recognition (experimental)
My ram and photo hdd partition without bad sectors.

Thanks!



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

Le 10/12/2018 à 13:35, zwyklydigikam a écrit :
> Hello,
> My database with 170 000 photos is corrupted (something went wrong with face
> recognition

didn't you make a backup before using experimental thing?

do you have data also saved on image or sidecar files?

jdd

--
http://dodin.org

[zwyklydigikam]

[hidden email] wrote
> didn't you make a backup before using experimental thing?
>
> do you have data also saved on image or sidecar files?

I use experimental from start, generating default SQL db takes about 2 days
with 200k images, and everything is slow. MySQL is the only choice.

I never save on image.

I have old backups but how can i be sure if it IS a good backup? Lets say
someone is making backup every month, database is 4gb. Now your DK crashed,
and you use your backup, but your backup is damaged, and you don't even know
that. DK is not showing any errors or warnings, i was "lucky" because it
crashed, and i checked windows logs. There should be warning in DK, and if
someone didnt make backup, DK should give auto-repair option (all say thats
one line of code) insted of hidden comment in windows logs asking to study
SQL (which took me 6 hours and didn't work, but it looks quite simple):

"InnoDB: Your database may be corrupt or you may have copied the InnoDB
tablespace but not the InnoDB log files. Please refer to
https://mariadb.com/kb/en/library/xtradbinnodb-recovery-modes/ for
information about forcing recovery.

For more information, see Help and Support Center at http://www.mysql.com."

I hope its because of 6.0.0 beta2, but if you ask me, not losing months of
work should be priority.

Going back to my problem, how to repair mysql database without backup? I'm
doing something wrong with mariadb force recovery, or maybe it's not the
right direction?

Sorry for my english
Thank you



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Maik Qualmann]

Use the last digiKam-6.0.0-beta3 from 08.12 from here:

https://files.kde.org/digikam/

This is important for the migration because this version fixes potential bugs.
Then use the migration tool to either export the database to a SQLite DB and
then import it back to MySQL. Or create an additional digiKam database and
export the broken ones to the new one.

Maik

Am Montag, 10. Dezember 2018, 13:35:49 CET schrieb zwyklydigikam:

> Hello,
> My database with 170 000 photos is corrupted (something went wrong with face
> recognition or even before that, thats the time i found error after crash).
> When DK starts, housands of errors in windows event viewer, two of them
> loop with different numbers:
> "
> InnoDB: Your database may be corrupt or you may have copied the InnoDB
> tablespace but not the InnoDB log files.
> InnoDB: Page [page id: space=9, page number=3136] log sequence number
> 114156703 is in the future! Current system log sequence number 1649127.
> "
>
> DK is working fine i think (you wont see any error info, without event
> viewer), but cannot save mysql_install_db.exe and mysqld.exe. When i click
> OK and go back, still missing.
> I don't know if database is corrupt because of missing .exe files or exe
> files missing because of corrupted database.
>
> I was trying to repair databse, but cannot find a good simple tutorial
> without studing whole mysql books. All say, use the innodb_force_recovery =
> 1,2,3,4,5,6, but where, how, which ini file, i tried all of them, nothing
> changes, maybe digikam is a bit different, they use different config files,
> but i cannot find them on my HDD.
>
> Maybe i'm doing something wrong, or everything, but is there any one/two
> click solution to fix those errors and repair database? If its just changing
> some files in confing files, why not make a button in digikam "Reapair
> problems" or sth like that, aby selfheal? ;)
> I could try to find a fix, probably can fix it by myself with google (i know
> some IT) but lots of users, photographers have no idea how to open .ini
> file, not to mention understanding windows event logs etc.
>
> I've installed digikam on so many computers (friends, familly...) and they
> will probably lose everything one day with 1 error, without IT support
> (months of tagging pictures).
>
> BTW. I got another digikam on linux, and its ok.
>
> Windows 10 64x
> Digikam 6.0.0 beta2 64x
> Mariadb 10.3.10
> Mysql (experimantal)
> Face recognition (experimental)
> My ram and photo hdd partition without bad sectors.
>
> Thanks!
>
>
>
> --
> Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[papertape]

At the risk of being tiresome...
Backup.  Backup!  BACKUP!!!

The database is probably small compared to your images, so you can back it
up real cheap.

If your image files average 20 MB each, 170,000 files occupy 3.4 TB.
Get yourself a 4 TB external disk for $110.  If that's too small, get an 8
TB for $160.  Get 2.
$320 is $0.0019 per image.
Is your work worth that?




--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

Le 11/12/2018 à 11:04, zwyklydigikam a écrit :
> [hidden email] wrote
>> didn't you make a backup before using experimental thing?
>>
>> do you have data also saved on image or sidecar files?
>
> I use experimental from start, generating default SQL db takes about 2 days
> with 200k images, and everything is slow. MySQL is the only choice.

I use personally internal database and write to image with approx 60.000
images. I never had corrupted database and have a loose database save
policy, the only thing I have really in the database is tags and they
are in image files anyway. I *sometime* have a faulty jpg (it shows with
part of the image grey) and I rebuild it from source (I always keep
original images in a separate folder, never touched)

>
> I never save on image.

may be you should or use sidecar file

>
> I have old backups but how can i be sure if it IS a good backup?

that's all of your own. make as many backup you feel necessary (for
example just before a huge work). Did you aver experiment a disk crash?
this I did, more than once.

  Lets say
> someone is making backup every month, database is 4gb. Now your DK crashed,
> and you use your backup, but your backup is damaged,

why should it be? if so take the previous one...

> Going back to my problem, how to repair mysql database without backup? I'm

may be begin to dump the database with mysqlbackup, so you will have a
text (sql) file to look at?

jdd


--
http://dodin.org

[zwyklydigikam]

Thank you for all the answers.
I will think about sidecar files, looks nice, but now i want to know what is
going on with my internal mysql.

New problem, or maybe just new information about my problem.

I've installed 6.0.0 beta3. Didnt help, cant migrate database, some errors,
dunno how to dump file, i'm not good with databases.
So i've installed new DK beta3, mariadb 10.3.10 and started creating fresh
new database from begining, all fresh.
Started finding new elements in my 170k images folder about 16:00 i think
(first start). At 23:30 it was 70%, so i left over night.
Now it's done (morning right now) but in windows log i've found 3 errors:

Aborted connection 15 to db: 'digikam' user: 'root' host: '' (Got timeout
reading communication packets)
Aborted connection 16 to db: 'digikam' user: 'root' host: '' (Got timeout
reading communication packets)
Aborted connection 11 to db: 'digikam' user: 'root' host: '' (Got timeout
reading communication packets)

I tought, maybe it just lost connection afer creating, maybe it'snothing
bad, but when i closed digikam, and started again, i got hundreds of errors
from my first post again:

"
InnoDB: Page [page id: space=0, page number=7] log sequence number 914615999
is in the future! Current system log sequence number 1631253.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
tablespace but not the InnoDB log files. Please refer to
https://mariadb.com/kb/en/library/xtradbinnodb-recovery-modes/ for
information about forcing recovery.
"
Whats wrong? Is it digikam? mariadb? Config files? I can work in DG as
usual, just lots of errors in windows event viewer like before, and cannot
pick mysql.exe files in configuration (it doesnt save like before).

And BTW one more different question.
Why my CPU uses 5-20% max when creating database or thumbnails? My HDD is
25%-30% used. Do i have to set up enything to use my full hardware power?
With SQLite uses max 5% cpu, and 15% HDD.
CPU G4600 12g RAM 150mb HDD W10.64

Thank you!



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html