Mysql/MariaDb database expert needs...

[eldk]

Hello,

I'm still using one old version of Digikam (3.5) with a mysql database.

I think I have tell before about this, but can't find the report :

"When one album with a lot of pictures is open, Digikam opens a lot of
concurrent connections to mysql and never close them or "forget" some of
them."

A simply way to see this is to use mytop
(http://jeremy.zawodny.com/mysql/mytop/) or mysqltuner.

In my case : up to 75 simultaneous connections (to digikam and
digikamThumb) are needed to display one album that have a little more
than 2000 pictures.

I should give logs tomorrow (I'm migrating Digikam databases to a 64 bit
server as I was still using a 32 bit one that limit usable memory for
mysql to 2 GO).

As I can't migrate now to a up to date digikam release, I think that it
is interesting to know if that still occure.

Thanks.

Eric






Le mercredi 11 novembre 2015 à 11:06 +0100, Gilles Caulier a écrit :

> Hi all,
>
>
> In my huge task of simplification of libraries puzzle, with current
> 5.0.0-beta implementation, i backported all libkface code in digiKam
> core, including database. Now digiKam 5.0.0 will not use libkface.
>
>
>
> I review all database management code, included recognition database,
> factorized all when it's possible. I tested Sqlite implementation and
> schema, and all work fine.
>
>
> Now recognition database is managed as thumbnails and core database :
> same place to store DB files, and same Sqlite engine.
>
>
> As database code is common with other DB engines, this want mean that
> recognition DB can be managed with Mysql/MariaDB too.
>
>
> So now, i started a tedious task : review all Mysql bugs, that i
> sorted this summer. The first task is to look into migration files :
>
>
> https://bugs.kde.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=ASSIGNED&bug_status=REOPENED&component=Database-Migration&list_id=1310837&product=digikam&query_format=advanced
>
>
>
> And i must admit that i'm not an expert with MySql/MariaDB. So i need
> external expertise and help here.
>
>
>
> The code from git/master is now fixed to reproduce 4.x mysql support,
> with same bugs of course.
>
>
> There are 2 solutions to use MySql/MariaDb : internal or external
> (remote) server.
>
>
> In a first step, i would to start with the first one, which will be
> more simpler to test and run.
>
>
> But before to continue, i would to know if there is somebody in this
> room who can help me in this task.
>
>
> Thanks in advance
>
>
> Best
>
>
> Gilles Caulier
> _______________________________________________
> Digikam-devel mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-devel

_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[eldk]

Hello,

Those unclosed connections seems to be a Kio-slaves consequence : number
of connections grow ... then cause aborted ones.

"As a part of code optimization, an important task was done by Mohamed
Anwer, a long-term contributor to digiKam. He worked last summer during
“Google Summer of Code” to remove all digiKam KIO-slaves used to query
the database. Instead, a robust multi-core/multi-threaded implementation
is now used. "

So forget my message, it's not a concern anymore for digikam 5.0. users.

Thanks,

Eric




Le vendredi 13 mai 2016 à 03:02 +0200, eric a écrit :

> Hello,
>
> I'm still using one old version of Digikam (3.5) with a mysql database.
>
> I think I have tell before about this, but can't find the report :
>
> "When one album with a lot of pictures is open, Digikam opens a lot of
> concurrent connections to mysql and never close them or "forget" some of
> them."
>
> A simply way to see this is to use mytop
> (http://jeremy.zawodny.com/mysql/mytop/) or mysqltuner.
>
> In my case : up to 75 simultaneous connections (to digikam and
> digikamThumb) are needed to display one album that have a little more
> than 2000 pictures.
>
> I should give logs tomorrow (I'm migrating Digikam databases to a 64 bit
> server as I was still using a 32 bit one that limit usable memory for
> mysql to 2 GO).
>
> As I can't migrate now to a up to date digikam release, I think that it
> is interesting to know if that still occure.
>
> Thanks.
>
> Eric
>
>
>
>
>
>
> Le mercredi 11 novembre 2015 à 11:06 +0100, Gilles Caulier a écrit :
> > Hi all,
> >
> >
> > In my huge task of simplification of libraries puzzle, with current
> > 5.0.0-beta implementation, i backported all libkface code in digiKam
> > core, including database. Now digiKam 5.0.0 will not use libkface.
> >
> >
> >
> > I review all database management code, included recognition database,
> > factorized all when it's possible. I tested Sqlite implementation and
> > schema, and all work fine.
> >
> >
> > Now recognition database is managed as thumbnails and core database :
> > same place to store DB files, and same Sqlite engine.
> >
> >
> > As database code is common with other DB engines, this want mean that
> > recognition DB can be managed with Mysql/MariaDB too.
> >
> >
> > So now, i started a tedious task : review all Mysql bugs, that i
> > sorted this summer. The first task is to look into migration files :
> >
> >
> > https://bugs.kde.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=ASSIGNED&bug_status=REOPENED&component=Database-Migration&list_id=1310837&product=digikam&query_format=advanced
> >
> >
> >
> > And i must admit that i'm not an expert with MySql/MariaDB. So i need
> > external expertise and help here.
> >
> >
> >
> > The code from git/master is now fixed to reproduce 4.x mysql support,
> > with same bugs of course.
> >
> >
> > There are 2 solutions to use MySql/MariaDb : internal or external
> > (remote) server.
> >
> >
> > In a first step, i would to start with the first one, which will be
> > more simpler to test and run.
> >
> >
> > But before to continue, i would to know if there is somebody in this
> > room who can help me in this task.
> >
> >
> > Thanks in advance
> >
> >
> > Best
> >
> >
> > Gilles Caulier
> > _______________________________________________
> > Digikam-devel mailing list
> > [hidden email]
> > https://mail.kde.org/mailman/listinfo/digikam-devel
>
>
> _______________________________________________
> Digikam-devel mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-devel

_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel