lots of photos and startup time

IMHO the 2 seconds mor or less with splash enabled is not to much to bother about.
I think the real time savings can be made in read and write performance to the db.
All gains in performance is welcome but in my case I want a reduction in minutes before bothering with seconds.
 
On my Ubuntu Studio 9.04 system I have now installed their kernel 2.6.31 rc2 and start time is reduced considerably. From close to one houer to mere five minutes. Flickering has reapperd but settles fast.
 
How this is done I do not know, mabye new algoritm in kernel?
/D.L.
 

> Date: Tue, 7 Jul 2009 20:23:55 +0200
> From: [hidden email]
> To: [hidden email]
> Subject: Re: [Digikam-users] lots of photos and startup time
>
> 2009/7/7 Andi Clemens <[hidden email]>:
> > Ok, it really is a little bit slower WITH the splash enabled then without it.
> > Approximately 2 seconds... weird.
>
> ok. i have also see a similar delay.
>
> This can be explained by the way to redraw full pixmap of splashscreen
> instead only the space where text is displayed.
>
> Gilles
>
> >
> > Andi
> >
> > On Tuesday 07 July 2009 20:12:01 Andi Clemens wrote:
> >> Oh I have not "scanning" enabled. But I will do so now...
> >>
> >> Andi
> >>
> >> On Tuesday 07 July 2009 20:00:21 Gilles Caulier wrote:
> >> > 2009/7/7 Andi Clemens <[hidden email]>:
> >> > > Hmm Gilles I don't have a progress bar when I disable the splashscreen.
> >> > > Am I misunderstanding you here?
> >> > > But it is not faster or slower, now that the splash is disabled.
> >> >
> >> > Here i have progress dialog, if scan at startup is enable.
> >> >
> >> > In theory, this must be the same time, but mechanisms in code to post
> >> > progress is a little bit different.
> >> >
> >> > Gilles
> >> >
> >> > > Andi
> >> > >
> >> > > On Tuesday 07 July 2009 19:55:42 Andi Clemens wrote:
> >> > >> Hmm there is something different then the splashcreen? Never seen
> >> > >> that! ;-)
> >> > >> Ok I will check. But again for me startup time is quite normal, I
> >> > >> don't think 7 seconds is too much for so many images (and remember,
> >> > >> all libs need to be loaded on first run, too).
> >> > >> But over 1 minute and more is strange and has been reported a lot for
> >> > >> reiserFS partitions.
> >> > >>
> >> > >> I will create a reiserFS partition later on and test it.
> >> > >>
> >> > >> Andi
> >> > >>
> >> > >> On Tuesday 07 July 2009 19:48:22 Gilles Caulier wrote:
> >> > >> > Andi,
> >> > >> >
> >> > >> > just a stupid question : do you run splashscreen at startup or
> >> > >> > progress dialog ? Can you switch to check the difference ?
> >> > >> >
> >> > >> > Gilles Caulier
> >> > >> >
> >> > >> > 2009/7/7 Bruce Marshall <[hidden email]>:
> >> > >> > > On Tuesday 07 July 2009, Andi Clemens wrote:
> >> > >> > >> Hmm seems to be a reiserFS problem, since every one reporting
> >> > >> > >> this is using reiserFS, also the older reports are talking about
> >> > >> > >> reiser.
> >> > >> > >
> >> > >> > > Takes me 1:20  on an EXT3 disk running XFCE  and Digikam  0.10.0
> >> > >> > >
> >> > >> > > One problem I have is that all of my pictures are in two albums,
> >> > >> > > but they are the same album.   In otherwords, two albums pointing
> >> > >> > > to the same directory. So it may be doing twice the work when it
> >> > >> > > wouldn't have to.
> >> > >> > >
> >> > >> > > I haven't yet figured out a way to get rid of the duplicate since
> >> > >> > > eliminating one album would delete all of the pictures in it!!
> >> > >> > > Don't know why Digikam was designed to operate that way.  You
> >> > >> > > should be able to remove an album from Digikam  without having to
> >> > >> > > delete anything.
> >> > >> > >
> >> > >> > > _______________________________________________
> >> > >> > > Digikam-users mailing list
> >> > >> > > [hidden email]
> >> > >> > > https://mail.kde.org/mailman/listinfo/digikam-users

Snip

---

Dela foton på ett smidigt sätt med Windows LiveT Photos. Dra och släpp
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

On Wednesday 08 July 2009 10:22:16 Daniel Larsson wrote:

> IMHO the 2 seconds mor or less with splash enabled is not to much to bother
> about.
>
> I think the real time savings can be made in read and write performance to
> the db.
>
> All gains in performance is welcome but in my case I want a reduction in
> minutes before bothering with seconds.
>
>
>
> On my Ubuntu Studio 9.04 system I have now installed their kernel 2.6.31
> rc2 and start time is reduced considerably. From close to one houer to mere
> five minutes. Flickering has reapperd but settles fast.

One hour? Seriously?
Then your system is broken, this is really not normal.
Even with reiserFS I had never such bad startup times.
Not even 5 Minutes.
DB access has been improved already and this is not what slows down the
startup so much, we don't query a lot during the application starts.

I really would consider to try another filesystem, or run a check (with
badblocks if possible), maybe the partition you use is about to die?

Andi

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Yes, I am serious. I do not think that the partition is bad or the  HDD is about to die, non of the drives I use are older than 2 years. FS is EXT4. Maby you are right about the broken system but I only notice it in digiKam. Maby I have to reinstall the system but it wount be this week as I am suposed to give two lectures on digiKam this week.
 

> From: [hidden email]
> To: [hidden email]
> Date: Wed, 8 Jul 2009 10:43:14 +0200
> Subject: Re: [Digikam-users] lots of photos and startup time
>
>
> On Wednesday 08 July 2009 10:22:16 Daniel Larsson wrote:
> > IMHO the 2 seconds mor or less with splash enabled is not to much to bother
> > about.
> >
> > I think the real time savings can be made in read and write performance to
> > the db.
> >
> > All gains in performance is welcome but in my case I want a reduction in
> > minutes before bothering with seconds.
> >
> >
> >
> > On my Ubuntu Studio 9.04 system I have now installed their kernel 2.6.31
> > rc2 and start time is reduced considerably. From close to one houer to mere
> > five minutes. Flickering has reapperd but settles fast.
>
> One hour? Seriously?
> Then your system is broken, this is really not normal.
> Even with reiserFS I had never such bad startup times.
> Not even 5 Minutes.
> DB access has been improved already and this is not what slows down the
> startup so much, we don't query a lot during the application starts.
>
> I really would consider to try another filesystem, or run a check (with
> badblocks if possible), maybe the partition you use is about to die?
>
> Andi
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

---

Hitta kärleken lagom till sommaren! Klicka här MSN Dejting
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Jacek Poplawski-2]

On Wed, Jul 8, 2009 at 10:43 AM, Andi Clemens <[hidden email]> wrote:

Then your system is broken, this is really not normal.
Even with reiserFS I had never such bad startup times.
Not even 5 Minutes.

So you assume that my system is broken too?
 

DB access has been improved already and this is not what slows down the
startup so much, we don't query a lot during the application starts.

I also see database loading on splash screen during all these minutes of loading.

Are there any debug logs I could enable to give you some more details?
Or could you tell me where in code I could measure time, so I could show you values (I am a programmer)?
 

--
Free Software - find interesting programs and change them
NetHack - meet interesting creatures, kill them and eat their bodies
Usenet - meet interesting people from all over the world and flame them
Decopter - unrealistic helicopter simulator, get it from http://decopter.sf.net


_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Ok one thing that causes bad performance with ext4 is the use of barriers.
If you are running ext4 with barriers turned on, try to add the following
mount option to the fstab:

barrier=0

sqlite performs bad with barriers, because it creates a lot of journal files
in a short time, with barriers enabled it can kill the performance.
I usually only had issues when moving files inside of digiKam, not on startup.
But maybe this helps.

Andi

On Wednesday 08 July 2009 12:43:42 Daniel Larsson wrote:

> Yes, I am serious. I do not think that the partition is bad or the  HDD is
> about to die, non of the drives I use are older than 2 years. FS is EXT4.
> Maby you are right about the broken system but I only notice it in digiKam.
> Maby I have to reinstall the system but it wount be this week as I am
> suposed to give two lectures on digiKam this week.
>
> > From: [hidden email]
> > To: [hidden email]
> > Date: Wed, 8 Jul 2009 10:43:14 +0200
> > Subject: Re: [Digikam-users] lots of photos and startup time
> >
> > On Wednesday 08 July 2009 10:22:16 Daniel Larsson wrote:
> > > IMHO the 2 seconds mor or less with splash enabled is not to much to
> > > bother about.
> > >
> > > I think the real time savings can be made in read and write performance
> > > to the db.
> > >
> > > All gains in performance is welcome but in my case I want a reduction
> > > in minutes before bothering with seconds.
> > >
> > >
> > >
> > > On my Ubuntu Studio 9.04 system I have now installed their kernel
> > > 2.6.31 rc2 and start time is reduced considerably. From close to one
> > > houer to mere five minutes. Flickering has reapperd but settles fast.
> >
> > One hour? Seriously?
> > Then your system is broken, this is really not normal.
> > Even with reiserFS I had never such bad startup times.
> > Not even 5 Minutes.
> > DB access has been improved already and this is not what slows down the
> > startup so much, we don't query a lot during the application starts.
> >
> > I really would consider to try another filesystem, or run a check (with
> > badblocks if possible), maybe the partition you use is about to die?
> >
> > Andi
> >
> > _______________________________________________
> > Digikam-users mailing list
> > [hidden email]
> > https://mail.kde.org/mailman/listinfo/digikam-users
>
> _________________________________________________________________
> Vi vet vem du passar ihop med! Klicka här för att få veta!
> http://dejting.se.msn.com/channel/index.aspx?trackingid=1002952

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

I'm not assuming all your systems are broken ;-)
But if someone is telling me digiKam needs an hour to startup, then something
must be really messed up.

There is nothing to improve directly, it is all related to sqlite.
It might be that sqlite doesn't like reiserFS that much, especially when the
database is bigger.

This is one of the reasons we might think of a different database backend,
like amarok did.
sqlite is really not so good for performance relevant operations.

Andi

On Wednesday 08 July 2009 12:53:37 Jacek Poplawski wrote:

> On Wed, Jul 8, 2009 at 10:43 AM, Andi Clemens <[hidden email]> wrote:
> > Then your system is broken, this is really not normal.
> > Even with reiserFS I had never such bad startup times.
> > Not even 5 Minutes.
>
> So you assume that my system is broken too?
>
> > DB access has been improved already and this is not what slows down the
> > startup so much, we don't query a lot during the application starts.
>
> I also see database loading on splash screen during all these minutes of
> loading.
>
> Are there any debug logs I could enable to give you some more details?
> Or could you tell me where in code I could measure time, so I could show
> you values (I am a programmer)?

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Marcel Wiesweg]

> > DB access has been improved already and this is not what slows down the
> > startup so much, we don't query a lot during the application starts.
>
> I also see database loading on splash screen during all these minutes of
> loading.

Dont rely on these splashscreen messages, they are not very precise.
The text doesnt mean we are reading from the database all the time.

>
> Are there any debug logs I could enable to give you some more details?
> Or could you tell me where in code I could measure time, so I could show
> you values (I am a programmer)?

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

And another guess:

Maybe your database file is somehow corrupted?
Can you try the following?

1. in /usr/bin, there should be a script called cleanup_digikamdb, run it
2. if you have not such a script, open your database file with the sqlite CLI:
    $ sqlite3 /path/to/images/digikam4.db

    and then run
    VACUUM;

    inside the sqlite3 shell.

digiKam should be closed, before doing this.

Andi

On Wednesday 08 July 2009 12:57:50 Andi Clemens wrote:

> Ok one thing that causes bad performance with ext4 is the use of barriers.
> If you are running ext4 with barriers turned on, try to add the following
> mount option to the fstab:
>
> barrier=0
>
> sqlite performs bad with barriers, because it creates a lot of journal
> files in a short time, with barriers enabled it can kill the performance. I
> usually only had issues when moving files inside of digiKam, not on
> startup. But maybe this helps.
>
> Andi
>
> On Wednesday 08 July 2009 12:43:42 Daniel Larsson wrote:
> > Yes, I am serious. I do not think that the partition is bad or the  HDD
> > is about to die, non of the drives I use are older than 2 years. FS is
> > EXT4. Maby you are right about the broken system but I only notice it in
> > digiKam. Maby I have to reinstall the system but it wount be this week as
> > I am suposed to give two lectures on digiKam this week.
> >
> > > From: [hidden email]
> > > To: [hidden email]
> > > Date: Wed, 8 Jul 2009 10:43:14 +0200
> > > Subject: Re: [Digikam-users] lots of photos and startup time
> > >
> > > On Wednesday 08 July 2009 10:22:16 Daniel Larsson wrote:
> > > > IMHO the 2 seconds mor or less with splash enabled is not to much to
> > > > bother about.
> > > >
> > > > I think the real time savings can be made in read and write
> > > > performance to the db.
> > > >
> > > > All gains in performance is welcome but in my case I want a reduction
> > > > in minutes before bothering with seconds.
> > > >
> > > >
> > > >
> > > > On my Ubuntu Studio 9.04 system I have now installed their kernel
> > > > 2.6.31 rc2 and start time is reduced considerably. From close to one
> > > > houer to mere five minutes. Flickering has reapperd but settles fast.
> > >
> > > One hour? Seriously?
> > > Then your system is broken, this is really not normal.
> > > Even with reiserFS I had never such bad startup times.
> > > Not even 5 Minutes.
> > > DB access has been improved already and this is not what slows down the
> > > startup so much, we don't query a lot during the application starts.
> > >
> > > I really would consider to try another filesystem, or run a check (with
> > > badblocks if possible), maybe the partition you use is about to die?
> > >
> > > Andi
> > >
> > > _______________________________________________
> > > Digikam-users mailing list
> > > [hidden email]
> > > https://mail.kde.org/mailman/listinfo/digikam-users
> >
> > _________________________________________________________________
> > Vi vet vem du passar ihop med! Klicka här för att få veta!
> > http://dejting.se.msn.com/channel/index.aspx?trackingid=1002952
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Jacek Poplawski-2]

On Wed, Jul 8, 2009 at 2:39 PM, Andi Clemens <[hidden email]> wrote:

And another guess:

Maybe your database file is somehow corrupted?

No, I just erased the digikam directory (or use another user) and there was no difference.
When I add big collection it starts slowly each time.

--
Free Software - find interesting programs and change them
NetHack - meet interesting creatures, kill them and eat their bodies
Usenet - meet interesting people from all over the world and flame them
Decopter - unrealistic helicopter simulator, get it from http://decopter.sf.net


_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

I have run VACUUM on the database and put barrier=0 in fstab.
It is still extremely slow on start with this 687 MB large db but start in seconds when i start with a 2.1 MB large db with my pictures on the internal disc. To me this sugest that either digiKam is bad att handeling large databases or my databas is still corupt. I will rebuild the entire database to se if this helps.
 
/D.L.
 

> From: [hidden email]
> To: [hidden email]
> Date: Wed, 8 Jul 2009 14:39:49 +0200
> Subject: Re: [Digikam-users] lots of photos and startup time
>
> And another guess:
>
> Maybe your database file is somehow corrupted?
> Can you try the following?
>
> 1. in /usr/bin, there should be a script called cleanup_digikamdb, run it
> 2. if you have not such a script, open your database file with the sqlite CLI:
> $ sqlite3 /path/to/images/digikam4.db
>
> and then run
> VACUUM;
>
> inside the sqlite3 shell.
>
> digiKam should be closed, before doing this.
>
> Andi
>
> On Wednesday 08 July 2009 12:57:50 Andi Clemens wrote:
> > Ok one thing that causes bad performance with ext4 is the use of barriers.
> > If you are running ext4 with barriers turned on, try to add the following
> > mount option to the fstab:
> >
> > barrier=0
> >
> > sqlite performs bad with barriers, because it creates a lot of journal
> > files in a short time, with barriers enabled it can kill the performance. I
> > usually only had issues when moving files inside of digiKam, not on
> > startup. But maybe this helps.
> >
> > Andi
> >
> > On Wednesday 08 July 2009 12:43:42 Daniel Larsson wrote:
> > > Yes, I am serious. I do not think that the partition is bad or the HDD
> > > is about to die, non of the drives I use are older than 2 years. FS is
> > > EXT4. Maby you are right about the broken system but I only notice it in
> > > digiKam. Maby I have to reinstall the system but it wount be this week as
> > > I am suposed to give two lectures on digiKam this week.
> > >
> > > > From: [hidden email]
> > > > To: [hidden email]
> > > > Date: Wed, 8 Jul 2009 10:43:14 +0200
> > > > Subject: Re: [Digikam-users] lots of photos and startup time
> > > >
> > > > On Wednesday 08 July 2009 10:22:16 Daniel Larsson wrote:
> > > > > IMHO the 2 seconds mor or less with splash enabled is not to much to
> > > > > bother about.
> > > > >
> > > > > I think the real time savings can be made in read and write
> > > > > performance to the db.
> > > > >
> > > > > All gains in performance is welcome but in my case I want a reduction
> > > > > in minutes before bothering with seconds.
> > > > >
> > > > >
> > > > >
> > > > > On my Ubuntu Studio 9.04 system I have now installed their kernel
> > > > > 2.6.31 rc2 and start time is reduced considerably. From close to one
> > > > > houer to mere five minutes. Flickering has reapperd but settles fast.
> > > >
> > > > One hour? Seriously?
> > > > Then your system is broken, this is really not normal.
> > > > Even with reiserFS I had never such bad startup times.
> > > > Not even 5 Minutes.
> > > > DB access has been improved already and this is not what slows down the
> > > > startup so much, we don't query a lot during the application starts.
> > > >
> > > > I really would consider to try another filesystem, or run a check (with
> > > > badblocks if possible), maybe the partition you use is about to die?
> > > >
> > > > Andi
> > > >
> > > > _______________________________________________
> > > > Digikam-users mailing list
> > > > [hidden email]
> > > > https://mail.kde.org/mailman/listinfo/digikam-users
> > >
> > > _________________________________________________________________
> > > Vi vet vem du passar ihop med! Klicka här för att få veta!
> > > http://dejting.se.msn.com/channel/index.aspx?trackingid=1002952
> >
> > _______________________________________________
> > Digikam-users mailing list
> > [hidden email]
> > https://mail.kde.org/mailman/listinfo/digikam-users
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

---

Dela foton på ett smidigt sätt med Windows LiveT Photos. Dra och släpp
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

Marcel, Andi,

All this thread tell me that we need to found urgently a solution to
this problem.

If sqlite cannot drive large database, another DB backend need to be
used. For me MySql sound like the right alternative, if performance
are there of course (how to make a quick test about ?)

About DB schema and all DB queries, it's really complicated to migrate
from SQLite to MySql. SQL syntax is really different between both DB
backend ?

Gilles

2009/7/9 Daniel Larsson <[hidden email]>:

> I have run VACUUM on the database and put barrier=0 in fstab.
> It is still extremely slow on start with this 687 MB large db but start in
> seconds when i start with a 2.1 MB large db with my pictures on the internal
> disc. To me this sugest that either digiKam is bad att handeling large
> databases or my databas is still corupt. I will rebuild the entire database
> to se if this helps.
>
> /D.L.
>
>> From: [hidden email]
>> To: [hidden email]
>> Date: Wed, 8 Jul 2009 14:39:49 +0200
>> Subject: Re: [Digikam-users] lots of photos and startup time
>>
>> And another guess:
>>
>> Maybe your database file is somehow corrupted?
>> Can you try the following?
>>
>> 1. in /usr/bin, there should be a script called cleanup_digikamdb, run it
>> 2. if you have not such a script, open your database file with the sqlite
>> CLI:
>> $ sqlite3 /path/to/images/digikam4.db
>>
>> and then run
>> VACUUM;
>>
>> inside the sqlite3 shell.
>>
>> digiKam should be closed, before doing this.
>>
>> Andi
>>
>> On Wednesday 08 July 2009 12:57:50 Andi Clemens wrote:
>> > Ok one thing that causes bad performance with ext4 is the use of
>> > barriers.
>> > If you are running ext4 with barriers turned on, try to add the
>> > following
>> > mount option to the fstab:
>> >
>> > barrier=0
>> >
>> > sqlite performs bad with barriers, because it creates a lot of journal
>> > files in a short time, with barriers enabled it can kill the
>> > performance. I
>> > usually only had issues when moving files inside of digiKam, not on
>> > startup. But maybe this helps.
>> >
>> > Andi
>> >
>> > On Wednesday 08 July 2009 12:43:42 Daniel Larsson wrote:
>> > > Yes, I am serious. I do not think that the partition is bad or the HDD
>> > > is about to die, non of the drives I use are older than 2 years. FS is
>> > > EXT4. Maby you are right about the broken system but I only notice it
>> > > in
>> > > digiKam. Maby I have to reinstall the system but it wount be this week
>> > > as
>> > > I am suposed to give two lectures on digiKam this week.
>> > >
>> > > > From: [hidden email]
>> > > > To: [hidden email]
>> > > > Date: Wed, 8 Jul 2009 10:43:14 +0200
>> > > > Subject: Re: [Digikam-users] lots of photos and startup time
>> > > >
>> > > > On Wednesday 08 July 2009 10:22:16 Daniel Larsson wrote:
>> > > > > IMHO the 2 seconds mor or less with splash enabled is not to much
>> > > > > to
>> > > > > bother about.
>> > > > >
>> > > > > I think the real time savings can be made in read and write
>> > > > > performance to the db.
>> > > > >
>> > > > > All gains in performance is welcome but in my case I want a
>> > > > > reduction
>> > > > > in minutes before bothering with seconds.
>> > > > >
>> > > > >
>> > > > >
>> > > > > On my Ubuntu Studio 9.04 system I have now installed their kernel
>> > > > > 2.6.31 rc2 and start time is reduced considerably. From close to
>> > > > > one
>> > > > > houer to mere five minutes. Flickering has reapperd but settles
>> > > > > fast.
>> > > >
>> > > > One hour? Seriously?
>> > > > Then your system is broken, this is really not normal.
>> > > > Even with reiserFS I had never such bad startup times.
>> > > > Not even 5 Minutes.
>> > > > DB access has been improved already and this is not what slows down
>> > > > the
>> > > > startup so much, we don't query a lot during the application starts.
>> > > >
>> > > > I really would consider to try another filesystem, or run a check
>> > > > (with
>> > > > badblocks if possible), maybe the partition you use is about to die?
>> > > >
>> > > > Andi
>> > > >
>> > > > _______________________________________________
>> > > > Digikam-users mailing list
>> > > > [hidden email]
>> > > > https://mail.kde.org/mailman/listinfo/digikam-users
>> > >
>> > > _________________________________________________________________
>> > > Vi vet vem du passar ihop med! Klicka här för att få veta!
>> > > http://dejting.se.msn.com/channel/index.aspx?trackingid=1002952
>> >
>> > _______________________________________________
>> > Digikam-users mailing list
>> > [hidden email]
>> > https://mail.kde.org/mailman/listinfo/digikam-users
>>
>> _______________________________________________
>> Digikam-users mailing list
>> [hidden email]
>> https://mail.kde.org/mailman/listinfo/digikam-users
>
> ________________________________
> Dela foton på ett smidigt sätt med Windows LiveT Photos. Dra och släpp
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Daniel Bauer-2]

Gilles,

I also followed this thread quitly (as I still use old digikam on old kde 3.5
plus reiserfes...). But I fully support your idea to change the database
system, if this is causing the delay.

Here I have several very large MySql databases and they are very fast, but
digikam is always very slow on opening for the first time after logging in as
a user (also the newer digikam I installed in kde 4 for testing, I thought  
it was kde4's fault...)

(Apart from the coding work) the only problem I see is that with MySql users
will have to install MySql too. While this is more or less one click on Linux
I don't know how it is on other systems.

Maybe there could be an option for the user: sqlite as a (slow) standard, but
working out of the box, and MySql as an option, if the user has installed it
and knows the path etc. Isn't amarok offering something similar?

regards

Daniel

On Thursday 09 July 2009 08:45:31, Gilles Caulier wrote:

> Marcel, Andi,
>
> All this thread tell me that we need to found urgently a solution to
> this problem.
>
> If sqlite cannot drive large database, another DB backend need to be
> used. For me MySql sound like the right alternative, if performance
> are there of course (how to make a quick test about ?)
>
> About DB schema and all DB queries, it's really complicated to migrate
> from SQLite to MySql. SQL syntax is really different between both DB
> backend ?
>
> Gilles

--
Daniel Bauer photographer Basel Barcelona
professional photography: http://www.daniel-bauer.com
erotic art photos: http://www.bauer-nudes.com
Madagascar special: http://www.fotograf-basel.ch/madagascar/
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Marcel Wiesweg]

> I also followed this thread quitly (as I still use old digikam on old kde
> 3.5 plus reiserfes...). But I fully support your idea to change the
> database system, if this is causing the delay.
>
> Here I have several very large MySql databases and they are very fast, but
> digikam is always very slow on opening for the first time after logging in
> as a user (also the newer digikam I installed in kde 4 for testing, I
> thought it was kde4's fault...)
>
> (Apart from the coding work) the only problem I see is that with MySql
> users will have to install MySql too. While this is more or less one click
> on Linux I don't know how it is on other systems.
>
> Maybe there could be an option for the user: sqlite as a (slow) standard,
> but working out of the box, and MySql as an option, if the user has
> installed it and knows the path etc. Isn't amarok offering something
> similar?

Amarok has dropped Sqlite support and has only one backend now, which is MySQL
embedded.
Akonadi requires MySQL (server (*)) currently, with plans to support
Postgresql as well, but not sqlite and as I understood, not Mysql embedded
(**).
This means if you want to use Amarok, you will have MySQL installed (embedded
is probably in the same package as the full server for most distros, but need
not).
If you want to use KMail in KDE4.5 or 4.6, you will have MySQL installed.

MySQL will in real life become a dependency for KDE applications (not
libraries or runtime) within the next twelve months.

There are voices asking for a "desktop database" as more and more applications
require a db; but there is no driving force for that as far as I see.

Marcel

(*) MySQL embedded's full name is "MySQL Embedded Server" but for short
distinction I speak of embedded vs. server
(**) They needed to optimize for high throughput, and found only dbs >= MySql
server sufficient for their needs. Not sure about the results for MySQL
embedded
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Marcel Wiesweg]

> Marcel, Andi,
>
> All this thread tell me that we need to found urgently a solution to
> this problem.
>
> If sqlite cannot drive large database, another DB backend need to be
> used. For me MySql sound like the right alternative, if performance
> are there of course (how to make a quick test about ?)
>
> About DB schema and all DB queries, it's really complicated to migrate
> from SQLite to MySql. SQL syntax is really different between both DB
> backend ?

I am not sure how much difference is there.
I would expect that 90% of our statements will work for both, as most are
simple.
We would need to test the schema definition, think about using the correct
character set everywhere - SQLite is fully UTF-8 so this always just worked,
MySQL may need some explicit options for UTF-8, and test with complex queries
from the advanced search.

>
> Gilles
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Amarok is using embedded MySQL, but which also assumes you have installed
mysql (but you need no server running in the background).

Gilles,
my thumbsDB (ok this is not the normal digikam DB) is 900MB big, but it opens
quite fast. So I don't know if this is really the issue here.

Daniel,
how many items  / images do you have in your database?
In my current test collection, I have 200.000, but my database is only 180MB
big.

Maybe yours is really corrupted (or has a lot of old entries, that you might
need to delete manually).

If you have less images, you could try to remove all entries from the table
"images" that have a status of 3 (deleted).

Andi

On Thursday 09 July 2009 09:37:24 Daniel Bauer wrote:

> Gilles,
>
> I also followed this thread quitly (as I still use old digikam on old kde
> 3.5 plus reiserfes...). But I fully support your idea to change the
> database system, if this is causing the delay.
>
> Here I have several very large MySql databases and they are very fast, but
> digikam is always very slow on opening for the first time after logging in
> as a user (also the newer digikam I installed in kde 4 for testing, I
> thought it was kde4's fault...)
>
> (Apart from the coding work) the only problem I see is that with MySql
> users will have to install MySql too. While this is more or less one click
> on Linux I don't know how it is on other systems.
>
> Maybe there could be an option for the user: sqlite as a (slow) standard,
> but working out of the box, and MySql as an option, if the user has
> installed it and knows the path etc. Isn't amarok offering something
> similar?
>
> regards
>
> Daniel
>
> On Thursday 09 July 2009 08:45:31, Gilles Caulier wrote:
> > Marcel, Andi,
> >
> > All this thread tell me that we need to found urgently a solution to
> > this problem.
> >
> > If sqlite cannot drive large database, another DB backend need to be
> > used. For me MySql sound like the right alternative, if performance
> > are there of course (how to make a quick test about ?)
> >
> > About DB schema and all DB queries, it's really complicated to migrate
> > from SQLite to MySql. SQL syntax is really different between both DB
> > backend ?
> >
> > Gilles

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Until now I have not seen any query that might not work.
I guess the other way around would be more difficult (MySQL->sqlite).

Startup times (especially with a MySQL server) will be faster for sure, since
the server already loads the database handles into memory).

Access will be faster, too, but unfortunately memory consumption will be
increased as well.
Right now digiKam uses 160 MB RAM on startup on my system, during a digiKam
session I mostly end up with 270MB RAM.
With MySQL the two databases will take much more memory, and especially with
the thumbsDB this can be a problem again.
MySQL caches a lot into memory, that's why it is (also) faster.

I will try to create the two databases in mysql today, import the data and see
how much memory the server will use initially.
Then I will query via a script some basic operations that I would do inside of
digiKam.

If memory usage goes high, we could have another problem again :-)
The "problem" with our two databases is that we use a lot of binary data (in
forms of blobs) in there.
When they are cached in memory, I don't know what will happen.

Andi

On Thursday 09 July 2009 10:30:35 Marcel Wiesweg wrote:

> > Marcel, Andi,
> >
> > All this thread tell me that we need to found urgently a solution to
> > this problem.
> >
> > If sqlite cannot drive large database, another DB backend need to be
> > used. For me MySql sound like the right alternative, if performance
> > are there of course (how to make a quick test about ?)
> >
> > About DB schema and all DB queries, it's really complicated to migrate
> > from SQLite to MySql. SQL syntax is really different between both DB
> > backend ?
>
> I am not sure how much difference is there.
> I would expect that 90% of our statements will work for both, as most are
> simple.
> We would need to test the schema definition, think about using the correct
> character set everywhere - SQLite is fully UTF-8 so this always just
> worked, MySQL may need some explicit options for UTF-8, and test with
> complex queries from the advanced search.
>
> > Gilles
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

Marcel,

Important question : MySQL API are thread safe ?

If yes, we can said good bye to all digiKam kioslave in the future :
more simple to debug, more simple to install (look all report about
kioslave install problems), and certainly faster.

Gilles Caulier



2009/7/9 Marcel Wiesweg <[hidden email]>:

>
>> Marcel, Andi,
>>
>> All this thread tell me that we need to found urgently a solution to
>> this problem.
>>
>> If sqlite cannot drive large database, another DB backend need to be
>> used. For me MySql sound like the right alternative, if performance
>> are there of course (how to make a quick test about ?)
>>
>> About DB schema and all DB queries, it's really complicated to migrate
>> from SQLite to MySql. SQL syntax is really different between both DB
>> backend ?
>
> I am not sure how much difference is there.
> I would expect that 90% of our statements will work for both, as most are
> simple.
> We would need to test the schema definition, think about using the correct
> character set everywhere - SQLite is fully UTF-8 so this always just worked,
> MySQL may need some explicit options for UTF-8, and test with complex queries
> from the advanced search.
>
>>
>> Gilles
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Andi, I hope you adress me with Daniel, I think there are two Daniel in this tread. Appoligies if mistaken.
I have 260 000 images in my collection. As long as the collection is only indexed performance is OK. It is after working with the collection, moving files, renaming and it realy degrades when I use the find duplicates feature. I have a lot of duplicates as I have used the option of capturing in both JPG and CR2.
/D.L.
 

> From: [hidden email]
> To: [hidden email]
> Date: Thu, 9 Jul 2009 10:37:13 +0200
> Subject: Re: [Digikam-users] lots of photos and startup time
>
> Amarok is using embedded MySQL, but which also assumes you have installed
> mysql (but you need no server running in the background).
>
> Gilles,
> my thumbsDB (ok this is not the normal digikam DB) is 900MB big, but it opens
> quite fast. So I don't know if this is really the issue here.
>
> Daniel,
> how many items / images do you have in your database?
> In my current test collection, I have 200.000, but my database is only 180MB
> big.
>
> Maybe yours is really corrupted (or has a lot of old entries, that you might
> need to delete manually).
>
> If you have less images, you could try to remove all entries from the table
> "images" that have a status of 3 (deleted).
>
> Andi
>
> On Thursday 09 July 2009 09:37:24 Daniel Bauer wrote:
> > Gilles,
> >
> > I also followed this thread quitly (as I still use old digikam on old kde
> > 3.5 plus reiserfes...). But I fully support your idea to change the
> > database system, if this is causing the delay.
> >
> > Here I have several very large MySql databases and they are very fast, but
> > digikam is always very slow on opening for the first time after logging in
> > as a user (also the newer digikam I installed in kde 4 for testing, I
> > thought it was kde4's fault...)
> >
> > (Apart from the coding work) the only problem I see is that with MySql
> > users will have to install MySql too. While this is more or less one click
> > on Linux I don't know how it is on other systems.
> >
> > Maybe there could be an option for the user: sqlite as a (slow) standard,
> > but working out of the box, and MySql as an option, if the user has
> > installed it and knows the path etc. Isn't amarok offering something
> > similar?
> >
> > regards
> >
> > Daniel
> >
> > On Thursday 09 July 2009 08:45:31, Gilles Caulier wrote:
> > > Marcel, Andi,
> > >
> > > All this thread tell me that we need to found urgently a solution to
> > > this problem.
> > >
> > > If sqlite cannot drive large database, another DB backend need to be
> > > used. For me MySql sound like the right alternative, if performance
> > > are there of course (how to make a quick test about ?)
> > >
> > > About DB schema and all DB queries, it's really complicated to migrate
> > > from SQLite to MySql. SQL syntax is really different between both DB
> > > backend ?
> > >
> > > Gilles
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

---

Gör personlighetstestet på MSN Dejting, se vem du passar ihop med! MSN Dejting
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Daniel Bauer-2]

On Thursday 09 July 2009 10:37:13, Andi Clemens wrote:

> ...
>
> Daniel,
> how many items  / images do you have in your database?
> In my current test collection, I have 200.000, but my database is only
> 180MB big.
>
> Maybe yours is really corrupted (or has a lot of old entries, that you
> might need to delete manually).
>
> If you have less images, you could try to remove all entries from the table
> "images" that have a status of 3 (deleted).

Hi Andi,

As I cleaned up my laptop last week there are now only the actual 9086 image
files in digikam. The db file is 1.6 MB. But please keep in mind that I still
use digikam 0.9.3.

However, after a reboot, starting digikam takes 66 seconds until the splash
screen appears and then another 10 seconds to appear the album.

At home (with many, many more images) it takes much longer, so that I start it
and then go and have a coffee. This applies only to the *first* start of
digikam, later starts are much faster.

I also have a kde4-digikam 0.10.0.The splash screen appeard immediately - and
styed there for 28 Minutes untill  I killed the process, renamed digikam4.db
and restartet 0.10.0.. It shows me that it imports/updates the database...
After 21 minutes it's is finally here.... One reason I decided to stay with
the old version....

The new digikam4.db is 4.5 MB.

in the 0.9.3 images table there is no field called "status"...

Daniel
--
Daniel Bauer photographer Basel Barcelona
professional photography: http://www.daniel-bauer.com
erotic art photos: http://www.bauer-nudes.com
Madagascar special: http://www.fotograf-basel.ch/madagascar/
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

2009/7/9 Daniel Bauer <[hidden email]>:

> On Thursday 09 July 2009 10:37:13, Andi Clemens wrote:
>> ...
>>
>> Daniel,
>> how many items  / images do you have in your database?
>> In my current test collection, I have 200.000, but my database is only
>> 180MB big.
>>
>> Maybe yours is really corrupted (or has a lot of old entries, that you
>> might need to delete manually).
>>
>> If you have less images, you could try to remove all entries from the table
>> "images" that have a status of 3 (deleted).
>
> Hi Andi,
>
> As I cleaned up my laptop last week there are now only the actual 9086 image
> files in digikam. The db file is 1.6 MB. But please keep in mind that I still
> use digikam 0.9.3.
>
> However, after a reboot, starting digikam takes 66 seconds until the splash
> screen appears and then another 10 seconds to appear the album.
>
> At home (with many, many more images) it takes much longer, so that I start it
> and then go and have a coffee. This applies only to the *first* start of
> digikam, later starts are much faster.
>
> I also have a kde4-digikam 0.10.0.The splash screen appeard immediately - and
> styed there for 28 Minutes untill  I killed the process, renamed digikam4.db
> and restartet 0.10.0.. It shows me that it imports/updates the database...
> After 21 minutes it's is finally here.... One reason I decided to stay with
> the old version....

28 mns !!! How many images ? Which file system ?

Gilles
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users