[Bug 155097] performance problems with recursive view and filtering

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 155097] performance problems with recursive view and filtering

Bugzilla from andi.clemens@gmx.net
http://bugs.kde.org/show_bug.cgi?id=155097





--- Comment #11 from Andi Clemens <andi clemens gmx net>  2008-09-03 16:42:46 ---
Right now I'm experimenting here... It is a bit faster right now, because I
removed some filter lines and signals. I think we filter too often and we also
use to many signals, this is slowing down.
When I'm done I will create a patch, and we can discuss what I messed up ;-)

Andi


--
Configure bugmail: http://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Bug 155097] performance problems with recursive view and filtering

Marcel Wiesweg
(sorry for posting with mail, but bko doesnt work at the moment)

I assume you are working on 0.9, but some thoughts on 0.10:
For 0.10, some caching of simple values takes place, so database queries
should be reduced in some cases. This means, if you are working on 0.10, then
there is a bug elsewhere.
Caching is easy for rating, date, format, size.
Tags are not cached by default for every image, but only on first demand, so
minor problem.
The one filter that cannot properly be solved with caching is text. Here it
would be most interesting to have a SQL solution, like one-big-query. You
cannot pass say 20000 image ids in one SQL statement. What we can do is use
prepared queries (this is all 0.10 only), so that the SQL is parsed and only
the bound values are changed in each run. This is potentially much faster,
but I have no benchmarks for SQLite. The other solution, building on top of
the query that was used for the items in the unfiltered view, is easy for
tags and albums, but difficult for search, from an implementation point of
view.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Bug 155097] performance problems with recursive view and filtering

Bugzilla from andi.clemens@gmx.net
Hi Marcel,

yes currently I'm working on 0.9.x.
For text I already have an SQL query that is able to filter my 3700 images in
the album in less than one second. But rating etc is still slow.
Right now it works as follows for text:

1. Run the query on the whole database
2. save resulting image IDs in LLongList
3. Filter out the ones that are existent in current icon view and LLongList
4. Display those images

This is really fast.
Also the general displaying of albums is faster now, because I removed some
redundant filtering (that was done twice and also when not using quickfilters
at all). If you click on a large album, the icons / thumbnails are displayed
much faster now. Before this, even with already cached thumbnails, displaying
was very slow.

But maybe I removed too much (:-)), because some things don't work the way the
should. For example if I assign a tag, the view is reloaded and the first
item is selected.

I will test this some more, don't know if I have time today. I'll also take a
look at the 0.10.x implementation, maybe I just skip the work on such a patch
if the KDE4 version is much faster. I don't think we need to do too much work
on the KDE3 branch anymore.

Andi


On Thursday 04 September 2008 14:51:15 Marcel Wiesweg wrote:

> (sorry for posting with mail, but bko doesnt work at the moment)
>
> I assume you are working on 0.9, but some thoughts on 0.10:
> For 0.10, some caching of simple values takes place, so database queries
> should be reduced in some cases. This means, if you are working on 0.10,
> then there is a bug elsewhere.
> Caching is easy for rating, date, format, size.
> Tags are not cached by default for every image, but only on first demand,
> so minor problem.
> The one filter that cannot properly be solved with caching is text. Here it
> would be most interesting to have a SQL solution, like one-big-query. You
> cannot pass say 20000 image ids in one SQL statement. What we can do is use
> prepared queries (this is all 0.10 only), so that the SQL is parsed and
> only the bound values are changed in each run. This is potentially much
> faster, but I have no benchmarks for SQLite. The other solution, building
> on top of the query that was used for the items in the unfiltered view, is
> easy for tags and albums, but difficult for search, from an implementation
> point of view.
> _______________________________________________
> Digikam-devel mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-devel

_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel