[digikam] [Bug 351658] New: Ubuntu 14.04: Digikam (4.12) cannot perform maintance tasks - memory leak.

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

            Bug ID: 351658
           Summary: Ubuntu 14.04:  Digikam (4.12) cannot perform maintance
                    tasks - memory leak.
           Product: digikam
           Version: unspecified
          Platform: Ubuntu Packages
                OS: Linux
            Status: UNCONFIRMED
          Severity: major
          Priority: NOR
         Component: Maintenance
          Assignee: [hidden email]
          Reporter: [hidden email]

Whatever maintenance taks I choose (thumbnails or metadata or... small or big
albums... with all cores used or not):  the program starts using all available
memory.  Endlessly.  And whenever I got the patience to wait it out:  silently
disappears...

Reproducible: Always

Steps to Reproduce:
1. Choose Maintenance
2. Choose album(s)/task(s).


Actual Results:  
Nothing happens.  After lots of time:  program crashes (no crash report).

Expected Results:  
Doing maintenance.  (Without having to work around it, for that is possible:
fi, opening an album (subtree included), selecting all images, and than
synchronising metadata via File/Write metadata to file.)

Nothing I can think of.

I have Debug Symbols installed.  But mostly, the crash reporter itself...
crashes.  Or doesn't come up with a useful crash report...  (that's a separate
issue).

And as said:  I don't get a crash message for this problem anyway.

I haven't gotten into Valgrind yet.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

Bizy <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Ubuntu 14.04:  Digikam      |Ubuntu 14.04:  Digikam
                   |(4.12) cannot perform       |(4.12) cannot perform
                   |maintance tasks - memory    |maintenance tasks - memory
                   |leak.                       |leak.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=351658

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
           Severity|major                       |normal
            Summary|Ubuntu 14.04:  Digikam      |Prevent to fill whole
                   |(4.12) cannot perform       |memory when all CPU core
                   |maintenance tasks - memory  |are used to process
                   |leak.                       |Maintenance tools

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=351658

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #1 from Gilles Caulier <[hidden email]> ---
Maik,

To prevent this problem, we need a function which will check whole available
memory and compute how many CPU core be used at the same time without to fill
all memory.

The same problem exist in BQM...

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=351658

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Prevent to fill whole       |Prevent to fill whole
                   |memory when all CPU core    |memory when all CPU cores
                   |are used to process         |are used to process
                   |Maintenance tools           |Maintenance tools

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #2 from Bizy <[hidden email]> ---
So I downgraded to 4.0... :-(
I'm doing what's necessary to (try) to save all the work that hasn't been saved
into the images yet.  And then I'll create a new database and will give
feedback about wether I'm able to do maintenance or not...

(The program became so time consuming...  The database became as big as 9GB, it
took 5 to 10 minutes before loading.  I tried to remove a big part of this
'collection', but it didn't reduce the loading time, didn't remove tons of tags
(while the images were removed)...  The slightest action (like changing a
setting or writing the tag of one image) and the program kept hanging or
crashed...  The menu disappeared (Ctrl + M didn't work, 'Show menubar' didnt
work... - I had to remove and reinstall the 'show menubar'-button to make the
menu visible...)
And so on...
No possibility to create crash reports (in spite of having bug symbols
installed), or the crash reporter itself crashing...
I had enough of it...

Still hoping that I can keep using Digikam, coz only Digikam has all possible
functionality that I need - all, and more than I could think of...

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #3 from Bizy <[hidden email]> ---
PS:  I don't know who changed the title, but this doesn't reflect anymore what
happened.  Only one core was 'hijacked' by Digikam (so using 100%, sometimes
switching to another core - I have 8 cores in total).  Total use of CPU was
only about 12 %.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Maik Qualmann]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #4 from Maik Qualmann <[hidden email]> ---
Which version of digikam you used? Which database was 9 GB great, digikam4.db
or thumbnails-digikam.db?

Maik

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #5 from Bizy <[hidden email]> ---
(In reply to Maik Qualmann from comment #4)
> Which version of digikam you used? Which database was 9 GB great,
> digikam4.db or thumbnails-digikam.db?
>
> Maik

Digikam Version 4.12
Ubuntu Versiion 14.04
Processor:  Intel® Core™ i7-4710MQ CPU @ 2.50GHz × 8
System:  64 bit

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #6 from Bizy <[hidden email]> ---
So I can confirm that I can do maintenance again...  :-)  (with downgrade to
Version 4.0)
(though it takes awfully long;  for now (about 13.000 images), it takes hours
and hours to write the tags from the images to the database, I wonder if that's
regular...  but most importantly, it works...!)

So somewhere along the installation process of version 4.12, something goes
wrong...
(I followed these guidelines:
<a>http://ubuntuhandbook.org/index.php/2015/08/install-latest-digikam-4-12-ubuntu/</a>
And I installed both packages (extra and kubuntu backports)
I'm not ready yet to try it again (too much work to do with a stable
program)... But one day, I will...

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #7 from Gilles Caulier <[hidden email]> ---
Downgrading digiKam will certainly downgrade other components, as Exiv2
library. With previous 0.24.0 release a lots of bugs have been introduced.
Using previous version do not have these problems. Writing metadata to image
use Exiv2 a lots.

Exiv2 0.25.0 fix the problem. This version must be used with digiKam 4.12.0...

Note : your problem sound like completely different than report title, but my
title changes are true. If you run maintenance tools with all CPU core, there
is no check to see how many memory is available to process, and all is filled.
Here with 16 or 32 GB, there are no problem to run 8 threads with a i7 cpu, but
with 8 or 4 Gb, it's completely different.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #8 from Bizy <[hidden email]> ---
(In reply to Gilles Caulier from comment #7)

> Downgrading digiKam will certainly downgrade other components, as Exiv2
> library. With previous 0.24.0 release a lots of bugs have been introduced.
> Using previous version do not have these problems. Writing metadata to image
> use Exiv2 a lots.
>
> Exiv2 0.25.0 fix the problem. This version must be used with digiKam
> 4.12.0...
>
> Note : your problem sound like completely different than report title, but
> my title changes are true. If you run maintenance tools with all CPU core,
> there is no check to see how many memory is available to process, and all is
> filled. Here with 16 or 32 GB, there are no problem to run 8 threads with a
> i7 cpu, but with 8 or 4 Gb, it's completely different.
>
> Gilles Caulier

Thanks, Gilles, for your comment!

1) Regarding Exiv:  can you tell me how to do this upgrade step by step (or
send me a link)?  Of course, I know about this problem, it's been reported
often.  But then again:  I didn't find any clear (meaning:  clear to me)
instructions on how to upgrade, as it got too technical.  I tried (or actually
did) upgrade exiv with the terminal, but I wonder if that's enough.  There seem
to be isues with older parts that need to be removed in another way... That's
where the too technical part starts...

2)  Regarding the title:  fine to me!  As long as it's clear enough for you
guys to idenitfy the problem... ;-)
Seems that I have to regret I didn't order more memory then? :-(
But I read about the 'same' (at least regarding memory leak) problem for
someone with 32GB of memory... (same behaviour, gradually using it all, and
then crashing or disappearing...)
(tried to google it, but can't find it for now)

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #9 from Bizy <[hidden email]> ---
(In reply to Bizy from comment #5)

> (In reply to Maik Qualmann from comment #4)
> > Which version of digikam you used? Which database was 9 GB great,
> > digikam4.db or thumbnails-digikam.db?
> >
> > Maik
>
> Digikam Version 4.12
> Ubuntu Versiion 14.04
> Processor:  Intel® Core™ i7-4710MQ CPU @ 2.50GHz × 8
> System:  64 bit

O, and i forgot:  the large database is (was) thumbnails-digikam.db

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #10 from Bizy <[hidden email]> ---
So, after 7 (!) hours, only 11% of the 'maintenance' (writing tags to new
database, in version 4.0, 8-core processor, 8GB memory)  is done.  Is that how
it is supposed to be?  (that would supprise me...)  
Should I create a new bug report?  Or is this hopelessly outdated? (as it is
about verson 4.0)
I really don't know what to do...

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #11 from Gilles Caulier <[hidden email]> ---
This kind of dysfunction have been already reported in the past.

I recommend to use 4.12.0 with right Exiv2 0.25 to prevent crashes.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #12 from Bizy <[hidden email]> ---
Okay, thanks, I know.
And then, how?
Apparently I have version 0.23.  And I need version 0.25.  And there is the
issue of files of the old version still being used...
I found this tutorial:
http://dev.exiv2.org/projects/exiv2/wiki/Building_on_Ubuntu_using_Cmake
(It's this kind of step by step, command line by command line guide that I
need.
But at step 3, it stops ('http://dev.exiv2.org/svn/trunk' doesn't exist
anymore).
So then what?



(In reply to Gilles Caulier from comment #11)
> This kind of dysfunction have been already reported in the past.
>
> I recommend to use 4.12.0 with right Exiv2 0.25 to prevent crashes.
>
> Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #13 from Bizy <[hidden email]> ---
There is this page of exiv2 itself (http://www.exiv2.org/download.html)
But when I download the package, I don't find a .sh-file.
And then I don't find the file I need -according to the guide - to make the
.sh-file...
Honestly...

(In reply to Bizy from comment #12)

> Okay, thanks, I know.
> And then, how?
> Apparently I have version 0.23.  And I need version 0.25.  And there is the
> issue of files of the old version still being used...
> I found this tutorial:
> http://dev.exiv2.org/projects/exiv2/wiki/Building_on_Ubuntu_using_Cmake
> (It's this kind of step by step, command line by command line guide that I
> need.
> But at step 3, it stops ('http://dev.exiv2.org/svn/trunk' doesn't exist
> anymore).
> So then what?
>
>
>
> (In reply to Gilles Caulier from comment #11)
> > This kind of dysfunction have been already reported in the past.
> >
> > I recommend to use 4.12.0 with right Exiv2 0.25 to prevent crashes.
> >
> > Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #14 from Bizy <[hidden email]> ---
And when I try to reinstall digiKam 4.12, I get a crash report (see
screenshot).
(using this guide:
http://ubuntuhandbook.org/index.php/2015/08/install-latest-digikam-4-12-ubuntu/)

(In reply to Bizy from comment #13)

> There is this page of exiv2 itself (http://www.exiv2.org/download.html)
> But when I download the package, I don't find a .sh-file.
> And then I don't find the file I need -according to the guide - to make the
> .sh-file...
> Honestly...
>
> (In reply to Bizy from comment #12)
> > Okay, thanks, I know.
> > And then, how?
> > Apparently I have version 0.23.  And I need version 0.25.  And there is the
> > issue of files of the old version still being used...
> > I found this tutorial:
> > http://dev.exiv2.org/projects/exiv2/wiki/Building_on_Ubuntu_using_Cmake
> > (It's this kind of step by step, command line by command line guide that I
> > need.
> > But at step 3, it stops ('http://dev.exiv2.org/svn/trunk' doesn't exist
> > anymore).
> > So then what?
> >
> >
> >
> > (In reply to Gilles Caulier from comment #11)
> > > This kind of dysfunction have been already reported in the past.
> > >
> > > I recommend to use 4.12.0 with right Exiv2 0.25 to prevent crashes.
> > >
> > > Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #15 from Bizy <[hidden email]> ---
Created attachment 94376
  --> https://bugs.kde.org/attachment.cgi?id=94376&action=edit
crash report (part 1)

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Bizy]

https://bugs.kde.org/show_bug.cgi?id=351658

--- Comment #16 from Bizy <[hidden email]> ---
Created attachment 94377
  --> https://bugs.kde.org/attachment.cgi?id=94377&action=edit
crash report (part 2)

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel