[Bug 255858] New: memory leak (?) while thumbnailing very large files!

https://bugs.kde.org/show_bug.cgi?id=255858

           Summary: memory leak (?) while thumbnailing very large files!
           Product: digikam
           Version: 1.5.0
          Platform: Compiled Sources
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: crash
          Priority: NOR
         Component: general
        AssignedTo: [hidden email]
        ReportedBy: [hidden email]


Version:           1.5.0 (using KDE 4.5.1)
OS:                Linux

When I open a folder with very large files (>600 Mb), and digikam starts
producing the thumbnails, the program eats up all memory resources in no time,
forcing everything else to swap and thus rendering the system unusable.
Ultimately, the thumbnails are generated but it takes forever, and, as I said,
the system is totally unusable. I am no programmer but I think there is
something wrong with the thumbnail generation routine. Thanks!

Reproducible: Always

Steps to Reproduce:
Put a large file in a folder (>600 Mb; mine are actually 800 Mb & 1 Gb) and go
to that folder for the program to thumbnail them.

Actual Results:  
Digikam eats all memory resources, rendering the system unusable.

Expected Results:  
Digikam should manage the available memory efficiently, not causing everything
else to hang, and choke on the data itself (as I said above, the process takes
forever)

The same happens in current svn

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=255858


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
          Component|general                     |Thumbnails




--- Comment #1 from Gilles Caulier <caulier gilles gmail com>  2010-11-02 10:18:47 ---
Can you run digiKam under valgrind and post the console trace here.

The command line to use is this one :

valgrind --tool=memcheck --leak-check=full --error-limit=no digikam

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=255858





--- Comment #2 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-11-02 10:24:26 ---
As we dont support tiles, the only solution would be to refuse thumbnailing
such a large image. Assuming the file is compressed and it's not a JPG of PGF,
you will need a multiple of the file size in memory to load the image.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=255858





--- Comment #3 from krzysztof pijarski <kpijarski gmx com>  2010-11-02 11:02:02 ---
ok, so there is the answer, thanks. it is not a memory leak then. why did you
decide against tiles? the files in question are indeed compressed tif's so
thumbnailing one of those needs multiple Gb of memory. I think that in such
situations it is critical to disable thumbnailing then, although this is
something that is hard to come at - what about somebody who has 1 Gb of RAM (I
have 4), and has files, of, say 250 Mb? This person will be in trouble as I am.

So if you do not plan to use tiles, would there be a possibility to use the
embedded thumbnail (that GIMP generates, for example?). I haven't tried in a
long time, but digikam has always had problems with embedding previews in tif
files.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=255858





--- Comment #4 from Gilles Caulier <caulier gilles gmail com>  2010-11-24 09:13:37 ---
digiKam 1.6.0 is out:

http://www.digikam.org/drupal/node/550

Please update and check if this entry still valid.

Thanks in advance

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=255858





--- Comment #5 from krzysztof pijarski <kpijarski gmx com>  2010-11-27 10:47:56 ---
Gilles,

I tried the svn version yesterday and nothing changed - I threw the biggest
files I have at it (two 1Gb 16bpp files) and it took 20+ minutes to render one
thumbnail, with the system totally unusable; digikam itself took up to 2.8 Gb
of memory, and the whole process took all my swap space (4 Gb) and all my RAM
(4 Gb)...

Thanks!

krzysztof

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=255858





--- Comment #6 from Marcel Wiesweg <marcel wiesweg gmx de>  2011-01-18 16:57:20 ---
Proposal:
There is a class KMemoryInfo which we can use to provide information about
system memory (Linux and Windows atm):
http://reviewboard.kde.org/r/2536/
It was not taken into kdelibs, so we will need to copy the source.

With this code we could check before loading a huge file if enough memory is
available (and btw, also adjust our cache size which has become small for
megapixel images of today cameras).

Gilles?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=255858


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |FIXED




--- Comment #7 from Marcel Wiesweg <marcel wiesweg gmx de>  2011-01-26 21:36:05 ---
SVN commit 1217348 by mwiesweg:

Check available memory before attempting large allocations.

BUG: 255858



 M  +2 -1      NEWS  
 M  +35 -0     libs/dimg/loaders/dimgloader.cpp  
 M  +5 -3      libs/dimg/loaders/dimgloader.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1217348

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=255858





--- Comment #8 from Marcel Wiesweg <marcel wiesweg gmx de>  2011-01-26 21:36:44 ---
SVN commit 1217349 by mwiesweg:

Missing commit part

CCBUG: 255858


 M  +3 -7      dimg.cpp  
 M  +0 -1      imagehistory/dimagehistory.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1217349

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=255858

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|                            |2.0.0

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel