[digiKam-users] Digikam 6.2 beta memory leaks?

[leoutation]

Hi
I had to downgrade from digikam-git 6.2 beta to digikam 6.1 because 6.2
eats huge memory, about 4 Go when rebuilding fingerprints (18.000
images). No dk error when launching it in console.
This dk version is unusable for me. Worth thing is computer freezing. I
had to kill user/digikam process in tty console.
Mysql eats only 200M.
A digikam memory leaks?
Mysql external db on local hard drive.
Archlinux system

--
Maderios

[Gilles Caulier-4]

Hi,

There is no special changes done in Fingerprints processing.
The main changes in whole source code in 2 ports to C++11
compatibility ; nullprt and virtual overide. Both change code
everywhere.

How did you investiguate the memory leak exactly ? With valgrind or
just to check the memory consumption through the linux kernel stats ?

Did you compile yourself whole code, or did you try the AppImage linux bundle ?

Best
Gilles Caulier

Le mer. 8 mai 2019 à 19:04, <[hidden email]> a écrit :

>
> Hi
> I had to downgrade from digikam-git 6.2 beta to digikam 6.1 because 6.2
> eats huge memory, about 4 Go when rebuilding fingerprints (18.000
> images). No dk error when launching it in console.
> This dk version is unusable for me. Worth thing is computer freezing. I
> had to kill user/digikam process in tty console.
> Mysql eats only 200M.
> A digikam memory leaks?
> Mysql external db on local hard drive.
> Archlinux system
>
> --
> Maderios

[Gilles Caulier-4]

Hi,

Here a quick test on my host computer, to rebuild all finger prints
indicate a memory allocation moving from 800Mb to 1.6Mb, depending of
the operations done in background (visible on the console). The memory
is allocated and free when transactions is complete to the database
(here Mysql). The memory graph provided by the desktop (Plasma) do not
indicate a huge memory leak. The fluctuations exists but it's
negligible. Here the collection is huge and the computation take age.
Image to register are JPEG, PNG, TIFF, and RAW.

So, typically, it's not reproducible here.

Gilles Caulier

Le mer. 8 mai 2019 à 19:09, Gilles Caulier <[hidden email]> a écrit :

>
> Hi,
>
> There is no special changes done in Fingerprints processing.
> The main changes in whole source code in 2 ports to C++11
> compatibility ; nullprt and virtual overide. Both change code
> everywhere.
>
> How did you investiguate the memory leak exactly ? With valgrind or
> just to check the memory consumption through the linux kernel stats ?
>
> Did you compile yourself whole code, or did you try the AppImage linux bundle ?
>
> Best
> Gilles Caulier
>
> Le mer. 8 mai 2019 à 19:04, <[hidden email]> a écrit :
> >
> > Hi
> > I had to downgrade from digikam-git 6.2 beta to digikam 6.1 because 6.2
> > eats huge memory, about 4 Go when rebuilding fingerprints (18.000
> > images). No dk error when launching it in console.
> > This dk version is unusable for me. Worth thing is computer freezing. I
> > had to kill user/digikam process in tty console.
> > Mysql eats only 200M.
> > A digikam memory leaks?
> > Mysql external db on local hard drive.
> > Archlinux system
> >
> > --
> > Maderios

[leoutation]

On 5/8/19 7:09 PM, Gilles Caulier wrote:

> Hi,
>
> There is no special changes done in Fingerprints processing.
> The main changes in whole source code in 2 ports to C++11
> compatibility ; nullprt and virtual overide. Both change code
> everywhere.
>
> How did you investiguate the memory leak exactly ? With valgrind or
> just to check the memory consumption through the linux kernel stats ?
>
> Did you compile yourself whole code, or did you try the AppImage linux bundle ?

I checked resident memory in htop.
I compile digikam git with Arch Aur this modified script (less dependencies)

pkgname=digikam-git
pkgver=r42728.bf87ed54b8
pkgrel=1
pkgdesc='Digital photo management application for KDE'
arch=('i686' 'x86_64')
license=('GPL')
url="http://www.digikam.org/"
depends=('liblqr' 'qtav' 'lensfun' 'knotifyconfig' 'kfilemetadata'
'opencv' 'threadweaver' 'qt5-xmlpatterns' 'libgphoto2' 'imagemagick')
makedepends=('git' 'extra-cmake-modules-git' 'eigen' 'doxygen' 'boost'
'mariadb' 'kdoctools')
optdepends=('hugin: panorama tool' 'kimageformats')
conflicts=('digikam')
provides=('digikam')
install=digikam-git.install
source=('digikam::git+git://anongit.kde.org/digikam')
md5sums=('SKIP')
groups=('digikamsc-git')

pkgver() {
   cd "${srcdir}/digikam"
   printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse
--short HEAD)"
}

prepare() {
if [[ -d "${srcdir}/build" ]]; then
       msg "Cleaning the previous build directory..."
       rm -rf "${srcdir}/build"
   fi
   mkdir "${srcdir}/build"
}

build() {
   cd "${srcdir}/build"
   cmake "${srcdir}/digikam" -DCMAKE_BUILD_TYPE=Release \
                -DCMAKE_INSTALL_PREFIX=/usr \
                -DLIB_INSTALL_DIR=lib \
                -DKDE_INSTALL_USE_QT_SYS_PATHS=ON \
                -DBUILD_TESTING=OFF \
                 -DENABLE_AKONADICONTACTSUPPORT=OFF \
                 -DENABLE_KFILEMETADATASUPPORT=ON \
                -DENABLE_MYSQLSUPPORT=ON \
                 -DENABLE_INTERNALMYSQL=ON \
                 -DENABLE_MEDIAPLAYER=ON \
                 -DENABLE_APPSTYLES=ON \
                -DENABLE_QWEBENGINE=OFF \
                -DOpenGL_GL_PREFERENCE=GLVND
   make
}

package() {
   cd "${srcdir}/build"
   make DESTDIR="${pkgdir}" install
}

>
> Best
> Gilles Caulier
>
> Le mer. 8 mai 2019 à 19:04, <[hidden email]> a écrit :
>>
>> Hi
>> I had to downgrade from digikam-git 6.2 beta to digikam 6.1 because 6.2
>> eats huge memory, about 4 Go when rebuilding fingerprints (18.000
>> images). No dk error when launching it in console.
>> This dk version is unusable for me. Worth thing is computer freezing. I
>> had to kill user/digikam process in tty console.
>> Mysql eats only 200M.
>> A digikam memory leaks?
>> Mysql external db on local hard drive.
>> Archlinux system
>>
>> --
>> Maderios

--
Maderios

[leoutation]

On 5/8/19 7:17 PM, Gilles Caulier wrote:

> Hi,
>
> Here a quick test on my host computer, to rebuild all finger prints
> indicate a memory allocation moving from 800Mb to 1.6Mb, depending of
> the operations done in background (visible on the console). The memory
> is allocated and free when transactions is complete to the database
> (here Mysql). The memory graph provided by the desktop (Plasma) do not
> indicate a huge memory leak. The fluctuations exists but it's
> negligible. Here the collection is huge and the computation take age.
> Image to register are JPEG, PNG, TIFF, and RAW.
>
> So, typically, it's not reproducible here.
>

Hi
I built last digikam-git, issue seems solved. I rebuilt fingerprints for
19.000 images. It took only 260M, certainly because i already did it
with digikam 6.1. I think, may be i'm wrong, it takes more ressources
with big files/images. This is my case. I have images (png xcf) files
bigger than 120 M
I have to say that I run lts 4.19 kernel. I upgraded it just before last
  digikam build. May be issue came from old kernel, i'll never know it...
Thanks
Greetings

--
Maderios

[leoutation]

On 5/9/19 5:38 PM, [hidden email] wrote:

> I built last digikam-git, issue seems solved. I rebuilt fingerprints for
> 19.000 images. It took only 260M, certainly because i already did it
> with digikam 6.1. I think, may be i'm wrong, it takes more ressources
> with big files/images. This is my case. I have images (png xcf) files
> bigger than 120 M
> I have to say that I run lts 4.19 kernel. I upgraded it just before last
>   digikam build. May be issue came from old kernel, i'll never know it...

Precision: it was not "old kernel" but longterm linux kernel 4.19.40
released in may 2019.

--
Maderios