[digikam] [Bug 365202] New: git master 2016-07-07 : Segfault on faces detect.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #17 from [hidden email] ---
Created attachment 99970
  --> https://bugs.kde.org/attachment.cgi?id=99970&action=edit
add mutex to prevent non re-entrency in OpenCV API

Try with my little patch if it fix the problem.
Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|External MySQL database     |External MySQL database
                   |faces update segfault       |faces update segfault
                   |                            |[patch]

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #18 from Evert Vorster <[hidden email]> ---
Hey there...
Unfortunately, that patch did not stop the crashing.
With just scanning for faces, it got about 6% through my collection of 30,000
photos ( 160Gb ) before crashing again. This time it's not a segfault, but the
program halts with this weird error:
digikam.dbengine: Prepare failed!
digikam.dbengine: Failure executing query:
 "SELECT orientation FROM ImagaInformation WHERE imageid=?;"
Error messages: "QMYSQL3: Unable to prepare statement" "MySQL server has gone
away" 2006 2
Bound values: ()

One thing worth noticing though, is that I also tried to scan and recognize
faces, and it zipped through that entire collection in seconds, and did not
find a single face.

Just recognizing faces on their own seems to work fine, however.

I'll start building digikam with debugging enabled, and see if I can track this
one down.,

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #19 from Evert Vorster <[hidden email]> ---
Created attachment 99977
  --> https://bugs.kde.org/attachment.cgi?id=99977&action=edit
Segfault, with the patch applied on detect faces only

Once I rebuilt digikam with debugging enabled, I ran it through gdb with
detecting faces only.
It failed fairly quickly with a segfault, and I have attached the log here.

Right now I am running the same through Valgrind, but it might be a while
before it crashes.

-Evert-

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #20 from Evert Vorster <[hidden email]> ---
Created attachment 99978
  --> https://bugs.kde.org/attachment.cgi?id=99978&action=edit
valgrind with mysql gone away error

There seems to be two distict modes of failure here. One is the "mysql gone
away" error, and the other is a segfault.
I just attached a valgrind for the mysql gone away error, I'll re-run and see
if I am "lucky" enough to catch the program segfaulting in valgrind.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

Maik Qualmann <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #21 from Maik Qualmann <[hidden email]> ---
> digikam.dbengine: Failure executing query:
>  "SELECT orientation FROM ImagaInformation WHERE imageid=?;"

Have you copied this string or it may contain a typo?

ImagaInformation => ImageInformation
The query string is correct in digKam source code.

Maik

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #22 from Evert Vorster <[hidden email]> ---
Re: Maik Qualmann,
I did indeed make a typo.

Thanks for pointing that out.
-Evert-

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #23 from Evert Vorster <[hidden email]> ---
Ah, Maik, the same error is repeated in the valgrind log that I just attached,
and since I did not type that one, it definitely does not contain typos.

Thanks for looking into this!
-Evert-

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #24 from Maik Qualmann <[hidden email]> ---
I see in the valgrind log that QuickTime video is included. I think it's
possible that Exiv2 crashes here. Can you these video files for test removed in
face recognition?

Maik

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #25 from Evert Vorster <[hidden email]> ---
Created attachment 99981
  --> https://bugs.kde.org/attachment.cgi?id=99981&action=edit
valgrind log

OK, Thanks for looking into this.
I have moved all the .mov and .mp4 from that directory.

There are still some cr2, png, and tiff mixed in, but exiv should not have a
problem with those.
Right now digikam is running through my collection, and let's see if it
crashes.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #26 from Evert Vorster <[hidden email]> ---
Created attachment 99984
  --> https://bugs.kde.org/attachment.cgi?id=99984&action=edit
Console output of crash after removing movie files.

So, I have removed all the movie files from the directory, and let the face
scanner do it's thing.
Now I'll try again through gdb, and also remove anything that is not a .jpg,
just to be sure.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #27 from Maik Qualmann <[hidden email]> ---
Can you check if these images are broken?

/data/DigiKam/Photos/Home/2015/02-28 Oval Track/Evert Pictures/IMG_7377.JPG
/data/DigiKam/Photos/Home/2015/02-28 Oval Track/Evert Pictures/IMG_7378.JPG
/data/DigiKam/Photos/Home/2015/02-28 Oval Track/Evert Pictures/IMG_7381.JPG
/data/DigiKam/Photos/Home/2015/02-28 Oval Track/Evert Pictures/IMG_7383.JPG

Maik

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #28 from [hidden email] ---
Maik,

even if images are broken, or if video/raw files crash Exiv2, all Exiv2 API are
wrapped around exception catch which must be handle by high level
implementation in digiKam.

Typically if a file break Exiv2, we must able to take the right direction to
prevent an broken workflow in face detection threads.

It's know that Exiv2 < 0.25 are very sensible of video files for ex, but it's
better now with last 0.25 stable release.

Evert;

Just to be sure, which Exiv2 version do you use exactly ?

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #29 from Evert Vorster <[hidden email]> ---
Maik, thanks again for looking into this.

I am currently doing a valgrind log hoping to catch another segfault. It's just
so terribly slow!
On every try I get about 3 - 6% through the database before digikam quits. Like
I mentioned before there seems to be two distict issues, one where the database
goes away, and one where digikam segfaults.

I have applied the small patch from Gilles,  but it does not seem to have made
a difference to either error.

I have to re-iterate that this is scanning through the database doing faces
detect only. If I do faces detect and recognize, the process completes very
quickly, and no faces are detected.

The four pictures listed above open fine with Gwenview.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #30 from Evert Vorster <[hidden email]> ---
Gilles:
extra/exiv2 0.25-3
and libkexiv2 from git, r782.6c196e4-1

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #31 from [hidden email] ---
My patch must be the right direction but certainly not at the right level in
source code.

Look well as we touch data not initialized while OpenCV API call  It sound like
a non re-entrancy somewhere.

Remember that face fingerprints while detection is computed in a separated
thread. It's more complex when multi-cores are used. Perhaps the patch must be
applied in some top level call in facedetector class.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #32 from [hidden email] ---
Exiv2 is fine.

We don't use libkexiv2 anymore. Implementation is not in digiKam core, to
reduce the puzzle.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #33 from [hidden email] ---
There is some test to do , if possible.

1/ using OpenCV2 instead OpenCV3. There is a flags to turn off in digiKam cmake
configuration script before to compile.
2/ using sqlite database instead Mysql, to see if crash is reproducible.
3/ in all case single core and multicore must be tested to validate.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #34 from Maik Qualmann <[hidden email]> ---
Created attachment 99986
  --> https://bugs.kde.org/attachment.cgi?id=99986&action=edit
facedetector.patch

Please test also this patch.

Maik

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #35 from Evert Vorster <[hidden email]> ---
I am currently running digikam through valgrind. Once that crashes, I will
install Maik's patch only, and see if I can reproduce the issue.

I will notify the maintainer of digikam-git that libexiv2 is no longer a
requirement for digikam.

I can turn off the external database for testing purposes, but having the
database external is very much a desired feature for me.

Unfortunately, downgrading to opencv 2 is not an option on this machine, there
is too much of my other software that depend on it.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel