[digikam] [Bug 365202] New: git master 2016-07-07 : Segfault on faces detect.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #56 from Evert Vorster <[hidden email]> ---
Now we are getting somewhere.

I finally managed to compile a version of opencv that has tbb disabled.

With Maik's patch installed I was able to scan through 25% of my library before
I hit the bug of "MySQL server has gone away", so definitely have one issue
nailed down to opencv's use of tbb.
I can try again with internal SQlite, and see if I can scan through my
collection completely.

Since this version of opencv I have now is the contrib version, I will try
again to build with Gilles' patch.

Looking back over the patches, which, if any, are recommended for me?
Do I file a bug report at OpenCV for the crashes in TBB? (I did a bit of
reading about it, and appears that the memory errors that valgrind reports is
because TBB uses their own scheduler and that can confuse valgrind)

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #57 from Evert Vorster <[hidden email]> ---
Created attachment 100002
  --> https://bugs.kde.org/attachment.cgi?id=100002&action=edit
Compile error Gilles patch to use external face module

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #58 from Evert Vorster <[hidden email]> ---
OK, so now DigiKam crashes with both internal and external SQL, at
approximately 30% through my collection. I'll start a new bug for that.
I get a compile error with Gilles' patch, and Maik's seems to head off the
error with TBB if it is disabled in opencv.
I'll compile digikam without any patch, and run it against my collection with
both databases to confirm that Maik's patch is needed in conjunction with a TBB
free opencv.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #59 from [hidden email] ---
Note : with my patch, multi-coree enabled, and mysql server, no crash. All work
fine.

I didn't see the tbb opencv dependency. I will look if mine as this module
enabled.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #60 from Evert Vorster <[hidden email]> ---
Hi there..

OK, so, I recompiled digikam with no patch, and ran the face detection against
my picture library.
The face detect would usually fail anywhere from 1 to 4% through the library.

Ever since I have recompiled opencv to not use TBB, digikam scans much further
into the library, regardsless of whether there is a patch applied or not. So I
think we are at the root of this issue.

I will open a bug report with opencv, as the bug is in their handling of TBB.

I will open another bug for the failures I am seeing now, just to keep the
troubleshooting that was done seperate and not confuse the issue. Thank you
very much for your help!

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #61 from Evert Vorster <[hidden email]> ---
Small correction, it seems my source tree was not cleaned as thoroughly as I
thought it was, and so I appear to have built digikam with all the patches
applied when I thought it was not.
My apologies. I am now rebuilding vanilla digikam, to see if it still fails
with TBB disabled in OpenCV.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #62 from Evert Vorster <[hidden email]> ---
Just tested with the vanilla code, and digikam does indeed crash earlier than
with the patch. I am now installing just Maik's patch, and re-running it.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #63 from Evert Vorster <[hidden email]> ---
Update on the status.
I am now compiling digikam with all the patches except for Gilles' patch to use
face module form opencv_contrib instead from digiKam core. That patch causes a
compile error. I think it's due to the abi version of opencv that I have is
slightly different.

With just Maik's face31.patch installed I was able to scan through my
collection with about 4 tries. I ended up with about 4000 faces identified out
of 32000 photos.
I am now going to try again with all the patches installed and see if they
address different issues.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #64 from [hidden email] ---
Evert,

My patch use OpenCV 3.1 tarball. OpenCV-contrib come from github as well
(current implementation. As i know there is no OpenCV-contrib tarball released.
This is very problematic as API/ABI version can be the hell to follow. This is
why i included face module code in digiKam core as well instead to use
OpenCV-contrib instead. Sound like my firts choice is not too bad after all.

As i already said somewhere and sometime, openCV is a another big puzzle...

I can update code in digiKam core about face module, as i know now what i need
to change to compile fine with OpenCV 3.1.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #65 from [hidden email] ---
As i said previously, i processed on my VM with my collection all faces scan
with Mysql server + multicore CPU support. No crash.

About TBB support in OpenCV 3.1 that i compiled, i don't set anything to
configure OpenCV, excepted the fact to pass path to OpenCV-Contrib modules
source code to include Face module as well. That all.

As i can see TBB is disabled by default, so i suspect that no TBB here.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

Maik Qualmann <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #99992|0                           |1
        is obsolete|                            |

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #66 from Maik Qualmann <[hidden email]> ---
Created attachment 100038
  --> https://bugs.kde.org/attachment.cgi?id=100038&action=edit
currentFace31.patch

New patch, face modul updated to the latest version of OpenCV3.1.
Please test it with  this patch.

Maik

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #67 from Evert Vorster <[hidden email]> ---
Thanks for the updated patch. It's building now.

My latest test was with all the patches installed, and it scanned all the way
through my collection.  There are a few smaller bugs where the program does not
work as expected that could very well be due to the ABI change, so I'll test
all of them again.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

Evert Vorster <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|UNCONFIRMED                 |NEEDSINFO

--- Comment #68 from Evert Vorster <[hidden email]> ---
Maik, your patch works beautifully.
With yours and Gilles patches installed I am unable to make digikam segfault.
This is running with external MySQL db, and updating the unknown pictures as
they come up..

Pretty solid. Unfortunately the actual face recognition is pretty poor, with
only a 10% successful recognition. ie: recognizes a face that I have provided
hundreds of examples for.

Will this become part of the standard digikam?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|FIXED                       |---
             Status|NEEDSINFO                   |UNCONFIRMED

--- Comment #69 from [hidden email] ---
Ever,

Please do not close this file, we must apply patches to git/master before to
mark it as resolved.

Q: Your opencv still compiled with Intel TBB support ?

The recognition algorithm need to be improved, that true. We have currently a
student working on a eyes auto detection and correction which will come as an
extension to face engine in digiKam. If project is completed, we will assign
recognition improvement next summer (not before as it will be a part of GoSC)

Maik,

What's will be the synthesis about this file to apply patches on git/master
exactly ?

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #70 from Maik Qualmann <[hidden email]> ---
The minimum version of OpenCV3 we must set to V3.1. OpenCV2 should compile, but
tested only a few months ago.

Maik

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #71 from [hidden email] ---
Maik, well apply the patches.

MXE still with openCV 2.4.X
Macport is already openCV 3.1.X

So i can check openCV 2 when patches will be applied.

Gilles

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #72 from Maik Qualmann <[hidden email]> ---
Git commit 8cdfcc52f402b44378fe8e6a9b7961585e17340e by Maik Qualmann.
Committed on 13/07/2016 at 17:17.
Pushed by mqualmann into branch 'master'.

apply patch #99970 to add mutex to prevent non re-entrency in OpenCV API

M  +6    -0    libs/facesengine/detection/opencvfacedetector.cpp

http://commits.kde.org/digikam/8cdfcc52f402b44378fe8e6a9b7961585e17340e

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #73 from Maik Qualmann <[hidden email]> ---
Git commit 3e31dad1c6c10bb6da6a269a007eee5cc209e412 by Maik Qualmann.
Committed on 13/07/2016 at 17:22.
Pushed by mqualmann into branch 'master'.

apply patch #99986 to check for a valid QImage

M  +5    -0    libs/facesengine/facedetector.cpp

http://commits.kde.org/digikam/3e31dad1c6c10bb6da6a269a007eee5cc209e412

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=365202

--- Comment #74 from Maik Qualmann <[hidden email]> ---
Git commit 88123604ccac3cdda4557273f1b280d6772adc31 by Maik Qualmann.
Committed on 13/07/2016 at 17:27.
Pushed by mqualmann into branch 'master'.

apply patch #100038 to update openCV3 face modul to the current version

M  +1    -0    libs/facesengine/CMakeLists.txt
M  +8    -24   libs/facesengine/opencv3-face/eigen_faces.cpp
M  +17   -5    libs/facesengine/opencv3-face/face.hpp
M  +0    -9    libs/facesengine/opencv3-face/face_basic.hpp
M  +13   -0    libs/facesengine/opencv3-face/facerec.cpp
M  +2    -5    libs/facesengine/opencv3-face/facerec.hpp
M  +8    -25   libs/facesengine/opencv3-face/fisher_faces.cpp
M  +8    -25   libs/facesengine/opencv3-face/lbph_faces.cpp
M  +5    -2    libs/facesengine/opencv3-face/precomp.hpp
A  +114  -0    libs/facesengine/opencv3-face/predict_collector.cpp    
[License: Unknown license]  *
A  +127  -0    libs/facesengine/opencv3-face/predict_collector.hpp    
[License: Unknown license]  *
M  +38   -0    libs/facesengine/recognition-opencv-lbph/facerec_borrowed.cpp
M  +9    -0    libs/facesengine/recognition-opencv-lbph/facerec_borrowed.h

The files marked with a * at the end have a non valid license. Please read:
http://techbase.kde.org/Policies/Licensing_Policy and use the headers which are
listed at that page.


http://commits.kde.org/digikam/88123604ccac3cdda4557273f1b280d6772adc31

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel