[Bug 146288] New: Face detection / recognition for digikam

[Christian Weiske]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         
           Summary: Face detection / recognition for digikam
           Product: digikam
           Version: unspecified
          Platform: Gentoo Packages
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: wishlist
          Priority: NOR
         Component: Tags
        AssignedTo: digikam-devel kde org
        ReportedBy: cweiske cweiske de


Version:            (using KDE KDE 3.5.5)
Installed from:    Gentoo Packages

Reaching my personal 10.000 photos barrier in the next weeks, I face a problem:
Tagging photos by hand is not doable anymore since it takes too much time. While setting tags for a whole album (france, wedding, 2007) is easy and fast to do, tagging each of the 200+ photos in an album with the people on it is impossible.

When searching for photos, you often try to find a person - how it looked at a time, or generally need all photos of a certain person. Tags help here, but not all photos are tagged through the reason above. What I would like to see is a plugin that automatically detects faces and allows one to drag a tag from the tag tree on, or just add a tag to the detected face.

There are some face detection and recognition libraries out there that could be used, one well-known is the Intel's OpenCV lib [1]. libFace [2] is another one that specializes in face rec/det and uses intel's lib internally.

Gnome's F-Spot will get this feature trough the summer of code project [3, 4]. If it gets that, it'll be my personal killer feature to use fspot.

It would be very cool if digkam would get such a feature. And by the way, thanks for making digikam. My girlfriend and I use it since years, and it's been unvaluable for managing our photo collection.


References:
1 http://sourceforge.net/projects/opencvlibrary
2 http://libface.sourceforge.net/
3 http://idea.zanestate.edu/archives/2007/04/face-recognition-coming-to-f-spot/
4 http://bugzilla.gnome.org/show_bug.cgi?id=324391

Mailing list thread:
http://mail.kde.org/pipermail/digikam-users/2007-June/003642.html
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         
stephanolbrich gmx de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
      everconfirmed|0                           |1



------- Additional Comments From stephanolbrich gmx de  2007-06-03 12:55 -------
*** This bug has been confirmed by popular vote. ***
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From caulier.gilles gmail com  2007-06-03 13:37 -------
Hey,  i have never seen a wish confirmed after 2 days than it have been open...

Marcel, what do you think about Face detection. For me it sound a killer feature...

Gilles Caulier
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From marcel.wiesweg gmx de  2007-06-03 18:38 -------
Yes it sounds very interesting.
We would need to find out what the workflow is (training of the face recognition?), and how well the library works.
Then it's about writing an easy to use wrapper class for the library.
And finally integrating it to digikam, perhaps in the image search or in some yet to be written tagging workflow?
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From caulier.gilles gmail com  2007-06-03 18:50 -------
Marcel,

... or during downloading from camera...

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Arnd Baecker]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From arnd.baecker web de  2007-06-03 20:44 -------
Well, this will be a long road until success:
I had a quick look at installing opencv (which fortunately exists
in debian etch as version 0.9.7-4).
Then libface needs some tweaking in the header files so that opencv is found,
but compiles fine after that. (I can post more details on that if needed).
Then one has to look in the examples directory of libface-0.1,
and compile
constructmat.cpp  facedetect.cpp  recognize.cpp  train.cpp
However, it is not clear from the (essentially lacking, unless
I missed something) documentation how to use these:
Presumably it is train followed by recognize or facedetect.
For train one has to supply a file which contains names and
images of faces to be trained.
So far I only managed to get a segmentation fault...
(note to self: don't use my own pictures, but those of models,
the code seems to be picky ... ;-)

Then there is ellip-mask.jpg which provides a mask, maybe(?) used
for the input images. Whether the algorithm is able to detect
the region of interest with a face to be recognized is not clear to me.
Presumably all this will require some more digging in the original opencv
documentation.

Also note that the libface project does not appear that active,
after an initial check-in 6 weeks ago nothing happened...

OK, I will stop at this point, maybe someone else has more luck ...
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From caulier.gilles gmail com  2007-06-03 20:54 -------
Arnd,

I'm not surprised by this feedback (:=))). I know very well the world of experimental algorithms study (i'm work in a French scientific research center)

With this experience, i'm always aware about it. It's always an instable world witch need a long time to be stabilized.

Also, if the project of this shared library is not really active, we need to wait is something will be done during Google Summer Code to improve this situation.

My viewpoint is this : Face Detection is an interressing feature of course, but we have more important task to do before KDE4/Qt4 or DB update port for ex. (:=)))

So we wait and see...

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Arnd Baecker]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From arnd.baecker web de  2007-06-03 21:17 -------
> So we wait and see...


Yes, there is a lot more concrete stuff...
Apart from the technical side of the KDE4 transition,
there are also non-trivial workflow aspects
to be addressed wrt tagging (e.g
http://bugs.kde.org/show_bug.cgi?id=114465)
and improving searching/tag-filtering ...
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Christian Weiske]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From cweiske cweiske de  2007-06-03 21:25 -------
What I understood so far about opencv:
- There are two things: Face detection and face recognition. The first is to actually detect the position and region of a face, or all the faces on the photo. The second is to recognize the face at the detected region and say who it is. This needs a training set.

The simple / first thing I think should be implemented is face detection and face outline display in digikam. The user OR a program would say where the the faces are (e.g. user could select a frame as done for selection now), and digikam would store this information somewhere in the database. Further, a tag shall be associatable to that selected region (that would not only allow faces, but other details to be selected - as flickr allows it). This way, you can use a person name/tag and tag photo regions.

The second is face recognition. With the things implemented in the first point, it should be possible to extract the faces/regions from already tagged photos and train the face recognitioner. Untagged regions could then be sent to the recognition engine/script and be tagged automatically after.


So in short: Digikam first needs a way to tag/describe regions of an image. I think rectangles should be fine, no need to make it complicated and support circles or even AND OR and whatnot. After this, even outside scripts can be used to facedetect and recognition.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Arnd Baecker]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From arnd.baecker web de  2007-06-03 21:44 -------
My impression is that first the algorithmic side of both steps
- face detection
- face recognition
has to be working first (well, it did not for me in the first try...),
before the integration into digikam can be done.
All can be tested in detail separately
and this should then also clarify the needs on the digikam side.

Your suggestion to describe regions of an image and
associate those with a tag is indeed something of use
independent of the face detection.
This should have many applications (--> separate B.K.O entry?)

Best, Arnd
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Christian Weiske]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From cweiske cweiske de  2007-06-03 21:59 -------
Arndt, I don't think so. OpenCV already provides a working algorithm set, so digikam needs the infrastructure to use the results of that algorithms - that is, mark a region of a photo as "face", and to say that this region/face is someone - tag it. Only then detection/recog are useful. You don't need fuel until your car has wheels.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Arnd Baecker]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From arnd.baecker web de  2007-06-03 22:27 -------
So obviously I missed this then - there is a fully working code based on OpenCV
to which you supply images (eg. just containing the face part) to train
and then do the recognition for new images? Do you have a pointer?
But what is the role of libface then? Is it needed? Did you manage to
get it working?
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Christian Weiske]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From cweiske cweiske de  2007-06-03 22:53 -------
Arnd, it's my fault. OpenCV is a lib that can be used if one wants to do face detect/recognition, and libface would be the one that could be used by digikam. But if it crashes regularly, it doesn't help. Are there other libs out there that could be used, or even a program via cmdline?
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Christian Weiske]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From cweiske cweiske de  2007-06-03 23:04 -------
http://openbio.sourceforge.net/ is pretty old, but it has the cmdline programs
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From kde colin guthr ie  2007-06-04 01:38 -------
In Reply to Comment #9:
> Your suggestion to describe regions of an image and
> associate those with a tag is indeed something of use
> independent of the face detection.
> This should have many applications (--> separate B.K.O entry?)

Yes this would be very good. I think a new BKO is needed. From my point of view this would help for my Sync plugin (yeah I know I've made 0 progress lately!) when integrating with e.g. Facebook - Tags of regions to represent your friends etc. I'd imagine an extension of the current tags concept to store always a region (defaults to the whole image) of an item. Smaller regions can be defined that represent e.g. faces or other special notes of interest.

I can see how Face Detection could automate the assignment of these people tags.

I wonder however if such a capability could be included as a kipi plugin? What more would the API need to do to enable this. We are always discussing when we'll break the API anyway so perhaps the requirements of face detection should help mould the API changes?

Col
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Christian Weiske]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From cweiske cweiske de  2007-06-04 08:16 -------
I opened a new bug #146337 for region tagging
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From kde colin guthr ie  2007-06-04 12:47 -------
To clarify: when I said:
> I wonder however if such a capability could be included as
> a kipi plugin? What more would the API need to do to enable
> this. We are always discussing when we'll break the API anyway
> so perhaps the requirements of face detection should help mould
> the API changes?

I was referring to the face detection stuff, rather than the region tagging stuff. Should be obvious if you think about it, but wanted to clarify :)
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         
kde colin guthr ie changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kde colin guthr ie
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Christian Weiske]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From cweiske cweiske de  2007-09-26 19:04 -------
Two interesting links for face recognition:
http://www.iis.fraunhofer.de/bf/bv/kognitiv/biom/dd.jsp
(Fraunhofer Institut provides a demo application to detect faces. Perhaps someone could talk to them and convince them to open-source the algorithm? Page is in German)

http://www.riya.com
Is a visual search engine where you can upload your pictures, unfortunately they do not allow/provide offline use.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Christian Weiske]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=146288         




------- Additional Comments From cweiske cweiske de  2007-11-06 17:35 -------
It seems the f-spot code is ready:
http://apart-dev.blogspot.com/
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel