libkface

[Gilles Caulier-4]

2014-06-29 12:17 GMT+02:00 Tobias Leupold <[hidden email]>:
>> > The detection and recognition code resides in
>> >
>> >         AnnotationDialog/ImagePreview.cpp
>>
>> Do you use multithreading here ?
> No, it's just a procedural approach. At the moment, all scanning is done when
> clicking a button on the image that is currently open and the widget is simply
> disabled until it's done.

You can try the very simple way to fork facedetection or
facerecognition method in a separated thread using :

http://qt-project.org/doc/qt-4.8/qtconcurrentrun.html#run

It's simple to use. Look my code to compute for ex :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/dimg/filters/sharp/sharpenfilter.cpp#L238

here there is a loop to sharpen an image decomposed as egual parts
(depending of number of CPU)

We create QFuture task using QtConcurrent::run(), which call a dedicated method.

We wait that task are complete to process new one, until image is
fully processed.

You can do the same with facedetection from ex, to process a group of images.

You have a list of items, to process to call facedetection method. you
decompose list by number of CPU and fork each item to process with
Qt::Concurent::run, which will run facedetection on each processor.
Image will be analyzed in parallel. Marcel has already lock database
calls, so it safe to play with multithreading with libkface. As
computation with OpenCV take a while, it's not negligible.

digiKam run like this (but it do not use QtConcurrent)

>
>> Do you check kphotoalbum into valgrind to see if memory is leak with
>> libkface ?
> I'll try to do that, but -- to be honest -- I don't really know what the
> valgrind output means ... I still have to learn about this.

valgrind is a run-time memory analyzer. It run program as well in a VM
like and check all memory calls, memory allocations, memory free, and
look if something is bad somewhere. Code must be compiled with debug
symbols.

There are other tools, but runing as static : cppcheck for ex, but
Coverity is the most powerfull, and very used in OpenSource :

https://scan.coverity.com/projects

>
>> All help about API doc can improve usability of libkface in other projects.
> Yeah, that's probably one thing I actually _can_ contribute to libkface.

sure. No need to give a patch, commit directly. We are CC
automatically by mail...

>
>> In KFaceGUI, there are some visual glitches with face rectangle, when
>> you move it for ex. I polished code a lots but not yet found the GUI
>> dysfunction. It's not very important but at least, if we can have a
>> clean implementation here, it will be cool.
> I just started coding C++/Qt/KDE back in April, so I think you're a bit
> smarter than me, but I'll have a look at it and do my very best. If I find
> something, I'll send a patch.

No problem

>
>> Don't forget that you must use current implementation of libkface for
>> your tests, where code is the most hacked/tested for the moment.
> At the moment, I use the libkface version shipped with Digikam 4.0.0, but I'll
> try to write an ebuild (I'm on Gentoo) installing the CVS version so that I
> can check it against the latest code.

No. use git/master code where i spare a lots if time (a least 3 weeks
of work to review code again and again, and again), to talk with
MArcel about all technical details, to check all bugzilla entries and
fix, consolidate and verify.

Gilles Caulier
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

Hi Gilles, hi list!

working on my face recognition implementation, I have a question concerning
database maintenance.

If I understand it correctly, I can change identities via the
setIdentityAttributes function and wipe an identity's training data with
clearTraining (or all training data with clearAllTraining).

But what about deleting an identity? When an identity's training data is
deleted, the identity as such still resides in the database!

Yours, Tobias
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

Hi Tobias,

This is a good question that i don't yet review from API.

Perhaps Marcel know already the response...

Gilles Caulier

2014-07-02 18:39 GMT+02:00 Tobias Leupold <[hidden email]>:

> Hi Gilles, hi list!
>
> working on my face recognition implementation, I have a question concerning
> database maintenance.
>
> If I understand it correctly, I can change identities via the
> setIdentityAttributes function and wipe an identity's training data with
> clearTraining (or all training data with clearAllTraining).
>
> But what about deleting an identity? When an identity's training data is
> deleted, the identity as such still resides in the database!
>
> Yours, Tobias
> _______________________________________________
> Digikam-devel mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-devel

_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

Actually, we have void TrainingDB::deleteIdentity(int id) in
database/trainingdb.cpp, but it's never used anywhere ... so perhaps, the
class simply lacks this functionality at the moment.

I'll see if I can add a function using this to recognitiondatabase.cpp ...
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

Probably, this is not that easy. Even when adding a function calling
TrainingDB::deleteIdentity (see the attached patch) and passing some identity
to it, the database seems to remain unchanged.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

I was a bit hasty, the patch actually does work like that! I just had the old
database still instanciated and not re-loaded, so the deleted Identity still
showed up. But it has been deleted from the database by
RecognitionDatabase::deleteIdentity!
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

The identityCache variable also has to be updated so that the deleted identity
does not show up after deletion (see the fixed attached patch)
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

> Actually, we have void TrainingDB::deleteIdentity(int id) in
> database/trainingdb.cpp, but it's never used anywhere ... so perhaps, the
> class simply lacks this functionality at the moment.
>
> I'll see if I can add a function using this to recognitiondatabase.cpp ...

Yes it has never been implemented in the API.

We should add some notes of caution that the database is shared between
applications and identity management should be aware of this.

In the age of Nepomuk, there were plans to have some KDE-wide library to
provide management of real people (linked with email, contacts, facebook etc.)
I dont know but assume it died together with Nepomuk.

Marcel
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

> Yes it has never been implemented in the API.
Could you have a look at the patch I just sent? It adds the very functionality
I was looking for (at least I hope so ...). Implementing something using this
is of course only meaningful if this function will be provided by libkface and
not only by my local git branch ;-)
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

The patch sound fine for me.

You forget to add API doc in header using Doxygen syntax.

I suppose that you have already an acount to git as developer and you
are able to commit this patch to libkface repository. Right ?

If yes, it's fine for me...

Gilles Caulier

2014-07-02 22:07 GMT+02:00 Tobias Leupold <[hidden email]>:
>> Yes it has never been implemented in the API.
> Could you have a look at the patch I just sent? It adds the very functionality
> I was looking for (at least I hope so ...). Implementing something using this
> is of course only meaningful if this function will be provided by libkface and
> not only by my local git branch ;-)
> _______________________________________________
> Digikam-devel mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-devel
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

Hi :-)

> The patch sound fine for me.
Nice :-)

> You forget to add API doc in header using Doxygen syntax.
I just added comments to the header file, also some more comments to clarify
what the clearTraining and clearAllTraining functions do.

> I suppose that you have already an acount to git as developer and you
> are able to commit this patch to libkface repository. Right ?
No, I don't have a developer account. I'm quite new to all this ;-)

I just created a libkface clone on github and pushed my changes to a branch,
see
https://github.com/l3u/libkface/tree/delete_identity

Perhaps, you want to commit it.

Another question:

What about a convenience function for training the recognition database to
make using libkface simpler? What I was looking for at my very start was a
function I give some user ID and a picture to learn.

What has to be done at the moment is:

* Query the database for an existing identity and creating one if none is
found.

* Instantiate a SimpleTrainingDataProvider with the identity I want to train
and the respective image, which apparently holds nothing but the identity and
the image I want to add

* Pass the identity and the SimpleTrainingDataProvider instance to the train
function (although the Identity I want to train has already been passed to the
SimpleTrainingDataProvider!)

What about the following: We could integrate the SimpleTrainingDataProvider
code in the lib (you wrote it anyways and provide it in the test dir ;-) and
create a function like

void train(const Identity& identityToBeTrained,
           const QImage& face,
           const QString& trainingContext);

which creates a SimpleTrainingDataProvider with the given identity and the
given image and then passes the data the "real" train function.

Querying the database for a probably already existing identity is okay I
think, as we don't know in which way the user wants to search for the
identity.

But for me as an "end-user" developer outside of digikam, it was not really
clear why I have to mess with that SimpleTrainingDataProvider (or a custom
class providing the same data) when I want to train the database with one
image for one identity in a procedural way ... probably, this is fine if one
wants to do something "special" or train a lot of images or such. But I think
it makes the use of libkface a bit more complicated as necessary.

It would not be much work to implement this convenience function, I think, and
I also think I can do this. If you also think that would be nice, of course.

What do you think?

Yours, Tobias
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

> What about the following: We could integrate the SimpleTrainingDataProvider
> code in the lib (you wrote it anyways and provide it in the test dir ;-) and
> create a function like
>
> void train(const Identity& identityToBeTrained,
>            const QImage& face,
>            const QString& trainingContext);

Cf:
https://github.com/l3u/libkface/tree/simple_training

_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

> > I suppose that you have already an acount to git as developer and you
> > are able to commit this patch to libkface repository. Right ?
>
> No, I don't have a developer account. I'm quite new to all this ;-)

I suppose kphotoalbum is under the KDE umbrella and hosted on git.kde.org, so
you will need an account? Anyway, get one.
We should review patches of course, but it's much easier if you have your
account and commit your patches.


>
> What about a convenience function for training the recognition database to
> make using libkface simpler? What I was looking for at my very start was a
> function I give some user ID and a picture to learn.

> void train(const Identity& identityToBeTrained,
>            const QImage& face,
>            const QString& trainingContext);

TThe current recognition algorithm fortunately supports incremental training.
There were earlier attempts which needed all images of an identity for a
training. The current API is prepared to support this, though it would break
with multiple-app support and we may never need it.

In any case, we cannot make the assumption that training image-by-image will
be efficient. The "provider" allows the algorithm to pull what it needs, and
allows the application to provide data on-demand.

If you add the above method, such hints will need to be added to the
documentation.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Martin Klapetek]

On Wed, Jul 2, 2014 at 10:03 PM, Marcel Wiesweg <[hidden email]> wrote:

> Actually, we have void TrainingDB::deleteIdentity(int id) in
> database/trainingdb.cpp, but it's never used anywhere ... so perhaps, the
> class simply lacks this functionality at the moment.
>
> I'll see if I can add a function using this to recognitiondatabase.cpp ...

Yes it has never been implemented in the API.

We should add some notes of caution that the database is shared between
applications and identity management should be aware of this.

In the age of Nepomuk, there were plans to have some KDE-wide library to
provide management of real people (linked with email, contacts, facebook etc.)
I dont know but assume it died together with Nepomuk.

Not at all, that library is alive and kicking! ^_- It's called KPeople (kde:libkpeople) and currently it's being used as a backend for KDE Telepathy and there's an ongoing GSoC to create a full featured address book based on it.

Currently it's plugin based and as a storage backend for the links it uses custom SQLite database with the idea being that each application is doing its own storage and just provides its data via that plugin.

At this point we have plugins for KTp (instant messaging) and Akonadi (any contacts stored in Akonadi). Should the face tagging/recognition be interfaced with address book, KPeople can sure be used, but it uses parts of KDEPIM-Libs and would add such dependency on digiKam.

Let me know if there's an interest in doing that --> CC me directly, I'm not following all of digiKam's mails ;)

Cheers

--

Martin Klapetek | KDE Developer

_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

> so you will need an account? Anyway, get one.
> We should review patches of course, but it's much easier if you have your
> account and commit your patches.
I'll try to do so ;-)

> If you add the above method, such hints will need to be added to the
> documentation.
I think it's not really up to me to add the method ;-)

I'm aware that adding the function I proposed at
https://github.com/l3u/libkface/tree/simple_training will not be the best way
to train the database, but it will be the easiest way. And -- speaking of my
current implementation for KPA -- it is what I need (or wished libkface
provided by default) at the moment.

So, of course, we should add a comment to the API that this is only the "easy"
way to do some training and one should implement an own data provider. But at
the moment, the docs are quite minimalistic and a simple ready-to-use function
to train the database will help newbies (like me) to get into libkface, won't
it?

A more versatile and elaborate way to do recognition training and database
maintenance can and will be implemented if it's needed I think.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

Tobias,

As Marcel suggest, i recommend to take an account as developer to be
able to commit your patches. This will be more easy to review and
work.

Look here for details :
http://techbase.kde.org/Contribute/Get_a_Contributor_Account

Gilles

2014-07-04 0:37 GMT+02:00 Tobias Leupold <[hidden email]>:

>> so you will need an account? Anyway, get one.
>> We should review patches of course, but it's much easier if you have your
>> account and commit your patches.
> I'll try to do so ;-)
>
>> If you add the above method, such hints will need to be added to the
>> documentation.
> I think it's not really up to me to add the method ;-)
>
> I'm aware that adding the function I proposed at
> https://github.com/l3u/libkface/tree/simple_training will not be the best way
> to train the database, but it will be the easiest way. And -- speaking of my
> current implementation for KPA -- it is what I need (or wished libkface
> provided by default) at the moment.
>
> So, of course, we should add a comment to the API that this is only the "easy"
> way to do some training and one should implement an own data provider. But at
> the moment, the docs are quite minimalistic and a simple ready-to-use function
> to train the database will help newbies (like me) to get into libkface, won't
> it?
>
> A more versatile and elaborate way to do recognition training and database
> maintenance can and will be implemented if it's needed I think.
> _______________________________________________
> Digikam-devel mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-devel

_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Tobias Leupold]

Hi Gilles, hi Marcel,

thanks for approving my developer account :-) Anyways, I would not "just" push
any changes, esp. if they concern the API. So would you please again have a
look at

    https://github.com/l3u/libkface/compare/delete_identity

(I think Gilles already said that was fine) and

    https://github.com/l3u/libkface/compare/simple_training

and tell me if you are okay with that? Thanks in advance!

Yours, Tobias
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel