Automatic Assignment of Tag Icons

[kartikx]

Hello all,

I've implemented automatic addition and removal of icons to face tags. Changes have been made to utilities/facemanagement and app/views/stack.

This automatic addition/removal will happen only on confirmed tags and not on unknown/unconfirmed/ignored.
Desired Functionality:
1. Whenever the user confirms the face to a tag (either pre-existing or just going to be created), if the tag does not have a tag icon, then the face just confirmed will be added as the tag icon. This icon remains the tag icon for all further confirmations to this tag, i.e. the tag icon does not change again and again.
2. When the user deletes a face from a tag (either using Rejection Overlay (i.e. the Red Cross Button) or using Remove Tag Context Menu), if after deletion the tag has no faces associated to it, the tag icon will get reset.

The implementation is such that the changes are TAlbum specific and not TagModel, hence even if we have a very large number of TAlbums(tags) performance should not go down. Furthermore, changes only happen once so repeated assignments aren't carried out.

Note on implementation : Changes to TAlbum icons require AlbumManager methods, which can only be used in files that are compiled in gui_digikam. Hence, I've made changes in upper level files (which are compiled with gui_digikam) and not the lower level database methods (where the actual confirm/reject happens). I have used QTimer at places, to ensure the confirm/reject operations (in the lower level) are carried out, before setting/removing icons.

One Issue that I notice is that since we're assigning faces as the Icon, the size of the Icon depends on the size of the Face (which was cropped by Face Detector). Hence, icon sizes are unequal. I will try to fix this.

Regards
Kartik

[Gilles Caulier-4]

Hi Kartik,

Thanks for your great job. You progress sounds interesting. I plan to
test your branch in the near future when time permits.

Usually, KDE core team asks students to written a blog about the
project advancements. This permit to follow the progress and to link
blogs entries in grouped announcements in dot.kde.org.

Typically in your case, this will include your mails in this mailing
list about your project.

A very good exemple is the Thanh blog post about the GoSC 2019 with
the DNN face engine :

https://community.kde.org/GSoC/2019/StatusReports/ThanhTrungDinh

I recommend doing the same... This will permit also the end users to
know the changes introduced with your project.

My best regards

Gilles Caulier


Le sam. 6 juin 2020 à 06:19, Kartik Ramesh <[hidden email]> a écrit :

>
> Hello all,
>
> I've implemented automatic addition and removal of icons to face tags. Changes have been made to utilities/facemanagement and app/views/stack.
>
> This automatic addition/removal will happen only on confirmed tags and not on unknown/unconfirmed/ignored.
> Desired Functionality:
> 1. Whenever the user confirms the face to a tag (either pre-existing or just going to be created), if the tag does not have a tag icon, then the face just confirmed will be added as the tag icon. This icon remains the tag icon for all further confirmations to this tag, i.e. the tag icon does not change again and again.
> 2. When the user deletes a face from a tag (either using Rejection Overlay (i.e. the Red Cross Button) or using Remove Tag Context Menu), if after deletion the tag has no faces associated to it, the tag icon will get reset.
>
> The implementation is such that the changes are TAlbum specific and not TagModel, hence even if we have a very large number of TAlbums(tags) performance should not go down. Furthermore, changes only happen once so repeated assignments aren't carried out.
>
> Note on implementation : Changes to TAlbum icons require AlbumManager methods, which can only be used in files that are compiled in gui_digikam. Hence, I've made changes in upper level files (which are compiled with gui_digikam) and not the lower level database methods (where the actual confirm/reject happens). I have used QTimer at places, to ensure the confirm/reject operations (in the lower level) are carried out, before setting/removing icons.
>
> One Issue that I notice is that since we're assigning faces as the Icon, the size of the Icon depends on the size of the Face (which was cropped by Face Detector). Hence, icon sizes are unequal. I will try to fix this.
>
> Regards
> Kartik

[kartikx]

Thank you for your kind words, Gilles.

I have already been writing about Project progress here. However it's not as detailed as Thanh's blogposts that you shared.

I've also been asked by KDE to publish Status Reports on the community.kde.org website, so I'll try to make my posts as detailed as possible there.

Regards

Kartik.

On Sat, Jun 6, 2020 at 2:46 PM Gilles Caulier <[hidden email]> wrote:

Hi Kartik,

Thanks for your great job. You progress sounds interesting. I plan to
test your branch in the near future when time permits.

Usually, KDE core team asks students to written a blog about the
project advancements. This permit to follow the progress and to link
blogs entries in grouped announcements in dot.kde.org.

Typically in your case, this will include your mails in this mailing
list about your project.

A very good exemple is the Thanh blog post about the GoSC 2019 with
the DNN face engine :

https://community.kde.org/GSoC/2019/StatusReports/ThanhTrungDinh

I recommend doing the same... This will permit also the end users to
know the changes introduced with your project.

My best regards

Gilles Caulier


Le sam. 6 juin 2020 à 06:19, Kartik Ramesh <[hidden email]> a écrit :
>
> Hello all,
>
> I've implemented automatic addition and removal of icons to face tags. Changes have been made to utilities/facemanagement and app/views/stack.
>
> This automatic addition/removal will happen only on confirmed tags and not on unknown/unconfirmed/ignored.
> Desired Functionality:
> 1. Whenever the user confirms the face to a tag (either pre-existing or just going to be created), if the tag does not have a tag icon, then the face just confirmed will be added as the tag icon. This icon remains the tag icon for all further confirmations to this tag, i.e. the tag icon does not change again and again.
> 2. When the user deletes a face from a tag (either using Rejection Overlay (i.e. the Red Cross Button) or using Remove Tag Context Menu), if after deletion the tag has no faces associated to it, the tag icon will get reset.
>
> The implementation is such that the changes are TAlbum specific and not TagModel, hence even if we have a very large number of TAlbums(tags) performance should not go down. Furthermore, changes only happen once so repeated assignments aren't carried out.
>
> Note on implementation : Changes to TAlbum icons require AlbumManager methods, which can only be used in files that are compiled in gui_digikam. Hence, I've made changes in upper level files (which are compiled with gui_digikam) and not the lower level database methods (where the actual confirm/reject happens). I have used QTimer at places, to ensure the confirm/reject operations (in the lower level) are carried out, before setting/removing icons.
>
> One Issue that I notice is that since we're assigning faces as the Icon, the size of the Icon depends on the size of the Face (which was cropped by Face Detector). Hence, icon sizes are unequal. I will try to fix this.
>
> Regards
> Kartik

[Gilles Caulier-4]

Hi Kartik,

Excellent. I'm waiting your detailles blog post to inform also end
users in digikam-users ML where some people want to know the current
works done on *Faces* features while this summer....

Nghia,

Do you plan to write a detailed blog post as Kartik about your project ?

Best
gilles Caulier

Le dim. 7 juin 2020 à 09:38, Kartik Ramesh <[hidden email]> a écrit :

>
> Thank you for your kind words, Gilles.
>
> I have already been writing about Project progress here. However it's not as detailed as Thanh's blogposts that you shared.
> I've also been asked by KDE to publish Status Reports on the community.kde.org website, so I'll try to make my posts as detailed as possible there.
>
> Regards
> Kartik.
>
> On Sat, Jun 6, 2020 at 2:46 PM Gilles Caulier <[hidden email]> wrote:
>>
>> Hi Kartik,
>>
>> Thanks for your great job. You progress sounds interesting. I plan to
>> test your branch in the near future when time permits.
>>
>> Usually, KDE core team asks students to written a blog about the
>> project advancements. This permit to follow the progress and to link
>> blogs entries in grouped announcements in dot.kde.org.
>>
>> Typically in your case, this will include your mails in this mailing
>> list about your project.
>>
>> A very good exemple is the Thanh blog post about the GoSC 2019 with
>> the DNN face engine :
>>
>> https://community.kde.org/GSoC/2019/StatusReports/ThanhTrungDinh
>>
>> I recommend doing the same... This will permit also the end users to
>> know the changes introduced with your project.
>>
>> My best regards
>>
>> Gilles Caulier
>>
>>
>> Le sam. 6 juin 2020 à 06:19, Kartik Ramesh <[hidden email]> a écrit :
>> >
>> > Hello all,
>> >
>> > I've implemented automatic addition and removal of icons to face tags. Changes have been made to utilities/facemanagement and app/views/stack.
>> >
>> > This automatic addition/removal will happen only on confirmed tags and not on unknown/unconfirmed/ignored.
>> > Desired Functionality:
>> > 1. Whenever the user confirms the face to a tag (either pre-existing or just going to be created), if the tag does not have a tag icon, then the face just confirmed will be added as the tag icon. This icon remains the tag icon for all further confirmations to this tag, i.e. the tag icon does not change again and again.
>> > 2. When the user deletes a face from a tag (either using Rejection Overlay (i.e. the Red Cross Button) or using Remove Tag Context Menu), if after deletion the tag has no faces associated to it, the tag icon will get reset.
>> >
>> > The implementation is such that the changes are TAlbum specific and not TagModel, hence even if we have a very large number of TAlbums(tags) performance should not go down. Furthermore, changes only happen once so repeated assignments aren't carried out.
>> >
>> > Note on implementation : Changes to TAlbum icons require AlbumManager methods, which can only be used in files that are compiled in gui_digikam. Hence, I've made changes in upper level files (which are compiled with gui_digikam) and not the lower level database methods (where the actual confirm/reject happens). I have used QTimer at places, to ensure the confirm/reject operations (in the lower level) are carried out, before setting/removing icons.
>> >
>> > One Issue that I notice is that since we're assigning faces as the Icon, the size of the Icon depends on the size of the Face (which was cropped by Face Detector). Hence, icon sizes are unequal. I will try to fix this.
>> >
>> > Regards
>> > Kartik

[nghiaduong]

Hello Gilles,

I am planning to, but I am not familiar with the syntax of Atom blog so it takes me a little much longer to finish. Furthermore, I am at the end of the semester, where many school projects due in the next 2 weeks, therefore there is not much time left for me to fill the report. However, I keep track of the progress in my project journal and I will publish it with the blog post.

A quick resumé of my progress. I have isolated the code of facial recognition with the rest of the faces engine, in order to test and evaluate this module alone. I implement many methods of label assignment and test them with YaleB extended dataset, with over 11000 face images. By far I have reimplemented L2 distance, cosine distance, and mean cosine distance (currently used by faces engine) comparison, Support vector machine, and K Nearest neighbor for face classification.

The mean cosine distance currently used by faces engine performs well on small datasets. However, when the data spread, say when there are many photos of a same face, the accuracy of this method falls to 0%. Currently, the most promising methods are the closest L2 distance, Cosine distance, and K-nearest neighbor. Both give an accuracy of around 75% on YaleB extended dataset and 70% on Lfw dataset. I follow the instructions given by OpenFace project but it seems to be stuck around this limit.

As a suggestion of Thanh, I am trying to apply UMAP method to find the general structure of face embedding vectors and plot it for debugging and clustering. However, if the neural network forward produces too many outliners, which is incorrect according to the paper, clustering could be very difficult to give a better result.

Nghia