Face Similarity sort (Nvidia) - MachineEditor/MachineVideoEditor GitHub Wiki

Face Similarity sort is used to sort pictures containing Faces based on how similar they are to each other, which can help in data preparation for DFL use.

An example: Say you have frames that have multiple people in them and extract faces from those frames, but you're only interested in one of the people from frames. You could now use Face Similarity sort to easily remove other people's faces from your set.

note: all the pictures on this page are made using MVE 0.7.9

Open your set of images (faces) in MVE, and under Sort select Face Similarity NVIDIA.
If you do not see Sort options available, first go to Detection Management in the side bar (on the right side of the screen) and switch from Frames to Faces, as shown in the picture below.

Please note that Face Similarity sort is memory intense, and the RAM usage will quickly rise with the number of pictures you want to sort. If your set is big, the recommendation would be to split pictures in sets of up to 20k pictures.

Once your sort finishes, you'll be presented with Frame Group Viewer, like in the picture below.

It is worth noting that for groups with more than two pictures, only the first one and the last one will be shown on this screen. It is possible to view all the pictures in the group by clicking Show Group Faces

Important parts of this interface are:

• Group threshold (marked in red; top-right corner)
• grouped up pictures (one of the groups marked in orange; 12 groups showed in the picture, four per row in 3 rows)
• group options - Delete, Select, and Show Group Faces (marked in yellow; each group has those 3 buttons)
• slider up-down (marked in lime; on the right edge)
• main two options - Close, and Process (marked in blue; bottom-left)

This is the slider with which you control how strict the similarity grouping is. The basics can be explained via the following chart:

At setting 0.5 you will have only a few pictures per group, while at setting 10 you will probably be presented with all the pictures in a single group (which is probably not very useful).

To see an example of increasing the slider (pushing to the right) click on the following link: image example
You can easily see the increase in number of pictures per group, and the significant reduction of the number of groups (no up-down slider in the right picture).

As was previously mentioned, every group has 3 options:

• Delete - marks all the pics from the group for deletion (actual deletion happens when you click Process)
• Select - marks all the pics from the group to be Selected when you click Process and go back to regular MVE view
• Show Group Faces - click this to get a preview of all the pictures in the group, allowing you to decide if it is safe to delete the whole group

Example picture of the preview if I click Show Group Faces on the first group from the previous example:

The two main options are located at the bottom of the interface:

• Close - closes the view without doing anything (you can come back to this view, see below)
• Process - executes deletion of groups you marked with Delete, returns to regular MVE interface, and makes all the pictures from groups marked with Select selected (they get the green border, same as if you mouse clicked them) Be sure that only groups you 100% want to delete are marked with Delete when clicking Process as there is no Undo button.

Once you close the Frame Group Viewer, either by clicking Close or Process you can come back to this screen (without needing to re-run the sort) by clicking Open groups window, which is the rightmost button at the top of the regular interface.
The button in question is highlighted in the following picture:

Short video showing the use of Face Similarity Grouping - recorded in one of the older versions of MVE

---

There have been reports recently (June 2022) of Face Similarity sort not working on fresh MVE installations. All identified cases were identified to be caused by issues with dependencies.

Current workaround is explained on MVE discord, in #support channel, in the following message:
https://discord.com/channels/730623345288151060/730635590185254974/986259726369169449

The solution is to:

1. uninstall current dependencies
2. replace the environment.yml file, with the one the developer upload in the previously linked message
3. install dependencies

UPDATE 23rd of July 2022:
The updated environment.yml is included in MVE 0.8.2 and later. All fresh installations should have Face Similarity working again.

---

Face Similarity sort is based on a VGG Face Descriptor implementation from Keras-VGGFace2-ResNet50. It is available under Creative Commons Attribution License.