iTC Meeting Minutes 2023 03 21 - biometricITC/cPP-biometrics GitHub Wiki
Call started at 10:03am EDT
-
Brian Wood
-
Greg Ott
-
Clare Parran
-
Quang Huy Nguyen
-
Greg Fiumara
-
Jim Arnold
-
Scott Chapman
-
Naruki Kai
-
Stephanie Schuckers (presenting FIDO information)
The call started with a review of the task list. There were no updates at this time.
Stephanie Schuckers then presented an overview of the FIDO Alliance Biometric Testing and Certification program (link is in CCUF OnlyOffice, contact Brian if you are unable to access the file). The notes here will not recount the presentation but the key points and discussions from the presentation.
Slide 3 of the presentation shows how the FIDO 3.0 specs have been changed to provide a mode PAD-focused evaluation. In this evaluation the FAR testing requires a much smaller number of users and provides more of a "sanity check" on the claimed values than a full test (to the minimum 1:10K FAR that is required for FIDO certification). This is intended as a lower cost evaluation option since the FAR testing requires a minimum of 123 people vs 25 for the low cost option. The rebalances the test focus on the PAD, and requires the vendor to provide FAR testing evidence (instead of tested proof).
The table on slide 4 provides more detailed information about the testing. Level 1 and Level 2 mainly differ in the IAPAR where Level 1 allows a 15% failure rate vs 7% failure rate. The testing is common across both levels, it is only the IAPAR results that have different expectations.
The IAPAR testing is considered to be black box; there is no expectation that the vendor provides results of the PAD results and the match results separately. So each PAI should be checked out-of-band to see that it is a good match before using it to test. This could be a visual inspection, testing on another type of sensor, etc, but as far as the tested system is concerned, a failure can be from any part of the system as the internal details are not reported.
In an evaluation there are a minimum of 14 PAI species that would be used (for each subject). While these are shown to be 6 Level A and 8 Level B, it is really 6 Level A, 4 Level B and then an additional 4 "unknown" derived from Level B. The lab uses the stock examples to pick 6 from Level A and the 4 from Level B. Then, from their review of the device, they create and additional 4 PAI types (based on the examples currently in use, generally from Level B) and then add this to the test suite. The vendor does not know what these will be before the evaluation test plan is created. This is similar to the ATE_IND + AVA_VAN testing we have where the lab can create some tweaked PAI beyond those specified in the toolboxes.
There were some questions about how FIDO determined the attack potential for their PAI. Generally the main point was to ensure that anything they thought was critical to be tested was covered in Level A or B while Level C is mainly reserved for more difficult or future types of attacks. Brian pointed out that the division between the current iTC toolboxes doesn’t exactly match up, but generally the same point is accomplished. It was agreed that over time the species types may be adjusted based on the threats that are seen in the wild. One area that was brought up for future consideration was the use of deep fakes. As video is already listed as a species, this was thought to be covered, but the specifics for creating a deep fake are not and may need to be added.
Stephanie stated that at this time FIDO does not have any toolbox recipes that could be followed for lab testing but relies on the labs to present their test plans to the secretariat for approval. The goal is to ensure commonality there until such toolbox recipes can be written. This is a potential place of collaboration between the iTC and FIDO.
There was a question about how FIDO confirms that the labs it uses can do the testing since most of the normal CC labs tend to not have much experience with biometrics testing. She stated that there is a program for adding labs (which can be found on the Accredited Biometrics Labs page). This shows the requirements a lab must meet before it can perform testing for FIDO.
Brian stated then that the iTC is working on a PADv2 which would be kept private since it would encompass more difficult attacks than what can be found readily on the internet (and not requiring much skill). This is another potential area of collaboration as we work to define the recipes and requirements around them.
Stephanie also said she would be interested in participating as we work on PADv2 as a researcher (depending on timing and availability).
The call ended at 10:53am EDT.