Customs and Border Protection will be advancing its facial recognition technology into a new algorithm next month while maintaining that software used by the Department of Homeland Security is not reflecting racial or gender bias.

Since CBP is using an algorithm from one of the highest-performing vendors identified in the report, we are confident that our results are corroborated with the findings of this report, Office of Field Operations Deputy Executive Assistant Commissioner John Wagner told the House Homeland Security Committee at a Thursday hearing on the National Institute of Standards and Technologys recent report studying facial recognition hits and misses. More specifically, the report indicates the highest performing algorithms had minimal to undetectable levels of demographic-based error rates.

Tests in the December report,Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects, showed a wide range in accuracy across developers, with the most accurate algorithms producing many fewer errors. In one-to-one matching, NIST reported higher rates of false positives for Asian- and African-American faces, notably African-American females, compared to Caucasians; elderly and young subjects were also more prone to false positives. U.S.-developed algorithms yielded more high rates of false positives in one-to-one matching for Asians, African-Americans and native groups, while some algorithms developed in Asian countries showed no such dramatic difference in false positives in comparing Asian and Caucasian faces.

NIST tested 189 face recognition algorithms from 99 developers using four collections of photographs with 18.27 million images of 8.49 million people, using images provided by the State Department, DHS and FBI.

The report also highlights some of the operational variables that impact error rates such as gallery size, photo age, photo quality, numbers of photos of each subject in the gallery, camera quality, lighting, human behavior factors all influence the accuracy of an algorithm, Wagner noted to the committee. That is why CBP is carefully constructed the operational variables in the deployment of the technology to ensure we can attain the highest levels of match rates, which remain in the 97 percent to 98 percent range.

Wagner noted that NIST did not test the specific CBP operational construct to measure the additional impact these variables may have, which is why we have recently entered into an MOU with NIST to evaluate our specific data.

Use of facial comparison technologies, he stressed, simply automates a process that is often done manually today and using facial comparison technology to date CBP has identified 252 imposters. to include people using 75 genuine U.S. travel documents.

We have met three times with representatives of the privacy advocacy community as well as discussions with the privacy and civil liberties oversight board and the DHS privacy and integrity advisory committee, Wagner said. In November, CBP submitted to the Office of Management and Budget a rulemaking that would solicit public comments on the proposed regulatory updates and amendments to the federal regulations.

Peter Mina, deputy officer for programs and compliance at DHS Office for Civil Rights and Civil Liberties, told lawmakers that his office has been and continues to be engaged with the DHS operational components to ensure use of facial recognition technology is consistent with civil rights and civil liberties law and policy.

Second, operators, researchers and civil rights policymakers must work together to prevent algorithms from leading to impermissible biases in the use of facial recognition technology and, third, facial recognition technology can serve as an important tool to increase the efficiency and effectiveness of the departments public protection mission as well as the facilitation of lawful travel, but it is vital that these programs utilize technology in a way that safeguards our constitutional rights and values, Mina said.

Mina said the civil rights office recognizes the potential risk of impermissible bias in facial recognition algorithms and has regularly engaged CBP on the implementation of facial recognition technology in its biometric entry and exit program, including advising on policy and implementation of appropriate accommodations for individuals wearing religious headwear, for individuals with a sincere religious objection to being photographed and for individuals who may have a significant injury or disability and for whom taking photographs may present challenges or not be possible.

Mina said the office has received one complaint regarding DHS use of facial recognition technology but we have not seen a trend and that is when we would actually, in fact, open an investigation in this matter.

NIST Information Technology Laboratory Director Chuck Romine testified that the false-positive differentials in one-to-one matching are much larger than those related to false negative and exist across many of the algorithms tested and said the racial false-positive rates could be attributed to the relationship between an algorithms performance and the data used to train the algorithm itself.

The impact of errors is application dependent. False positives in one-to-many search are particularly important because the consequences could include false accusations. For most algorithms, the NIST study measured higher false-positive rates in women, African-Americans, and particularly in African-American women, Romine said. However, the study found that some one-to-many algorithms gave similar false-positive rates across these specific demographics. Some of the most accurate algorithms fell into this group. This last point underscores one overall message of the report: algorithms perform differently.

Romine told lawmakers that they did test NEC, the brand used by CBP, but we have no independent way to correlate whether those are the identical algorithms that are being used in the field.

CBP is using an earlier version of NEC right now, Wagner noted, and I believe we are testing NEC-3, which is the version that was tested. And the plan is to use it next month in March, to switch over to upgrade, basically to that one.

Wagner stressed that the 2 to 3 percent failure-to-match rate with CBP software does not mean that the person is misidentified, but that we didnt match them to a picture in the gallery that we did have of them, so we should have matched them.

It should be at zero, he said. And that is where we look at the operational variables, the camera, the picture quality, the human behaviors when the photo was taken, the lighting, those different types, and then the age of the photo.

Wagner said there may be a small handful of false positives; Im just not aware of any, but as we built this and tested it we are just not seeing that.

(Visited 200 times, 90 visits today)

The rest is here:
CBP Moving Forward with Algorithm Update, MOU with NIST on Facial Recognition Evaluation - HSToday