Computer engineers at the University of Wisconsin-Madison have developed a new automated dataset that can help pinpoint why facial recognition fails, helping developers improve the accuracy and reliability of these systems.
Facial recognition systems are already a huge part of daily life and are becoming even more common. They safeguard data on our phones, speed us through airport security, and help locate missing persons, among other uses.
Despite their prevalence, these systems are still not fail-proof.
Change your hair color, grow a goatee, get a sunburn or wear a little extra makeup and these digital gatekeepers might struggle to recognize you. Facial recognition developers are not unaware of these problems; what they struggle with is how to pinpoint, understand and fix the types of changing attributes that cause the systems to fail.
That’s why engineers led by Guruprasad Viswanathan Ramesh, a PhD student in electrical and computer engineering, have developed a new automated dataset called CounterFace that can be used to evaluate the performance of facial recognition systems. The UW-Madison team, which also included Kassem Fawaz, an associate professor of electrical and computer engineering, and Ramya Korlakai Vinayak, an assistant professor of electrical and computer engineering, recently presented the project at the ACM Conference on Fairness, Accountability, and Transparency held in Montreal in late June 2026.
To evaluate facial recognition systems, developers usually test them on large datasets of human faces from different identities, but these datasets can’t be used to pinpoint identification failures. A less commonly used alternative is a dataset containing pairs of human faces in which one attribute is changed—for instance, images showing someone with and without glasses. Such datasets allow developers to reason why their systems fail. Gathering or producing the huge sample of paired images needed to test the systems, however, is time- and cost-prohibitive. So researchers use generative AI models to create datasets of paired human faces showing all sorts of variables.
As anyone who has used generative AI knows, however, the models don’t always do exactly what they are prompted. They often change the images—altering positions, features and expressions. And those changes create “confounding attributes” that make the images less useful for facial recognition evaluation. “When you want to change one single attribute in these images, the way these AI models are trained, they end up making a lot more changes, like adding a smile,” says Viswanathan Ramesh.
These raw data sets are considered “noisy” and need some degree of costly, time-intensive inspection and human curation to remove spoiled images and make the dataset useful.
CounterFace does this type of tedious curation automatically. It uses off-the-shelf image generation software to produce pairs of faces with just one altered attribute—adding, for instance, facial hair, glasses or a different skin tone. Then it uses custom detectors to automatically assess these images with strict criteria, rejecting any that display distortions or other confounding factors. This results in a much cleaner dataset, enabling more precise analysis of what attributes give facial recognitions systems trouble.
To validate CounterFace, the team generated 11,821 face pairs with 20 different attributes and eight demographic variables for 160 potential combinations. The most advanced previous datasets used six attributes and six demographics. The researchers then selected 1,583 random image pairs from the dataset for analysis, finding that 96.9% of the time, images retained their identity. They displayed the correct counterfactual or changed attribute—like adding makeup or a hat—in 84% of cases.
The team has used the CounterFace dataset to test the accuracy of four open-source and two commercial facial recognition systems. They found that the systems had some common failure points as well as some individual flaws. The Amazon Web Services system, for example, was able to recognize individuals wearing sunglasses, except for East Asian people, while almost all the other systems could not recognize people wearing sunglasses at all. Face masks also led to significant problems for all of the systems.
Each system also had its own idiosyncrasies, like struggling with specific demographic and attribute combos, like recognizing East Asian males with thick beards, female faces with facial hair, male faces with shoulder length hair, and white males with darker skin tones.
The goal of highlighting these flaws is to help developers more precisely pinpoint the failure modes in their systems and find solutions to correct them. “Based on the data they get from CounterFace, facial recognition deployers can identify where their models can fail, and they can retrain or improve the system for those failing modes,” says Viswanathan Ramesh.
In the future, the researchers say they would like to expand the number of demographic variables and attributes assessed by CounterFace. They also say that advances in image generation technology could improve datasets over time, making them cleaner and even more useful.
While CounterFace and other advancing technologies are making facial recognition even more accurate, reliable and autonomous, Ramesh thinks humans still need to remain in the loop. “In my opinion, you always need to have a human collaborator to support and bookend these facial recognition systems,” he says. “There are always going to be some edge cases where the systems can fail. So you need to have this collaborative mechanism with humans, especially in security-sensitive systems and settings.”
Kassem Fawaz is the Grainger Institute for Engineering Associate Professor. Ramya Korlakai Vinayak is the Dugald C. Jackson Assistant Professor.
Other UW-Madison authors include Ashish Hooda, now at Google DeepMind; Shimaa Ahmed, now at Visa Research; and Harrison Rosenberg, now at ALL3D Inc.
The authors acknowledge support from National Science Foundation awards CNS-1942014 and 2247381, the NSF I-Corps award 2431502, and the Wisconsin Alumni Research Foundation (WARF) Accelerator Program.
Top image caption: PhD student Guruprasad Viswanathan Ramesh led development of CounterFace, an autonomous image dataset that can pinpoint attributes that cause facial recognition to fail.