Uber facial recognition blocks Indian drivers’ access to their accounts

Uber checks that a driver’s face matches what the company has on file through a program called “Real-Time Identity Verification.” It was rolled out in the United States in 2016, in India in 2017, and later in other markets. “This prevents fraud and protects drivers’ accounts from being compromised. It also protects riders by creating another layer of accountability in the app to make sure the right person is driving,” Joe Sullivan, Uber’s chief security officer, said in a 2017 statement.

But the company’s driver vetting procedures are far from transparent. Adnan Taqi, an Uber driver in Mumbai, got in trouble with her when the app tricked him into taking a selfie at dusk. He was locked out for 48 hours, a big dent in his work schedule – he says he drives 18 hours straight, sometimes up to 24 hours, so he can make a living. A few days later, he took a selfie that again banned him from his account, this time for an entire week. This time, Taqi suspects, it was a matter of hair: “I hadn’t shaved for a few days and my hair had also grown out a bit,” he says.

More than a dozen drivers interviewed for this story detailed examples of having to find better lighting to avoid being locked out of their Uber accounts. “Whenever Uber asks for a selfie in the evening or at night, I have to stop and go under a lamp post to click a clear image, otherwise there is a chance of being rejected,” said Santosh Kumar, a Uber driver from Hyderabad. .

Others have struggled with scratches on their budget cameras and smartphones. The problem is not unique to Uber. Pilots with Ola, which is backed by SoftBank, face similar issues.

Some of these difficulties can be explained by the natural limitations of facial recognition technology. The software starts by converting your face into a set of points, says Jernej Kavka, an independent technology consultant with access to Microsoft’s Face API, which is what Uber uses to power real-time identity verification.

“With excessive facial hair, the points change and he may not recognize where the chin is,” Kavka says. The same thing happens when the lighting is poor or the phone camera does not have good contrast. “This makes it difficult for the computer to detect edges,” he explains.

But the software can be particularly fragile in India. In December 2021, technology policy researchers Smriti Parsheera (CyberBRICS project fellow) and Gaurav Jain (economist at the International Finance Corporation) published a preprint paper that audited four commercial face processing tools – Amazon’s Rekognition, Microsoft Azure’s Face, Face++, and FaceX—for their performance on Indian faces. When the software was applied to a database of 32,184 election candidates, Microsoft’s Face failed to even detect the presence of a face in more than 1,000 images, resulting in an error rate over 3%, the worst of the four.

The Uber app may be failing drivers because its software hasn’t been trained on a wide range of Indian faces, Parsheera says. But she says there may also be other issues at play. “There could be a number of other contributing factors like lighting, angle, aging effects, etc.”, a- she explained in writing. “But the lack of transparency surrounding the use of such systems makes it difficult to provide a more concrete explanation.”