See for yourself how biased AI image models are with these new tools

According to Jernite, one theory as to why it could be that non-binary brown people may have had more exposure in the press recently, which means their images are ending up in the datasets that AI models use to Training.

OpenAI and Stability.AI, the company that built Stable Diffusion, say they have introduced fixes to mitigate biases ingrained in their systems, such as blocking certain prompts that seem likely to generate offensive images. However, these new Hugging Face tools show just how limited those fixes are.

A spokesperson for Stability.AI told us that the company trains its models on “datasets specific to different countries and cultures,” adding that this should “serve to mitigate biases caused by overrepresentation in datasets.” general data”.

An OpenAI spokesperson didn’t comment specifically on the tools, but pointed us to a blog post detailing how the company added various techniques to DALL-E 2 to filter out bias and sexual and violent imagery.

Bias is becoming an increasingly pressing issue as these AI models are increasingly adopted and produce increasingly realistic images. They are already deployed in a multitude of products, such as stock photos. Luccioni says he is concerned that the models risk reinforcing harmful biases on a large scale. She hopes the tools she and her team have created will bring more transparency to image-generating AI systems and highlight the importance of making them less biased.

Part of the problem is that these models are trained on mostly US-centric data, which means they primarily reflect American associations, biases, values ​​and culture, says Aylin Caliskan, associate professor at the University of Washington which studies biases in AI systems. and did not participate in this research.

“What ends up happening is the fingerprinting of this American culture online… which is perpetuated across the world,” Caliskan says.

Caliskan says Hugging Face’s tools will help AI developers better understand and reduce bias in their AI models. “When people see these examples directly, I think they can better understand the significance of these biases,” she says.