The viral AI avatar app Lensa stripped me naked, without my consent

Stability.AI, the company that developed Stable Diffusion, launched a new version of the AI ​​model at the end of November. A spokesperson says the original model was released with a security filter, which Lensa doesn’t appear to have used because it would remove those outputs. One way Stable Diffusion 2.0 filters content is to remove images that repeat often. The more something is repeated, like Asian women in sexually graphic scenes, the stronger the association becomes in the AI ​​model.

Caliskan studied CLIP (Contrastive Language Image Pretraining), which is a system that helps Stable Diffusion generate images. CLIP learns to match images from a data set to descriptive text prompts. Caliskan found it to be full of problematic gender and racial biases.

“Women are associated with sexual content, while men are associated with professional career-related content in any important field like medicine, science, business, etc.,” Caliskan explains.

Oddly enough, my Lensa avatars looked more realistic when my photos went through male content filters. I have avatars of me wearing clothes (!) and in neutral poses. In several images, I was wearing a white coat that looked like it belonged either to a chef or a doctor.

But it’s not just the workout data that’s to blame. The companies developing these models and apps are making active choices about how they use the data, says Ryan Steed, a doctoral student at Carnegie Mellon University who has studied biases in image-generating algorithms.

“Someone has to choose the training data, decide to build the model, decide to take certain steps to mitigate or not to mitigate those biases,” he says.

The app developers have chosen for the male avatars to appear in space suits, while the female avatars are given cosmic thongs and fairy wings.

A spokesperson for Prisma Labs says the “sporadic sexualization” of photos happens to people of all genders, but in different ways.