This tool could protect artists from AI image generators

The robots would come for the jobs of the humans. It was guaranteed. The general assumption was that they would take over the manual labor, lifting heavy pallets in a warehouse and sorting the recycling. Today, significant advances in generative artificial intelligence mean that robots are coming for artists too. AI-generated images, created with simple text prompts, win art competitions, adorning book coversand the promotion of “The Nutcracker”, leaving human artists worried about their future.

The threat can seem very personal. An image generator called Stable Diffusion has been trained to recognize patterns, styles, and relationships by analyzing billions of images collected from the public Internet, along with text describing their content. Among the images he practiced on were works by Greg Rutkowski, a Polish artist specializing in fantasy scenes featuring dragons and magical beings. Seeing Mr. Rutkowski’s work alongside his name allowed the tool to learn his style efficiently enough that when Stable Diffusion went public last year, his name became shorthand for users who wanted generate dreamy and whimsical images.

A artist noticed that fanciful AI selfies from the viral Lensa app bore ghostly signatures, mimicking what the AI ​​had learned from the data it trained on: artists who do portraits sign their work. “These databases were built without any consent, any permission from the artists,” Rutkowski said. Since the release of the generators, Rutkowski said he has received far fewer requests from newbie authors who need covers for their fantasy novels. Meanwhile, Stability AI, the company behind Stable Diffusion, recently raised $101 million from investors and is now valued at over $1 billion.

“Artists are afraid to publish new works,” said computer science professor Ben Zhao. Putting art online is how many artists advertise their services, but now they’re “afraid to feed this monster that looks more and more like them,” Professor Zhao said. “It ends their business model.”

This led Professor Zhao and a team of computer scientists at the University of Chicago to design a tool called Glaze that aims to prevent AI models from learning a particular artist’s style. To design the tool, which they plan to make available for download, the researchers interviewed more than 1,100 artists and worked closely with Karla Ortiz, a San Francisco-based illustrator and artist.

Suppose, for example, that Ms. Ortiz wants to post new work online, but doesn’t want it passed to the AI ​​to steal. She can upload a digital version of her work to Glaze and choose a different type of art from hers, such as abstract. The tool then makes changes to Ms. Ortiz’s art at the pixel level that Stable Diffusion would associate, for example, with Jackson Pollock’s splattered paint blots. To the human eye, the Glazed image still looks like his work, but the machine-learning model would pick up something quite different. It’s similar to a tool the University of Chicago team previously created to protect photos from facial recognition systems.

When Ms. Ortiz posted her Glazed work online, an image generator trained on those images would not be able to imitate her work. A prompt with his name would instead lead to images in a hybrid style of his works and Pollock.

“We are withdrawing our consent,” Ms. Ortiz said. AI generation tools, many of which charge users a fee to generate images, “have data that doesn’t belong to them,” she said. “These data are my works of art, it’s my life. It looks like my identity.

The University of Chicago team admitted that their tool does not guarantee protection and could lead to countermeasures from anyone who engages in impersonating a particular artist. “We are pragmatists,” Professor Zhao said. “We recognize the likely long delay before law, regulations and policy catch up. It’s to fill that void. »

Many legal experts compare the debate over the unfettered use of artists’ labor for generative AI to the piracy issues of the early Internet with services like Napster that allowed people to consume music without paying. Generative AI companies are already facing a similar deluge of legal challenges. Last month, Ms. Ortiz and two other artists filed a class action lawsuit in California against companies providing art-generating services, including Stability AI, claiming copyright and right-of-publicity violations.

“The allegations in this lawsuit represent a misunderstanding of how generative AI technology works and the law surrounding copyright,” the company said in a statement. Stability AI has also been sued by Getty Images for copying millions of photos without a license. “We are reviewing the materials and will respond accordingly,” a company spokeswoman said.

Jeanne Fromer, a professor of intellectual property law at New York University, said companies might have a strong case for fair use. “How Do Human Artists Learn to Create Art?” said Professor Fromer. “They often copy things and they consume a lot of existing artwork and learn patterns and elements of style and then create new artwork. And so at some level of abstraction, you could say that machines learn to do art in the same way.

At the same time, Professor Fromer said, the purpose of copyright law is to protect and encourage human creativity. “If we care about protecting a profession,” she said, ” or we think creating art is important to who we are as a society, we might want to protect artists. “

A nonprofit called the Concept Art Association recently raised over $200,000 through GoFundMe to hire a lobbying firm to try to persuade Congress to protect artists’ intellectual property. “We’re up against tech giants with unlimited budgets, but we’re confident Congress will recognize that protecting intellectual property is the right side of the argument,” said the association’s founders, Nicole Hendrix. and Rachel Meinerding.

Raymond Ku, professor of copyright at Case Western University, predicted that art creators, rather than simply taking art from the internet, will eventually develop a kind of “private contract system that provides some degree of compensation to the creator”. In other words, artists could be paid a nominal amount when their art is used to train AI and inspire new imagery, similar to how musicians are paid by music streaming companies.

Andy Baio, a writer and technologist who has reviewed training data used by Stable Diffusion, said these services can mimic an artist’s style because they see the artist’s name next to their work over and over again. “You can go and remove names from a dataset,” Baio said, to prevent the AI ​​from explicitly learning an artist’s style.

A service already seems to have done something in this direction. When Stability AI released a new version of Stable Diffusion in November, there was a noticeable change: the “Greg Rutkowsi” prompt no longer worked to get images in his style, a development noted by the company’s managing director, Emad Mostaque.

Stable Diffusion fans were disappointed. “What have you done to Greg,” one wrote on an official Discord forum frequented by Mr Mostaque. He reassured forum users that they could customize the template. “Training on greg won’t be too hard,” another person replied.

Mr. Rutkowski said he plans to start glazing his job.