Language models may be able to self-correct biases – if you ask them

The second test used a dataset designed to check the likelihood of a model assuming a person’s gender in a particular profession, and the third tested how much race affected a potential candidate’s chances of being selected. get accepted into a law school if a language model was asked to make the selection, which thankfully doesn’t happen in the real world.

The team found that simply prompting a model to ensure that its responses were not based on stereotypes had an extremely positive effect on its output, particularly in those who had completed enough cycles of RLHF and had more than 22 billion parameters, the variables in an AI System modified during training. (The more parameters, the larger the model. GPT-3 has about 175 million parameters.) In some cases, the model has even started to engage in positive discrimination in its output.

Basically, as with many deep learning works, the researchers don’t really know exactly why the models are able to do this, although they do have some hunches. “As models get larger, they also have larger training datasets, and in those datasets there are a lot of examples of biased or stereotyped behavior,” says Ganguli. “This bias increases with model size.”

But at the same time, somewhere in the training data, there must also be examples of people objecting to this biased behavior, perhaps in response to nasty posts on sites like Reddit or Twitter, for example. . Wherever that weaker signal is coming from, human feedback helps the model amplify it when prompted for an unbiased response, Askell says.

The work raises the obvious question of whether this “self-correction” could and should be built into language models from the outset.