How OpenAI tries to make ChatGPT safer and less biased

It’s not just about freaking out journalists (some of whom really should know better than to anthropomorphize and exaggerate a dumb chatbot’s ability to have feelings.) The startup has also received a lot of heat from the from conservatives in the US who claim its ChatGPT chatbot has a “woke” bias.

All this outrage is finally having an impact. Bing’s trippy content is powered by AI language technology called ChatGPT developed by startup OpenAI, and last Friday OpenAI released a blog post aimed at clarifying the behavior of its chatbots. He also posted his guidelines on how ChatGPT should react when asked to talk about “culture wars” in the United States. The rules include not affiliating with political parties or judging a group as good or bad, for example.

I spoke to Sandhini Agarwal and Lama Ahmad, two AI policy researchers at OpenAI, on how the company is making ChatGPT safer and less crazy. The company declined to comment on its relationship with Microsoft, but it had some interesting insights nonetheless. Here’s what they had to say:

How to get better answers: In research on AI language models, one of the biggest open questions is how to prevent models from “hallucinating,” a polite term for making things up. ChatGPT has been used by millions of people for months, but we haven’t seen the kind of lies and hallucinations that Bing generates.

This is because OpenAI used a technique in ChatGPT called reinforcement learning from human feedback, which improves model responses based on user feedback. The technique works by asking people to choose between a range of different results before ranking them according to various different criteria, such as reality and veracity. Some experts believe Microsoft may have skipped or rushed this step to launch Bing, although the company has yet to confirm or deny this claim.

But this method is not perfect, according to Agarwal. People could have been presented with options that were all wrong and then picked the least wrong option, she says. In an effort to make ChatGPT more reliable, the company focused on cleaning up its data set and removing instances where the model had a preference for things that are wrong.

Jailbreaker ChatGPT: Since the release of ChatGPT, people have tried to “jailbreak” it, which means finding workarounds to trick the model into breaking its own rules and generating racist or conspiratorial material. This work has not gone unnoticed at OpenAI headquarters. Agarwal says OpenAI went through its entire database and picked out prompts that led to unwanted content to improve the model and prevent it from repeating those builds.

OpenAI wants to listen: The company said it will start collecting more public feedback to shape its models. OpenAI is considering using polls or setting up citizens’ assemblies to discuss what content should be completely banned, says Lama Ahmad. “In the context of art, for example, nudity might not be considered vulgar, but how do you think about it in the context of ChatGPT in the classroom,” she says.