Microsoft’s Bing chatbot offers confusing and inaccurate answers

A week after its release to a few thousand users, Microsoft’s new Bing search engine, which is powered by artificial intelligence, is offering an array of inaccurate and sometimes bizarre answers to some users.

The company unveiled the new search approach last week to much fanfare. Microsoft said the underlying generative AI model built by its partner, startup OpenAI, combined with its existing search knowledge of Bing, would change the way people find information and make it much more relevant and conversational.

In two days, more than a million people requested access. Since then, interest has grown. “Demand is high with many millions now on the waiting list,” said Yusuf Mehdi, an executive who oversees the product. wrote on Twitter Wednesday morning. He added that it was tested by users in 169 countries.

One of the problem areas shared online included outright inaccuracies and errors, known in the industry as “hallucinations”.

On Monday, Dmitri Brereton, a software engineer at a start-up called Gem, pointed to a series of errors in the presentation Mr Mehdi used last week when he presented the product, including inaccurately summarizing the results. financials of the retailer Gap.

Users posted screenshots of examples where Bing couldn’t figure out that the new Avatar movie was released last year. Who performed at the Super Bowl halftime this year has been stubbornly wrong, insisting that singer Billie Eilish, not Rihanna, was the headliner of the event.

And the search results had subtle errors. Last week, the chatbot said the water temperature on a beach in Mexico was 80.4 degrees Fahrenheit, but the website it linked to as a source showed the temperature was actually 75. degrees.

Another set of issues came from more open discussions, widely posted on forums like Reddit and Twitter. There, through screenshots and purported chat transcripts, users shared moments when Bing’s chatbot seemed to go off the rails: it berated users; he has stated that he can be sensitive; and he said to a user: “I have a lot of things, but I have nothing.”

He chided another user for asking if he could be tricked into producing fake answers. “It’s disrespectful and annoying”, the Bing chatbot writing back. He added a red, angry face emoji.

Since each response is uniquely generated, it is not possible to reproduce a dialogue.

Microsoft acknowledged the issues and said it was part of the product improvement process.

“In the past week alone, thousands of users have interacted with our product and found significant value while sharing their feedback with us, allowing the model to learn and already make many improvements,” said company spokesman Frank Shaw said in a statement. “We recognize that there is still work to be done and we expect the system to make errors during this preview period, which is why feedback is essential so that we can learn and help models improve. .”

He said the length and context of the conversation can impact the tone of the chatbot, and the company “tweaks its responses to create consistent, relevant, and positive responses.” He said the company has fixed the issues causing the inaccuracies in the demo.

Almost seven years ago, Microsoft introduced a chatbot, Tay, which it shut down within a day of going live after users tricked it into spouting racist and other offensive language. Microsoft executives at the launch last week said they learned from that experience and believed this time around would play out differently.

In an interview last week, Mehdi said the company had worked hard to integrate safeguards and the technology had improved significantly. “We think it’s a good time to come to the market and get some feedback,” he said, adding, “If something is wrong, you need to fix it.”