How Chatbots and Large Language Models, or LLMs, Really Work

In the second of our five-part seriesI’ll explain how the technology actually works.

The artificial intelligences that power ChatGPT, Microsoft’s Bing chatbot, and Google’s Bard can conduct human conversations and write natural, fluent prose on an endless variety of topics. They can also perform complex tasks, from writing code to planning a child’s birthday party.

But how does it all work? To answer that, we need to take a look under the hood of what’s called a Grand Language Model – the kind of AI that drives these systems.

Large Language Models, or LLMs, are relatively new to the AI ​​scene. The first ones only appeared about five years ago, and they weren’t very good. But today they can write emails, presentations and memos and teach you a foreign language. Even more capabilities are sure to surface in the months and years to come as technology improves and Silicon Valley scrambles to cash in.

I’ll walk you through creating a great language model from scratch, keeping things simple and leaving out a lot of hard math. Let’s say we’re trying to create an LLM to help you answer your emails. We’ll call it MailBot.

Step 1: Set a goal

Every AI system needs a goal. Researchers call it a objective function. It can be simple – for example, “winning as many chess games as possible” – or complicated, such as “predicting the three-dimensional shapes of proteins, using only their amino acid sequences”.

Most great language models have the same basic objective function: given a sequence of text, guess what comes next. We’ll give MailBot more specific goals later, but let’s stick with that for now.

Step 2: Collect lots of data

Next, we need to assemble the training data that will teach MailBot how to write. Ideally, we’ll create a colossal repository of text, which typically means billions of pages pulled from the internet, like blog posts, tweets, Wikipedia articles, and news.

To start, we will use free and publicly available data libraries, such as the Common Crawl Web Data Repository. But we’ll also want to add our own secret sauce, in the form of proprietary or specialized data. Maybe we will have a license for foreign language text, so MailBot learns to compose emails in French or Spanish as well as English. In general, the more data we have and the more diverse the sources, the better our model will be.

Before we can feed data into our model, we need to break it down into units called tokens, which can be words, phrases, or even individual characters. Turning text into bite-sized chunks helps a model parse it more easily.

Step 3: Build your neural network

Once our data has been symbolized, we need to assemble the “brain” of the AI, a type of system known as a neural network. It is a complex network of interconnected nodes (or “neurons”) that process and store information.

For MailBot, we’re going to want to use a relatively new type of neural network known as transformer model. They can analyze multiple pieces of text at the same time, which makes them faster and more efficient. (Transformer models are the key to systems like ChatGPT – whose full acronym stands for “Generative Pretrained Transformer”.)

Step 4: Train your neural network

Then the model will analyze the data, token by token, identifying patterns and relationships. He may notice that “Dear” is often followed by a name, or that “Best regards” usually comes before your name. By identifying these patterns, the AI ​​learns to construct messages that make sense.

The system also develops a sense of context. For example, he might learn that “bank” can refer to a financial institution or a river bank, depending on the surrounding words.

As it learns these patterns, the transformer model draws a map: an extremely complex mathematical representation of human language. It keeps track of these relationships using numeric values ​​called settings. Many of today’s top LLMs have hundreds of billions of metrics or more.

Training can take days or even weeks and will require immense computing power. But once it’s done, it’ll be almost ready to start writing your emails.

Oddly, it can also develop other skills. As LLMs learn to predict the next word in a sequence, they may gain other unexpected abilities, such as knowing how to code. AI researchers call these emergent behaviors, and they are still sometimes mystified by them.

Step 5: Refine your model

Once a large language model is trained, it needs to be calibrated for a specific job. A chatbot used by a hospital may need to understand medical terms, for example.

To fine-tune MailBot, we could have it generate a bunch of emails, hire people to rate them on accuracy, and then feed the ratings back into the model until it improves.

This is a rough approximation of the approach used with ChatGPT, known as reinforcement learning with human feedback.

Step 6: Launch, carefully

Congratulations! Once MailBot has been trained and fine-tuned, it’s ready to use. After creating some sort of user interface, like a Chrome extension that connects to your email app, it can start sending emails.

But no matter how good it looks, you’ll still want to keep an eye on your new assistant. As companies like Microsoft and Meta have learned the hard way, AI systems can be erratic and unpredictable, and even become frightening and dangerous.

Tomorrow we will know more about how things can go wrong in unexpected and sometimes disturbing ways.

Your homework

Let’s explore one of the most creative abilities of LLMs: the ability to combine disparate concepts and formats into something bizarre and new. For example, our colleagues at Well asked ChatGPT to “write a song with Taylor Swift’s vocals that uses themes from a Dr. Seuss book.”

For today’s assignments, try mixing and matching format, style and topic, for example: “Write a limerick in the style of Snoop Dogg on global warming.”

Don’t forget to share your creation in the comments.

---

Quiz

Question 1 of 3

What is the main objective function of large language models like ChatGPT?

Start the quiz by choosing your answer.

---

Glossary

• Transformer model: A neural network architecture useful for understanding language, which does not have to analyze words one by one but can look at an entire sentence at a time. A technique called self-attention allows the model to focus on particular words that are important to understanding the meaning of the sentence.

• Settings: Numerical values ​​that define the structure and behavior of a large language model, like clues that help it guess which words come next. Modern systems like GPT-4 are believed to have hundreds of billions of parameters.

• Reinforcement learning: A technique that teaches an AI model to find the best result through trial and error, receiving rewards or punishments from an algorithm based on its results. This system can be improved by humans giving feedback on its performance.

Click here for more glossary terms.