What a language model really does

Type half a sentence into your phone and it suggests the next word. A large language model is that idea, scaled up by a factor that is hard to picture: trained on a huge slice of the text humans have ever written, with enough internal settings to capture extraordinarily deep patterns in how words follow words. Understanding this one mechanism explains almost everything a chatbot does, both the impressive parts and the strange ones.

One trick, repeated

A language model does exactly one thing: given the text so far, it estimates which small chunk of text, called a token, is likely to come next. It picks one, adds it to the text, and repeats. Every essay, poem, apology and explanation a chatbot has ever produced was built this way, one token at a time. There is no separate module for facts, no inner database being consulted, no plan written out in advance. There is only the question: given everything so far, what plausibly comes next?

Words become numbers

So what is a token, and why not just use whole words? A model cannot read letters or words at all. It only works with numbers, so before any text reaches it, the text is chopped into tokens, which are common chunks of characters, and each token is turned into a number the model can process. A token is often a whole short word, sometimes a word ending like ing, sometimes a single character. As a rough guide, 100 tokens come out to about 75 words of English. Tokens act like reusable building blocks: a few tens of thousands of them can be combined to write almost anything, which is far more efficient than keeping a separate number for every word a language has.

This has a consequence worth knowing, especially in a country where millions of people speak a language other than English at home. Because most of a model's training text is English, the token chunks are tuned for English, so English packs a lot of meaning into each token. Other languages tend to break into more tokens for the same sentence, so the same message in Mandarin or Arabic can use two to three times as many tokens as it does in English. Since these tools often charge by the token and have a fixed token limit, the language you work in quietly changes how much you can fit and what it costs.

Why prediction produces fluency

Here is the surprising part. Predicting text well turns out to require absorbing enormous amounts of structure. To continue a sentence about photosynthesis plausibly, the model must have internalised patterns about grammar, tone, and how explanations of photosynthesis tend to go. Because its training data contains millions of good explanations, arguments and stories, the patterns it learned produce text that is fluent, confident and usually sensible. But notice what the model is actually good at: producing text shaped like a right answer. Whether the content is true was never the target, only whether it is likely. Most of the time likely text and true text overlap. When they part ways, the model follows likely, and says something false with perfect confidence.

A worked example, one step

Walk through a single step. Suppose the text so far is: the capital of France is. The model turns that into tokens, runs its numbers, and produces a ranked list of candidates for the next token with a score on each. Paris might score very high, Lyon and Nice far lower, and a random word like tomato almost nothing. The system picks from the top of that list, adds Paris, and now the text reads the capital of France is Paris. Then it does the whole thing again to choose the word after that. Notice there was no lookup of a fact called the capital of France. Paris won because, across billions of sentences in the training data, Paris is overwhelmingly the token that followed those exact words. The right answer and the likely next token happened to be the same, which is why the model looks like it knows geography when really it is counting.

The context window

A model can only attend to a limited amount of text at once, called its context window. Your conversation lives inside it. While the chat fits, the model can use every detail you have given. Once the conversation grows past the window, the earliest parts fall out, which is why a long chat can suddenly forget the name you gave it an hour ago. The model has no memory beyond the window and its frozen training. It is not being forgetful in the human sense. The information is simply no longer in front of it.

Why the same prompt gives different answers

At each step the model produces a ranked list of plausible next tokens, and the system samples from the top of that list rather than always taking the single most likely option. A little randomness makes the writing feel natural instead of robotic. It also means the same prompt can produce a different answer every time, sometimes differing in wording, sometimes in substance. A chatbot is not a calculator that returns the one correct output. It is a sampler drawing from a space of plausible outputs.

A common misconception

The most common wrong picture of a chatbot is that it looks things up. People imagine it reading an internal encyclopedia, or quietly searching the web, then writing the answer in its own words. Unless the particular tool has been given a live search feature, none of that happens. The knowledge is not stored as facts it can look up. It is baked into those internal numbers as patterns, blurred together from everything it read during training, which is why it can be fluent about a topic and still get a specific name, date or quote wrong. It is not consulting a source and misreading it. There is no source in there to consult. Once you stop picturing a lookup, its mistakes stop being surprising.

Why it invents things for your homework

This explains a trap students fall into constantly. Ask a chatbot for three sources on a history topic and it will often produce three references complete with authors, titles and years that look exactly right and do not exist. It is not lying or being lazy. You asked for text shaped like a reference list, so it generated text shaped like a reference list, filling in a plausible author and a plausible year the way it fills in any next token. The same goes for a quote attributed to a famous person, or a statistic with a neat looking figure on it. When the thing you need is specific, recent or checkable, treat the model as a confident writer, not a source, and verify the detail somewhere real before you hand it in.

This one mechanism carries the rest of the AI literacy work. The next lesson maps out what the model is genuinely good and bad at, and every entry on that map falls straight out of what you have just read: it is strong wherever producing likely text is the whole job, and weak wherever the truth and the likely next token can come apart. Hold on to the difference between fluent and correct, because almost every mistake people make with these tools comes from mistaking one for the other.