Prompts, context and outputs

A model is fixed once trained: same dials, same patterns. What changes between a useless answer and a brilliant one is almost entirely what you feed in. That input has a name, a size limit and a couple of settings worth knowing about.

The prompt is the whole input

The prompt is everything you give the model: your instructions plus any context. Summarise this is an instruction. The article you paste underneath is context. Both together are the prompt, and the model treats it all as one block of text to continue. This is why adding context changes answers so much. Ask for study tips and you get generic advice. Ask for study tips, then explain that you are in year 11, sitting chemistry in three weeks, and study best in short bursts, and the model has far more pattern to work with.

Tokens: how the model reads your words

So how does the model actually read your prompt? Not in words, and not in letters. A model can only work with numbers, so before anything else happens your text is chopped into tokens, small reusable chunks that each map to a number. A token is often a whole common word, and sometimes a piece of one: unhappiness might split into un, happi and ness. As a rough guide, 100 tokens is about 75 words of English. This matters for two practical reasons. Context windows are measured in tokens, so a long pasted document fills the window faster than you might guess. And because most training text is English, other languages often break into more tokens per sentence, so the same window holds less Mandarin or Arabic than English.

The context window: working memory

A model can only pay attention to a limited amount of text at once, called the context window. Think of it as working memory. Everything inside the window, your messages, its replies, any pasted documents, can shape the next answer. Anything outside it may as well not exist. This is why a long conversation can drift: as early messages fall outside the window or get compressed, the model genuinely loses track of them. It is not being forgetful in a human way. The text simply is not in front of it any more.

People often assume the model remembers them between conversations. By default it does not. Each new chat starts with an empty window, so the model that helped you plan an essay yesterday has no trace of it today. When a chatbot does seem to remember you across sessions, that is a memory feature at work: the app has saved a note about you and is quietly placing it into the window at the start of each chat. The memory lives in the product wrapped around the model, not in the model itself. And the fix for a conversation that has drifted follows directly from the mechanism: put the important details back in front of the model, by restating them or pasting them in again, and it will pick them straight back up.

Why the same prompt gives different answers

Ask the same question twice and you will often get two different answers. At each step the model holds a ranked list of plausible next words, and rather than always taking the single top pick, it usually samples from the strong candidates. A setting called temperature controls how adventurous that sampling is. Low temperature means play it safe, take the likely words, which suits factual tasks. High temperature means take more chances, which suits brainstorming and can produce more nonsense. Variation between runs is normal behaviour, not a glitch.

Garbage in garbage out

Programmers have a blunt old saying: garbage in garbage out. It applies doubly to AI. A vague prompt gets a vague answer. A prompt built on a false assumption often gets that assumption politely reflected back, because the model continues the text you started rather than auditing it. Ask why goldfish have three second memories and many models will happily explain a myth. The quality of what comes out is capped by the quality of what went in, and that part is entirely in your hands.

Where this lands in your week

This is not abstract advice, because you will run into it this week. If you ask for help with a maths problem but do not paste the actual question, you get help with a similar problem, not yours. If you ask a chatbot to draft a message to your boss about swapping a shift and do not mention which shift, which day or why, you get a template that sounds like nobody. And if someone in your group chat pastes an AI answer that starts from a wrong assumption, the polish of the reply hides the rot in the question. The people who get the most out of these tools are not using secret techniques. They are just feeding in better inputs: the real question, the real constraints, the real context.

There is a darker side to garbage in garbage out, though. Even with a perfect prompt, the model can produce text that is confidently and completely wrong, because generating plausible text is what it does whether or not the truth is available to it. That failure has a name, and it gets the next lesson to itself.