AI works by predicting the next piece of text
The systems behind chat assistants, support agents and document tools are language models, and a language model does one narrow thing. Given a stretch of text, it estimates what is likely to come next and emits one small piece of it. Then it reads its own output back and does the same thing again. Conversation, summary, analysis, code all of it is that single step, run a few thousand times over.
What makes that useful is scale. To predict text written by people who knew their subject, a model has to internalise a great deal about how the subject behaves. Grammar, factual association, the shape of an argument, code syntax and something that resembles reasoning all fall out of the one objective, because every one of them makes the next word easier to guess.
The floor above is what happens when you put that predictor in a loop and give it tools. Each worker picks up a task, decides what to do, calls something, reads the result and continues the same predict-and-continue step, now with consequences attached.
What happens when you press send
Six stages, in the order a request moves through them, each running at a station you can watch above. None of them involves the model looking anything up unless someone built that in.
Your text becomes numbers
A model has no notion of letters. Your input is split into tokens common word fragments, around four characters each and every token is swapped for a long list of numbers called a vector. Tokens used in similar ways end up with similar vectors. From this point on, the whole thing is arithmetic.
Training is where the knowledge went
Training is the expensive, one-off part. The model is shown text with the next token hidden, guesses it, and every wrong guess nudges billions of internal numbers its weights a fraction in the right direction. Repeat across trillions of tokens and those weights come to encode the structure of the material. Nothing is filed away verbatim: there is no document in there to look up.
Attention decides what matters
To produce each token, the numbers pass through dozens of layers. In every layer an attention step lets each token look at all the others and weigh which ones are relevant that pronoun refers to this name, this figure belongs to that column, this clause negates the one before it. Almost everything that reads as comprehension comes from that step.
One token at a time
The final layer emits a probability for every token in the vocabulary: perhaps a 60 per cent chance of one word, 12 per cent of another. One is picked, appended to the text, and the entire pass runs again for the token after it. There is no outline held in reserve a long answer is thousands of these steps, each conditioned on everything written so far.
It only knows what is in the context
The weights are frozen at training time and nothing carries over between calls. Anything specific or current a policy document, a customer record, the date has to be placed in the context window alongside the question. That is all retrieval-augmented generation is: search your own systems first, paste the relevant passages in, then ask. The window is finite, so what goes into it is a design decision.
Feedback turns a predictor into an assistant
A freshly trained model only continues text; ask it a question and it may well produce more questions. A second, much smaller training stage fixes that, using worked examples and human rankings of competing answers to pull it toward responding, following instructions and refusing what it should refuse. Same machinery underneath what changed is what it was rewarded for.
What follows from that
The mechanism explains the behaviour, including the parts people find surprising the first time they put one of these systems in front of customers.
It predicts plausibility, not truth
A wrong answer is produced by exactly the same process as a right one, which is why fabrications arrive in the same confident prose. The model is not consulting a store of facts, so something it has never seen and something it has read a thousand times both come out as fluent text. The practical fix is not a sterner prompt: it is putting the source in the context and checking the answer against it.
The same question can get different answers
Because each token is sampled from a distribution rather than chosen outright, output varies between runs a setting usually called temperature governs how much. Useful when drafting, awkward when you need repeatability, and the reason a system should be judged on a few hundred test cases rather than one impressive demo.
Put it in a loop and it can act
On its own a model only emits text. Give it tools it may call run this query, send this email, open this file feed the results back into its context and let it predict again, and you have an agent: predict, act, observe, repeat until the work is done or it hands over. Every worker on the floor above is that loop, running unattended.
Illustrative animation. The worker names, tasks and counters above are representative of the work we automate for clients, not a live system feed.
Talk to us about putting this to work →