Language models · Next-token prediction

Predict the Next Word

A language model does not look anything up. It guesses the next fragment of text, then does it again, and the answer falls out of the repetition. Ask it something and watch your words become tokens and vectors then, for every single word it writes, the real odds it was choosing between.

What you are looking at

One token at a time

It never plans the sentence

Nothing decides the answer up front. The model picks one fragment, adds it to what it has written so far, and runs the entire network again to pick the next. A finished sentence is what forty of those passes happen to add up to.

A distribution, not an answer

Every word had rivals

At each step the model holds a ranked list of candidates with a probability on each, and samples one. The bars are those real numbers. Click any word in the finished answer to see what almost got written in its place.

What is real here

The probabilities, not the plumbing

The answer, the candidates, their percentages and which one was picked are genuine output from the model. Tokenising, the embedding grid, the attention lines and the layer stack are illustrations the API does not hand out a model's internals, so those stages stand in for them.

Where it runs

This one does use a server

The rest of the playground runs entirely in your browser. This demo cannot: the model is far too large, so the question goes to an API and the answer comes back with its probabilities attached. Everything you then watch happen to that answer is drawn here, live.

A demonstration of the mechanism, not a product. The same models, wired to a client’s own documents and processes rather than to a demo box, are what we put into production.

Talk to us about your own problem →