How large language models work

Tokens, transformers, training and why context windows matter.

What it is

An LLM turns text into tokens, passes them through layers of a transformer network that weigh how every token relates to every other one (attention), and outputs a probability for the next token. Pre-training teaches language; fine-tuning and feedback teach it to be a helpful assistant.

Why it matters

Knowing about tokens, context windows and training cut-offs explains most of the odd behavior you'll see: forgotten instructions, outdated facts and miscounted letters.

Key ideas

• Tokens are word pieces. Limits and pricing are counted in tokens.
• Attention lets the model relate distant parts of the input.
• The context window is everything the model can see right now.
• Knowledge stops at the training cut-off unless a tool fetches more.

Try it: Probe the context window

1. Paste a long article and ask a question about its first paragraph.
2. Keep chatting for twenty turns, then ask about that paragraph again.
3. Ask the model what it knows about an event after its training cut-off.