In one sentence
A token is a small piece of text, roughly a word or part of a word, that a language model reads and produces in.
The simple model
Models do not process text as whole sentences or characters, but as tokens: chunks that are often a word, sometimes part of a longer word, sometimes a piece of punctuation. As a rough guide, a token is about three quarters of a word in English, so a page of text is a few hundred tokens.
You almost never handle tokens directly. But two practical things are measured in them, so the word is worth knowing.
Why you care
Tokens show up in two places that affect your design decisions:
- Cost. Most AI services charge by the number of tokens processed, both what you send and what you get back. A workflow that sends large documents to the model on every request costs more than one that sends only the relevant few passages.
- Limits. The context window, the amount the model can consider at once, is measured in tokens. This is another reason to send relevant chunks rather than entire documents.
So the practical lesson is the same one that shows up throughout: give the model what it needs and little else. It is cheaper and it works better.
A small example
A team pasted whole manuals into every request, which was both expensive and less accurate. Switching to retrieving only the relevant passages cut their token usage sharply, lowered the cost, and improved answers, because the model was no longer wading through irrelevant text.
What this does not solve
Understanding tokens helps you reason about cost and limits; it does not improve the quality of answers on its own. Sending fewer, more relevant tokens usually helps both, but accuracy still depends on the sources, the design and the review around the model.
