Context window
Also called: context length · context limit · context size
How much an AI can hold in mind at once; anything outside it is invisible to the model when it answers.
Why it matters
Every conversation with an AI runs inside a hard limit. Paste in a 200-page contract and the model may only be reading part of it. It will not warn you: it answers confidently from what it can see, and what it cannot see simply does not exist. Many 'the AI ignored my instructions' complaints are context problems, not intelligence problems.
How it works
Everything a model reads before it answers (your instructions, the conversation so far, any documents you paste in) has to fit inside one fixed space: the context window. It is measured in tokens, not pages, and it includes the answer the model is writing as well as everything you gave it.
When a conversation outgrows the window, something has to give. Depending on the product, the oldest messages are dropped, silently summarised, or trimmed. The model does not announce this. It simply answers using whatever still fits.
Two clarifications that save arguments later: the window is per conversation, not per account, and it resets every time you start a new chat. Nothing “learned” in one conversation carries into the next unless the product has an explicit memory feature layered on top.
Where it goes wrong in practice
The classic failure is quiet truncation. A long working session, an agreed decision two hours ago, and an answer near the end that contradicts it. The model has not changed its mind. The decision scrolled out of the window, and for the model it no longer exists.
The subtler failure: fitting is not the same as being read. Models attend unevenly across long inputs, and material buried in the middle of a large paste is used less reliably than material at the start or the end. Teams test with a question about page 2, it works, and they conclude the whole document is covered. It is not. The research literature calls this “lost in the middle”.
And there is a bill. You pay for every token the model reads, on every request. Pasting the entire policy manual into every question is the expensive way to get worse answers.
What to do about it
Habits first, tooling second. Keep one task per conversation and start fresh chats freely. Restate the decisions that matter instead of trusting the model to remember them. Put the material the answer depends on at the start or the end of your prompt, not the middle. And when the same corpus is needed across many conversations, stop pasting: that is the problem RAG exists to solve.
The question to ask before relying on one
Not “which model has the biggest window”. The window is a budget, and every request spends it. The question is: for this task, what does the model actually need to see, and who is deciding what gets left out? If the answer is “whatever the tool happens to keep”, the failures will be silent and you will meet them in production.
Common questions
Why does the AI forget what we said earlier in the conversation?
Because the conversation outgrew the window. Once the total text exceeds the limit, the earliest turns are dropped or compressed to make room, and the model answers as if they never happened. It is not being careless; the information is genuinely gone. The fix is procedural: restate what matters, keep one task per conversation, and treat long-running chats with suspicion.
A bigger context window means better answers, right?
Not by itself. A bigger window means more can fit, not that everything in it gets equal attention. Long inputs are read unevenly, the middle sections least reliably of all, and every extra token adds cost and latency. A small, well-chosen set of passages routinely beats a giant paste. Window size is capacity, not comprehension.
How much text actually fits?
Windows are measured in tokens, and a token is roughly three quarters of an English word. A dense page of text is in the order of 500 tokens, so a 128,000-token window is a few hundred pages in theory. In practice, reserve room for the conversation itself and the answer being written, and remember that fitting is not the same as being read carefully.
Is the context window the same as the model's memory?
No. The window is working memory: it holds this conversation, and it empties when the conversation ends. Products that appear to remember you across sessions are adding a separate feature on top, usually by saving notes and re-inserting them into the window later. Anything not written back in is forgotten.
Working through this with your team?
Olaia Labs runs AI training and automation programmes for SMEs. Start with a diagnostic: we look at where your team already uses AI, where it breaks, and what is worth automating.
Request a diagnostic