Context Window

In detail

The context window is the total number of tokens a language model can attend to at once. This includes the prompt, any conversation history, retrieved documents (in a RAG setup), tool call results and the model's generated response, all of which count against the limit. Older models had context windows of 4K or 8K tokens. Current models range from 128K to one million tokens or more. A larger context window lets you send more documents, longer conversations and richer tool outputs without chunking, but larger contexts cost more and can cause the model to lose focus on information in the middle of a very long context.