Context window

Every model has a context window: the maximum span of tokens it can attend to in one request, covering both what you send in and what it writes back. Go past it and the earliest text falls out of view.

A bigger window helps, but it does not replace good retrieval. Stuffing everything into the prompt spreads the model's attention thin and runs the cost up faster than the text grows, which is why pulling in only the relevant passages usually beats dumping in everything.