You upload a long report. The AI accepts it, produces a fluent summary, and then misses a small exception in the middle. “The file was accepted” and “the answer covered every relevant detail” are different observations. Increasing the input limit can solve a capacity problem without solving that omission.
A useful check starts with one answer you can verify yourself. Choose an ordinary document you are allowed to share, find a passage with a clear exception, and keep its location beside your question.
Capacity is measured in tokens, not pages
Google’s token documentation explains that input and output are tokenized, including non-text inputs. A token may be a character, a whole word or part of a word. Two documents with the same page count can therefore make different demands on a model.
If you develop an application, count the actual request with the model’s supported token-counting method rather than estimating from the file’s megabytes. If you use a chat app, check that app’s file and conversation limits. A model’s advertised capacity does not establish which material a particular interface put into its request.
Ask for the exception before the overall summary
Google’s long-context guide warns that finding multiple pieces of information is not the same test as retrieving one isolated fact. Performance depends on the context. It also recommends avoiding unnecessary input and generally placing the question after the reference material.
Try a narrower request: identify the rule, the exception and the section that contains each. Check those locations in the original. Then ask for the summary with both included. This makes a missing exception visible before it is buried inside an otherwise plausible paragraph.
For example, in a fictional equipment manual, a weekly inspection rule might have a separate after-transport exception. A summary that includes only “inspect weekly” is incomplete even if the sentence is accurate. The missing relationship is the problem.
Retrieval can lose the surrounding meaning
In its 2024 retrieval explanation, Anthropic describes how a passage can lose its subject or time period when a document is split into chunks. Its method adds contextual information to improve retrieval. That is a different intervention from merely increasing the maximum prompt length.
When an answer keeps selecting the wrong edition or product, supply the exact document title and section rather than adding more loosely related files. If you control the application, inspect the retrieved passages. If you do not, ask the assistant to identify which source and section it actually used, then verify them yourself.
Use a small coverage test before a large commitment
Make a short answer key with a routine fact, an exception, a comparison across two sections and a question the document cannot answer. Score each response separately. A good system should preserve the exception and admit the missing information, not invent something to fill every row.
Repeat the same check after changing the model or input arrangement. This is a practical trial, not a statistically complete benchmark. It will not prove universal reliability, but it can reveal whether your actual bottleneck is accepting the document, finding evidence or combining it correctly.
