You have a ten-page report that you need to read in preparation for a meeting. The application’s summary has an air of confidence and fluency. However, the concern is that there are aspects you may not see. Has the AI changed the meaning of “approximately” to “exactly”? Did the AI eliminate the word “not”? Did the AI miss the page where the most significant alert to be aware of is located?
A fluent summary is more difficult to detect than a summary written in a disjointed manner. To evaluate how well any AI has summarized your information, you can take the summary and break down its content into individual statements and then search for validation of each of the statements based on your source document. This method will be explained further below, along with a comparison of six of the other AI tools you can use for conversing with your PDF. The comparison discusses how each tool interprets and processes your document, when visitors to the site reach their free membership tier, and the fate of a document once uploaded to the tool. These results may vary based on the document submitted for summarization and date; therefore, please refer to the guidelines and policy statements listed as of September 2026.
Why AI Summarisers Change Facts and Drop Qualifiers
A summary is an individual’s decision of what to preserve. When a Summariser writes new sentence construction rather than simply copying previously created sentence constructions of original text, there could be variations and/or omissions in the facts. Errors that carry the greatest level of concern are those that can be read as accurate.
The model predicts plausible text, not true text
According to IBM’s hallucination explainer, Generative AI’s output has been optimised for plausibility, not accuracy, and it is said to not understand Truth and Falsity. In addition, IBM discusses a specific kind of hallucination that is important for document processing. Specifically, a “Contextual Hallucinator” is a type of model that has a fixed set of answers for specific questions. When the model has been fed additional contextual information (i.e., when you give it a provided document to assist with retrieval of an answer), it can revert back to its earlier provided training dataset information, rather than using the contextual data provided. The same source also states that there are ways to minimise the frequency of hallucinations, however, it is not possible to completely eliminate them.
In relation to PDF Summaries, the dangerous error will not be an obvious one; instead, it will be an innocent sounding error which is also not present in the document itself.
The five errors to look for
The list below outlines the developing working list of error types created based upon these identified classifications.
- Missing fact: something important in the source doesn’t appear in the summary. You can’t spot this without the source open.
- Lost qualifier: the fact survives but its hedge doesn’t. Words like “approximately”, “up to”, “estimated”, “in 2023 only” or “not including” disappear. For example, “revenue grew by approximately 5%” becoming “revenue grew by 5%” is a change of fact, not of style.
- Changed number: a figure is rounded, attached to the wrong item, or stated as exact when the source hedged it.
- Invented claim: a fact, figure or citation that appears nowhere in the source.
- Broken reasoning: each sentence looks right, but a cause becomes an effect, or a caveat on page 3 stops applying to the claim on page 7.
Long files make it worse
When a large document cannot be assessed at once, it will be processed as smaller pieces of work. An OpenAI developer forum member who created a summariser stated on that forum that “most methods lose the important details and connections between itemised arguments.” This programmer proposed a method of summarising the content using the content’s table of contents. In similar fashion, Google’s Gemini’s upload help page warns that an upload may yield a less detailed summarised output if the upload is overly large.
Not every wrong summary is the AI’s fault
Two other causes produce the same symptom. First, the PDF may never have been read properly. OpenAI’s file uploads FAQ says that outside ChatGPT Enterprise, ChatGPT pulls out the digital text and discards images. A scanned page is an image, so a summary of a scanned file may be built on almost nothing. Second, your instructions can force losses: asking for three bullet points on ten dense pages means something has to go. Each cause needs a different fix, which is why the test below separates them.
The 10-Page PDF Test: How We Measured Factual Consistency
The outlined procedure utilizes a method called FActScore, which stands for Fact Score. This method is defined by peer-reviewed research and generates summaries of a document based on “atomic facts.” Each atomic fact is evaluated against a reliable source, which in this case is the PDF file that contains the document being summarised. The evaluation process does not just look to determine whether each atomic fact in the summary has a corresponding piece of evidence from a reliable source, but also checks to ensure that no information from the PDF is missing or misrepresented in the summary. The evaluation process has an added conceptual direction as this article is adapting the evaluation.
Choosing the document
A test document must be selected that is publicly available and contains selectable text. It should include what trips summarisers up:
- specific figures
- hedged claims
- at least one negation
- a caveat printed pages away from the claim it limits
Examples of what could be used as a test document are a published annual report or a public research brief. It is important that the selected document does not contain any confidential data or variables, such as numbers or information related to an ongoing project. The reasoning for this is explained within the Privacy section.
Step 1: Build a Claim List prior to running any of the tools. For each important claim on the ten pages of the test document, generate a one-sentence claim with the page number and the qualifier underlined. By creating a Claim List before running any of the tools, the generated summary will not influence your thoughts on what the document has said.
Step 2: Provide identical instructions to each of the tools being tested. Use one basic prompt. For example, you can instruct each of the tools to “Summarise this document in approximately 300 words with numbers and qualifiers included.” By providing a fixed-length summary to each tool, you can compare the summaries fairly and determine if any forced omissions occurred as an effect of text reduction.
Step 3: Grade each sentence of each tool’s generated summary. Break down each summary to identify the key claims within the summary. After identifying the claims in the summary, categorise them as either Supported, Missing qualifier, Number Changed, or Unsupported. Finally, compare the claims in the summary to those listed in the Step 1 List and mark as Missing any items from the Step 1 List that were not referenced in the summary.
Fourth step: Record results. Write down the date, plan type (free service or paid service), and number of times you used each tool/s. Each tool may provide different answers for the same document on subsequent runs. Additionally, vendors update their algorithms regularly without warning. The result of one successful run is a snapshot of how that particular document performed at that moment. It is not an indication of which tool is superior overall.
Claim-by-Claim Accuracy Comparison
Every vendor documents how their tool will best convert a PDF into a usable format based on the same criteria that will help determine which one will cause you to make the five errors listed above (it is not a ranking of accuracy). That determination will be made based on comparing the output to the original source (using the source comparison sheet below).
| Tool | How it reads your PDF (vendor-documented) | Accuracy catch to watch |
|---|---|---|
| ChatGPT | Text-only retrieval outside Enterprise; images are discarded | Charts and scanned pages aren’t read |
| Claude | Text and visuals up to 100 pages; text only from 101 to 1,000 pages | Charts in long PDFs are skipped |
| Gemini app | Accepts up to 10 files; Google warns large uploads may miss details | Long files risk lost connections |
| Gemini Notebook (formerly NotebookLM) | Answers from your sources and says it won’t answer if the source lacks it | Copy-protected PDFs fail to import |
| Adobe Acrobat AI Assistant | Citations link to exact source locations | No image or complex-graphics support; prompts must be under 500 characters |
| ChatPDF | Citations and a side-by-side view; routes queries between GPT-4o and GPT-4o-mini | You don’t choose which model answers |
The general trend is that tools that link the answer back to the specific place in the document will make Step 3 quicker because you are able to go directly to the original text. Therefore, while citations will direct you to the appropriate area of the document, they will not prove the sentence is correct. If a citation’s page reference is in the quote and that page has the qualifier, that page reference may be incomplete.
The grading sheet
So, make a copy of this list for every tool and record your claims in the same manner as above.
| # | Source wording (page) | Summary wording | Grade |
|---|---|---|---|
| 1 | “…grew by approximately 5%” (p. 3) | “…grew by 5%” | Qualifier lost |
| 2 | “…not associated with…” (p. 6) | (absent) | Missing |
| 3 | — | “…a 2022 survey found…” | Unsupported |
The example rows above were created as hypothetical examples and do not represent actual results from any of the tools listed.
Strengths, and a reason to skip each
- ChatGPT: it accepts large files, up to 512MB per file. Skip it for scanned or chart-heavy PDFs, because anything that isn’t digital text is discarded outside Enterprise.
- Claude: it reads charts and images in PDFs of up to 100 pages. Beyond that it reads text only, so a 300-page report’s figures won’t be analysed.
- Gemini app: it takes files up to 100MB. Google’s own warning about missed details on large uploads is a reason to break long documents into parts first.
- Gemini Notebook: it’s built to answer from your sources rather than general knowledge, which makes grading easier. It won’t import a copy-protected PDF.
- Adobe Acrobat AI Assistant: its citations point to exact locations. It only supports certain languages, and it doesn’t handle images or complex graphics.
- ChatPDF: you can start without an account. Its homepage doesn’t say whether uploads are used for model training, so read its policy before uploading anything sensitive.
Free-Tier Limits, File Caps, and Upload Privacy
Both the limits of the tool and the privacy statement determine whether your document will be read entirely, and neither of those statements is guaranteed to be accurate as of September 2026.
| Tool | Free-tier or file cap | Training on your uploads (vendor’s statement) |
|---|---|---|
| ChatGPT | 512MB and 2M tokens per file; Free users get 3 uploads a day | May be used unless you turn off “Improve the model for everyone” |
| Claude | PDFs up to 1,000 pages | If you allow it, kept de-identified for up to 5 years |
| Gemini app | Up to 10 files per prompt; 100MB each for non-video files | Used for training, with human review, while Keep Activity is on |
| Gemini Notebook | 500,000 words or 200MB per source; no page limit | Not used to directly train foundational models unless you send feedback |
| Adobe Acrobat AI Assistant | Under 100MB, up to 600 pages, no password protection | Adobe states it never uses your documents to train AI models |
| ChatPDF | 2 documents a day on the free plan | Not addressed on its homepage, which describes encryption and deletion |
Vendor pricing varies frequently; check each vendor’s pricing plan before purchasing. As an example, Adobe sells AI Assistant Plus as a supplementary product to both free and paid Acrobat plans.
The exceptions hidden in “opt out”
You may not be able to disable the training.
- OpenAI: after you opt out, a thumbs-up or thumbs-down rating can still send the whole conversation into training.
- Anthropic: it keeps feedback data for 5 years.
- Google (Gemini app): with Keep Activity off, chats are still kept for 72 hours, and chats already reviewed by humans are kept for up to three years even if you delete your activity.
- Google (Gemini Notebook): feedback, including any uploaded content that comes with it, is reviewed by trained teams.
Google states in their own privacy policy: “Please do not include any confidential or sensitive information in your documents.”
Before you upload anything
Step 1: Open the PDF file to confirm that it contains real text and not just scanned images. To do so, press Ctrl+F (Cmd+F on a Mac) and type in a word from the document. If the document contains only scanned images, then a text processor such as ChatGPT outside of an enterprise environment will not recognize those images.
Step 2: You must decide whether or not you will upload those documents outside your company. Contracts with confidentiality clauses, patient medical records, paystubs, and unpublished research should never be uploaded to consumer-level accounts, even if the service is advertised as safe.
In Step 3, you should use Private Browsing Mode—ChatGPT’s Temporary Chat; Claude’s Incognito; or Gemini’s temporary chat option. Each vendor claims data from each will not be used for further training. Please refrain from rating their response.
In Step 4, perform a Coverage Check. Request the tool, Section by Section, the Outline of the file and compare this with the actual Headers of the Document. If the Generated Outline terminates early, it indicates either Cap or Truncation, not compression.
The Quick Version
Generally speaking, no summariser is completely accurate, as accuracy depends on a single tool in a specific document at a given time. To ascertain accuracy, you must compare the generated output to the original document. Most commonly, the summaries will contain misinterpretations that their qualifiers get left out; while that may be the case, the sentence does not appear wrong when looking at it from this perspective. Always investigate the most critical aspects of the summary and compare them against the PDF directly, as well as reviewing the vendor’s training specification prior to exporting the file from your device.
Frequently Asked Questions
Why did the tool only summarise part of my PDF?
There are many reasons why your results might not match what you were expecting. Maybe there are too many pages being scanned, perhaps the cap has been hit for the number of pages scanned, or maybe the scanning process did not pick up the pages correctly and too many were compressed into a summary. Request a section-by-section outline of your document from ChatGPT and compare it to the headings of your original document; this will tell you which of those three possibilities applied to your document.
Will paying for a plan make summaries more accurate?
Additionally, Paid subscriptions increase the amount of files users can upload at one time; whereas the $20 monthly subscription for ChatGPT Free allows 3 uploads per day. However, regardless of the upload limit for your Plan, if you are using either subscription, you must still validate your summaries by reviewing them against what was in the original document.
