Ask questions about a document without uploading the document

Uploading a PDF and asking questions about it is genuinely useful. It is also the most data-intensive thing you can do with a document, and the mechanics are worth understanding before you point it at a contract, a diagnosis, or a due diligence pack.

Document Q&A transmits more than you think

A summariser reads the document once. A chat assistant keeps reading it.

The standard architecture is retrieval-augmented generation. The document is split into passages, each passage is converted into a vector embedding, and the embeddings are stored in an index. When you ask a question, the system finds the passages most similar to your question and pastes them into the model's prompt alongside it.

That means:

• the whole document is processed at least once, to build the index
• the index — a derived representation of the content — is stored somewhere
• every question sends the question and several passages of the document
• ten questions can transmit substantially more of the document than a single read would have

The embeddings themselves are not innocuous either. They are a lossy but meaningful encoding of the source text, and research on embedding inversion has repeatedly shown that a surprising amount of the original wording can be reconstructed from them.

Read three separate terms, not one

"We don't train on your data" is a narrower promise than most readers hear. Check these independently:

1. Training. Is your content excluded from model training by default, or only after you find and toggle a setting? Consumer and business tiers of the same product frequently differ.
2. Retention. How long are the file, the derived index, and the conversation history kept? Zero-retention modes exist, but they are usually an enterprise API option rather than the default in a web interface.
3. Human review. Many providers reserve the right to have staff inspect flagged conversations for safety and abuse. That is a legitimate practice and a completely different exposure from training, and it survives a training opt-out.

Add the ordinary questions on top: which subprocessors are involved, in which jurisdictions the data is handled, and what happens to conversation logs during an incident investigation.

For a business handling client material, none of these are answerable by inspection. They are contractual, which is exactly the position the safety guide for online PDF tools describes: you end up trusting a pipeline you cannot see.

What running the model locally changes

Browsers can now execute quantised language models directly, using WebGPU where it is available and WebAssembly as a fallback. The browser-local chat tool uses that capability: the model weights are fetched like any other static asset, the document is parsed and indexed in the tab, and generation runs on your own hardware.

The consequences are concrete rather than rhetorical:

• there is no upload, so there is no retention window and no deletion promise to evaluate
• the index lives in the tab's memory and disappears with the tab
• there is no conversation log on anyone's server
• the work continues after the runtime has loaded even if you disconnect

The local summarisation tool works the same way for the simpler case where you want an overview rather than a conversation.

The honest trade-offs

An on-device model has to fit in a browser tab, and that has costs worth stating plainly:

• Capability. A model that downloads in seconds is not the frontier model you are used to. It is good at extraction, locating passages, and summarisation; it is weaker at multi-step reasoning across a long document.
• First-run cost. The weights have to be downloaded once. That is a real wait on a slow connection, though it is cached afterwards.
• Hardware sensitivity. A recent machine with working WebGPU is dramatically faster than an old laptop falling back to CPU execution.
• Context length. Smaller models hold less at once, so retrieval quality matters more, and very broad questions across a 400-page document are harder.

Treat the answers as a first pass regardless of where the model runs. Language models state incorrect things fluently, and a confident summary of a clause you did not read is worse than no summary. Verify anything you plan to act on against the page it came from.

A workable split

• Public or already-published documents. Use whatever is most capable. There is nothing to protect.
• Internal, client, personal, medical, legal, or unreleased material. Keep it local. The capability gap is smaller than the exposure gap.
• Unsure? Run the privacy risk scanner over the file first. Seeing the identifiers, names, and metadata a document actually contains usually settles the question quickly.

The useful reframing is that this is not a question about AI at all. It is the same question as any other document tool: does this thing need to receive my file? For a growing set of tasks, the answer is now no.