Document Parsing: GPT-5 vs Claude vs Gemini vs Invofox — with Code Samples (2026)

Originally published comparing GPT-4o and Claude 3.5 Sonnet; fully updated August 2026 with current models. The model IDs, prices and code below are as of August 2026 — which is itself one of the points this post makes.

Every few months the model you benchmarked is no longer the model you can call. This post is a working comparison of four ways to turn an invoice into structured JSON: three general-purpose LLM APIs and one specialised extraction API. The code runs. The differences that matter, though, are not the ones a benchmark measures — so the second half is about what separates these options by design, and how to measure the rest on your own documents rather than trusting anyone’s published number.

A disclosure before anything else: Invofox publishes this blog and is one of the four options compared. Read the comparison sections with that in mind. The rule this post follows is that every claim about a third party is either a design fact linked to that vendor’s documentation or something you can reproduce yourself — and there are no accuracy percentages for anyone, including us. The last section explains why that is the only intellectually honest position, and what to measure instead.

Parsing an invoice with the OpenAI API

pip install openai pdfplumber python-dotenv

Unlike a document API, a chat model needs the text extracted first — a PDF goes through pdfplumber (or an OCR pass for scans) before the model ever sees it. That preprocessing step is yours to own, and on a scanned or photographed document it is where most of the errors are introduced, before the model gets a chance.

import json, os
import pdfplumber
from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

INVOICE_SCHEMA = {
    "type": "object",
    "properties": {
        "invoice_number": {"type": "string"},
        "issue_date": {"type": "string"},
        "supplier_name": {"type": "string"},
        "total_amount": {"type": "number"},
        "currency": {"type": "string"},
    },
    "required": ["invoice_number", "issue_date", "supplier_name", "total_amount", "currency"],
    "additionalProperties": False,
}

def extract_text(pdf_path: str) -> str:
    with pdfplumber.open(pdf_path) as pdf:
        return "\n".join(page.extract_text() or "" for page in pdf.pages)

response = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[
        {"role": "system", "content": "Extract invoice fields. Use null for anything not present. Do not guess."},
        {"role": "user", "content": extract_text("invoice.pdf")},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "invoice", "schema": INVOICE_SCHEMA, "strict": True},
    },
)

invoice = json.loads(response.choices[0].message.content)
print(invoice["invoice_number"], invoice["total_amount"])

Parsing an invoice with the Anthropic API

pip install anthropic

Claude reads PDFs natively — no pdfplumber step. The snippets below reuse the same INVOICE_SCHEMA defined in the OpenAI example above, so the three APIs are asked for exactly the same thing.

Two things changed since this post was first written and both are worth noting, because they are the version-drift thesis in miniature: temperature is no longer accepted on current Claude models (a non-default value returns a 400), and the JSON-forcing trick this post originally used — prefilling the assistant turn with { — now returns a 400 as well. Structured outputs replace it.

import base64, os
from anthropic import Anthropic

client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

with open("invoice.pdf", "rb") as f:
    pdf_b64 = base64.standard_b64encode(f.read()).decode()

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=2048,
    messages=[{
        "role": "user",
        "content": [
            {"type": "document",
             "source": {"type": "base64", "media_type": "application/pdf", "data": pdf_b64}},
            {"type": "text",
             "text": "Extract the invoice fields defined in the schema. Use null for anything not present. Do not guess."},
        ],
    }],
    output_config={"format": {"type": "json_schema", "schema": INVOICE_SCHEMA}},
)

print(response.content[0].text)

Parsing an invoice with the Gemini API

pip install google-genai

Gemini also reads PDFs natively, via the Files API. Google now documents the Interactions API as the recommended surface for new development — the generateContent method most tutorials still show is the previous generation.

import os
from google import genai

client = genai.Client()  # reads GEMINI_API_KEY

doc = client.files.upload(file="invoice.pdf")

interaction = client.interactions.create(
    model="gemini-3.5-flash",
    input=[
        {"type": "text",
         "text": "Extract the invoice fields defined in the schema. Use null for anything not present. Do not guess."},
        {"type": "document", "uri": doc.uri, "mime_type": doc.mime_type},
    ],
    response_format={"type": "text", "mime_type": "application/json", "schema": INVOICE_SCHEMA},
)

print(interaction.output_text)

Gemini’s document handling has its own set of design constraints — page-to-token accounting, rate-limit tiers, prompt sensitivity — which we covered in depth in Gemini OCR: how well it works and how to use it.

Parsing an invoice with the Invofox API

curl -X POST https://api.invofox.com/v1/ingest/uploads \
  -H "x-api-key: $INVOFOX_API_KEY" \
  -F "[email protected]" \
  -F 'info={"type":"invoice"}'

No schema in the request and no prompt: the document type selects a schema that is already defined, validated and versioned. Processing is asynchronous — you subscribe to the document.processed webhook rather than blocking — and the delivered JSON has already been checked against the schema and your business rules.

{
  "type": "invoice",
  "data": {
    "invoiceNumber": "INV-2026-0841",
    "issueDate": "2026-07-02",
    "provider": { "name": "Acme Supplies GmbH", "taxId": "DE811570831" },
    "totalAmount": 1804.28,
    "currency": "EUR"
  }
}

The full quickstart in cURL, Python and Node is on the OCR API page. If what you want is the whole document as structured Markdown or JSON rather than named fields — for a RAG or search pipeline — that is the Document Parsing API instead.

What actually separates these, by design

This is the part a benchmark can’t tell you, and it doesn’t change with the next model release.

The output depends on the prompt

Run the same invoice through the same API three times with three prompts and you get three different answers. Not wrong answers — different ones. Here is the shape of what comes back:

(a) “Extract the data” — the model picks the keys, and copies the date format from the document:

{
  "Invoice Number": "INV-2026-0841",
  "Date": "02/07/2026",
  "Vendor": "Acme Supplies GmbH",
  "Total": "€1,804.28"
}

(b) “Extract invoice number, issue date, supplier name, total amount” — roughly your names now, but the total is still a string with a currency symbol in it, and the date is still whatever the document said:

{
  "invoice_number": "INV-2026-0841",
  "issue_date": "02/07/2026",
  "supplier_name": "Acme Supplies GmbH",
  "total_amount": "€1,804.28"
}

(c) The same prompt plus a JSON schema — your names, your types, and a shape that will be identical on the next document:

{
  "invoice_number": "INV-2026-0841",
  "issue_date": "2026-07-02",
  "supplier_name": "Acme Supplies GmbH",
  "total_amount": 1804.28,
  "currency": "EUR"
}

Note what changed between (b) and (c): nothing about the model’s reading of the document. It found the same values all three times. What moved was whether total_amount arrives as a number you can add up or a string you have to parse, and whether issue_date is sortable. Those are the differences that break a downstream system.

This is illustrative, not a scoreboard — it is not evidence that any model is worse than another, and it should never be turned into a percentage. It is evidence that with a raw model your schema lives in natural language, so every prompt edit is an untested deploy. A specialised API inverts that: the schema is a versioned artefact and the prompt doesn’t exist. Try it yourself before committing to a design; it’s a five-minute experiment and it reframes the problem.

Structured outputs guarantee the shape, not the truth

The single most useful sentence to internalise: a total the model invented, wrapped in a perfectly valid schema, is still an invented total. Google’s own structured output docs put it plainly — “always validate values in your application”. JSON mode moved the failure from a parse error you’d catch to a clean-looking value you won’t.

No confidence scores

Classic OCR engines return a per-word confidence; extraction products return one per field. A chat completion returns generated text. You can ask a model to rate its own certainty and it will produce a plausible-looking number, generated exactly the way it generates everything else — it is not an observation of internal state, and a human-review routing rule built on it is built on fiction. Without confidence there is no middle tier: every document is either fully trusted or fully reviewed.

No validation layer — failures are silent

When a parser can’t read a field it fails loudly. When a model can’t, it answers anyway. Feed it a rotated photo and you get fluent, well-formed JSON with a total that doesn’t equal the sum of the line items, a tax ID with a transposed digit, an issue date lifted from the due-date box. Nothing in the response distinguishes those from correct output.

The model underneath you keeps changing

The three model IDs in this post did not exist when it was first published, and the code that worked then returns a 400 now — temperature on Claude, the assistant-prefill JSON trick, Gemini’s older API surface. None of that was a breaking change you caused. Every model update silently changes extraction behaviour: fields that parsed one way parse another, edge cases shift, and your measured accuracy moves without a line of your code changing. Pinning a version buys months, not stability. We wrote up what Gemini’s deprecation cadence means in production.

Rate limits and per-token pricing

All three model APIs meter in tokens and tier their throughput. Documents make that awkward: a page is billed as image or text tokens, the prompt bills as text, the extracted JSON bills as output, and every retry or re-validation is a full-price second pass over the same document. Per-token pricing is excellent for chat and genuinely hard to forecast for “process every page exactly once at a known cost.”

What the specialised side contracts for

The mirror image: a fixed schema you version rather than a prompt you tune, per-field confidence, validation applied before delivery, a stable contract as models change underneath, and — with Perfect Docs Guaranteed — a per-field accuracy target in writing, with wrongly-extracted pages not billed. The trade-off is real and runs the other way too: less direct control over the model and the prompt.

What each option costs

Published list prices, August 2026. These are arithmetic on public price lists, not a measurement — the token counts depend entirely on your documents:

API	Input	Output	Source
gpt-5.4-mini	$0.75 / 1M tokens	$4.50 / 1M tokens	OpenAI pricing
claude-sonnet-5	$3.00 / 1M tokens	$15.00 / 1M tokens	Anthropic pricing
gemini-3.5-flash	$1.50 / 1M tokens	$9.00 / 1M tokens	Gemini pricing
Invofox	Per page processed, from 500 pages free	—	Pricing

A worked example so the shape is clear rather than the number: Gemini bills a document page at roughly 258 tokens of image input (documented here), so a 3-page invoice is about 774 input tokens plus your prompt, plus whatever the JSON output costs. That is cents. The reason per-token pricing still surprises teams is not the unit price — it is the retries, the re-validation after a model update, and the second pass with a refined prompt, each of which is another full-price document.

Three things will move these numbers, and all of them are yours: your documents (page count, scan quality, how much text sits on a page — a dense multi-page contract and a one-page receipt are not the same request), your prompts (a longer system prompt is billed on every call), and your retry rate (a 429, a second pass with a refined prompt, or a re-validation after a model update each costs a full document again). Run the code above on a sample of your own before you budget from a table.

We are deliberately not publishing latency figures, and the prices above are list prices rather than measurements. We have not run a metered benchmark across the four APIs, so there is no number here we would ask you to trust — anything we measured on our machine, on our documents, on a given afternoon would be a figure you could not reproduce, and a comparison table of unreproducible latencies is exactly the genre this post is trying not to be. Measuring it on your own pipeline takes about an hour and the result is worth more than ours would be.

How to measure accuracy yourself

The queries that bring people to this post ask about accuracy, and the honest answer to “which is more accurate” is that nobody can tell you — including every vendor with a number on their homepage, ours included. Accuracy is a property of a model and a prompt and a document set, and only one of those three is shared between you and whoever published the benchmark. The method:

1. Build a ground-truth set from real traffic. 50–100 documents, hand-verified field by field. Include the ugly tail — rotated photos, low-DPI faxes, stamps across amounts, multi-document PDFs. The tail is where the decision is actually made; on clean documents everything scores well and nothing is learned.
2. Score per field, not per document. “95% of documents mostly right” and “the total is right 95% of the time” are radically different claims for an AP workflow. Per-document scoring hides exactly the failures that cost money.
3. Count silent errors as their own metric. Not accuracy — how many wrong values arrived looking right. That is the number your downstream systems will consume without questioning, and it is the one that separates the options in this post.
4. Test prompt sensitivity. Run your set with two prompt variants. If the scores move, you have learned that your accuracy figure is a property of your prompt, and it will drift the next time someone edits it.
5. Re-run on every model version. Not optional — see the drift section above. A number measured against a model ID you no longer call is a historical artefact.
6. Price the whole loop. Inputs, outputs, retries, re-validation. Not the single-pass estimate.

Same discipline we apply internally, described in more depth in evaluation and accuracy.

When a raw LLM is enough

Genuinely, and often:

• Prototypes and feasibility checks. Nothing gets you from “can this be extracted at all?” to an answer faster.
• Low volume with a human in the loop. If someone reviews every result anyway, silent failures are caught by design and the validation gap never bites.
• One-off digitisation. A box of archive documents that becomes a spreadsheet once, not a pipeline that runs daily.
• Internal tools with soft failure costs. If a wrong value costs an eye-roll rather than a mis-payment, ship it.

If most of your workload looks like that list, use the API you already have credentials for and skip the rest of this decision.

When you need extraction you can contract for

The line is not “which model reads better” — they all read well. It is whether wrong data has a cost that someone eventually pays, and whether you want the surrounding layer (validation, confidence, review routing, correction feedback, stability across model updates) to be your team’s ongoing work or a vendor’s.

If that layer is what you’re weighing, the useful next step is not another benchmark: it is a performance report — your ground-truth set, per-field results, silent-error rate, measured on your documents. And if you’d rather run it yourself first, the API is free for 500 pages, which is enough for a real evaluation rather than a demo.