Use cases

Pick your shape

Everything below is the same API call. What changes is which part of your payload is the material, what you put in the question, and which profile fits. If you have not read what to send where, start there — the rest of this page assumes it.

Ranges are what we measured on public benchmarks, not a promise — savings are a property of your text. The benchmarks page shows the spread by category.

A question about one long document

Contracts, policies, reports, medical notes. The document is the material; what you want to know is the question. Name the things you care about — compression protects what your question mentions.

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content":
        "What are the termination rights, the notice period, and any auto-renewal?"}],
    extra_body={"context": contract_text},
)

Asking for three named things instead of "tell me about this contract" is the single biggest lever you have. The question is the ruler.

RAG over retrieved chunks

Retrieval already over-fetches on purpose — you pull twenty chunks so the right three are certainly among them. That is exactly the shape compression is for: the other seventeen are paid for on every call.

chunks = retriever.search(user_question, k=20)

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": user_question}],
    extra_body={"context": "\n\n".join(c.text for c in chunks)},
)

An agent reading tool output

A test run, a stack trace, an API response, a CI log. Tool results are usually the largest thing in an agent's context and the most repetitive — hundreds of passing lines around the one that failed.

Nothing special to do: tool messages in the array are compressed in place, with the user's request as the question.

messages=[
    {"role": "user", "content": "Which test failed and why?"},
    {"role": "assistant", "tool_calls": [...]},
    {"role": "tool", "tool_call_id": "call_1", "content": pytest_output},
]

Measured on real production logs: the failing line, its assertion values and the file and line number survive, while the routine lines around them collapse.

A long-running chat

Every turn re-sends the whole history, so a long conversation pays for its own past on each call. Send the array as you already do — earlier turns are compressed against the newest question, and the newest question is never touched.

Note the interaction with provider prompt caching: compression rewrites the prefix, so a cached prefix will miss the first time it changes. On stable, long-lived prefixes measure both before choosing.

"List every…" extraction

CVs, invoices, rosters, audit findings — anything where a missing row is a wrong answer. Use exact: it drops nothing unless something equivalent stays, so a document of distinct facts comes through almost whole.

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content":
        "List every position with employer and dates. Do not skip any."}],
    extra_body={"context": cv_text, "compression_profile": "exact"},
)

A summary or overview

The best case for savings: transcripts and meeting notes repeat themselves, and auto removed 72% of them in our benchmark — more than on any other workload.

Measure it on your own material

Every alias has a -raw twin that runs the identical request with compression off. Same document, same question, same model — the only difference is us.

# Compressed
model="gpt-4o-mini"
# The control
model="gpt-4o-mini-raw"

Compare the answers and the usage.prompt_tokens on both. That is exactly how the benchmark was run, and it works on the free tier.