Compression profiles

Picking a profile

Three profiles, switchable per call. They differ in how much of your document may go and how that limit is decided — not in speed: measured end-to-end, the three are indistinguishable from each other, and on large documents all of them answer sooner than sending the full text, because the model has less to read. Leave compression_profile unset and you get auto — and if you are not sure what to pick, stay on auto: it adapts per request, so you cannot get it wrong.

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": question}],
    extra_body={"compression_profile": "exact"},
)

Does compression change the answer?

This is the only question that matters, so it is measured directly rather than inferred from how much text survived. 218 cases from public datasets where a single wrong answer counts as a failure — exact contract spans (CUAD), negation-heavy NDA entailment (ContractNLI), biomedical decisions (PubMedQA), multi-hop questions (HotpotQA distractor), exhaustive "find every id" retrieval, code-file location (SWE-bench) and a real production log. Same document, same question, same model, temperature 0.

One answer out of 218 separates the default from sending the whole document, and the 95% interval on that difference runs from −3.2 to +2.2 percentage points — it is noise, not a cost. Four of the seven categories came out identical under every profile, including the exhaustive "return all of them" set, where all 20 cases passed everywhere. On negation, auto and exact each answered one case that the uncompressed document got wrong. Compression is buying you tokens, not costing you answers — and where a document genuinely cannot be compressed safely, the profiles that know it say so by compressing less.

What to expect from your own documents

Savings are not a property of the service — they are a property of your text. Repetition is what can be removed; a document where every line says something new has nothing safe to give up. Measured on real documents. The middle column is the deepest setting we have ever run, so read it as a ceiling — fast lands below it and auto below that:

If you are extracting records — every position, every line item, every finding — use exact and expect little compression. That is the honest outcome, and it is cheaper than re-running a job that silently lost half its rows.

Your prompt matters more than the profile

Compression scores every passage against what it believes you are asking. A precise question makes that judgement easy and a vague one makes it guesswork, so the same document can compress well or badly depending on how you ask. Three habits pay for themselves:

# Weaker: the question is buried and unspecific
client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": long_contract + "\n\nThoughts?"}],
)

# Better: separate document from question, and be specific
client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user",
               "content": "What is the termination notice period in clause 14.2?"}],
    extra_body={"context": long_contract, "compression_profile": "exact"},
)

If a document is mostly unique facts — a log where every line has its own id — there is little any profile is allowed to drop, and savings will be modest whatever you pick. That is the compressor refusing to guess, and it is the behaviour you want.

Compressing output tokens

Output tokens often cost more than input. Set compress_output to true when you want fewer completion tokens on that request while keeping answer quality. It works with any model you call through CompresLM (the numbers on the home page are examples from tests, not a limited model list). Set it to false (or omit it) for the default behavior (input compression only).

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Which store is cheaper after tax?"}],
    extra_body={
        "context": long_document,
        "compress_output": True,  # opt-in output savings
    },
)

curl https://compreslm.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_COMPRESLM_TOKEN" \
  -H "X-LLM-API-Key: YOUR_PROVIDER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "compress_output": true,
    "context": "...long document...",
    "messages": [{"role":"user","content":"What fee is owed, and when does termination take effect?"}]
  }'

On controlled paired tests (same cases, same gold answers), enabling output compression kept 100% answer accuracy while cutting mean completion tokens by about 60% on GPT-4o, 68% on Claude Sonnet, and 26% on o3-mini versus a verbose baseline — example models only; the same flag applies to other models you route through CompresLM. See the charts on the home page.

Billing: input compression alone is billed at 20% of measured dollar savings on the input. When compress_output is true, our fee is 50% of those same input-token dollar savings (the fee base stays input savings — so the amount is always computable). Your dashboard also shows output tokens saved for those calls (scaled from observed completions using ~60% average reduction) — that display number is not used for fees. Details: How billing works · output savings on the dashboard.