Public benchmark set

Prompt compression with deliberate passthrough, measured end to end.

This page uses a curated launch benchmark slice from the broader Tokelang Lite evaluation corpus: strong compaction examples across incident response, legal, sales, analytics, research, and operations, plus two deliberate passthrough cases where exact wording should stay intact.

Integration docs Try the API

Curated prompts

Public launch set selected from the larger internal benchmark corpus.

Compiled savings

34.29%

Weighted token savings on the prompts that compiled to compact Tokelang output.

Average latency

9.27 ms

Client-measured end-to-end HTTPS latency through the public endpoint.

Mode split

8 / 2

Strong compiled examples plus two exactness-sensitive passthrough examples.

Method

POST /v1/compile with expand=false, return_ir=false, metrics=false. Token counts use tiktoken cl100k_base, and latency uses Client-measured end-to-end HTTPS latency to the live public deployment from the public benchmark runner.

Why passthrough exists

Passthrough is not a miss. It is the compiler choosing fidelity over compression for code, exact math, row-heavy inputs, and contract-sensitive prompts.

Sample method

Each prompt was measured with 1 warmup + 5 measured requests. The table below shows average latency and p95 latency per prompt.

Representative examples

What the benchmark set actually contains

These examples are selected to show both sides of the product: compaction on stronger workflow-style prompts, and passthrough when exact wording should not be compressed.

pb-06tokelangsalesextraction-plus-strategy

Sales Follow-Up Strategy

35 to 24 tokens, 31.43% savings, 9.54 ms average latency.

Original prompt

Turn the sales call notes into follow-up actions.

Extract buyer pain points, timeline, budget signals, procurement blockers, and return a short account strategy memo for the account executive.

Compiled output

Turn sales call notes into follow-up actions Extract buyer pain points timeline budget signals procurement blockers return short account strategy memo account executive

pb-08tokelanglegaldecision-tree

Contract Exception Memo

53 to 33 tokens, 37.74% savings, 8.92 ms average latency.

Original prompt

Contract review workflow.

Step 1: Read the payment terms.
Step 2: If the side letter changes payment timing, surface the exception.
Step 3: Otherwise compare warranty and indemnity language.
Step 4: Return a short legal memo for procurement.

Compiled output

process
default simple
analyze payment terms
analyze side letter changes payment timing surface exception
else compare warranty indemnity language
output
return legal memo for procurement brief

pb-17passthroughtranslationcontract-sensitive

Exact Translation

25 to 25 tokens, 0.00% savings, 9.09 ms average latency.

Original prompt

Translate this customer message into German.

Keep the tone polite, preserve the order number exactly, and return only the translated text.

Returned output

Translate this customer message into German.

Keep the tone polite, preserve the order number exactly, and return only the translated text.

pb-21passthroughmathexactness-sensitive

Exact Symbolic Math

42 to 42 tokens, 0.00% savings, 8.08 ms average latency.

Original prompt

Solve this exactly.

Let f(x) = x^4 - 6x^2 + 8x - 3.
Find all critical points, classify them, and keep the symbolic derivation explicit.

Returned output

Solve this exactly.

Let f(x) = x^4 - 6x^2 + 8x - 3.
Find all critical points, classify them, and keep the symbolic derivation explicit.

Full table

Curated launch benchmark prompts

This public table keeps the cleaner launch-facing examples and two deliberate passthrough cases. Expected mode reflects the benchmark design intent. Actual mode reflects what the current benchmarked build returned during measurement.

Prompt	Expected	Actual	Original	Output	Savings	Avg latency	p95 latency
Grant Proposal Review pb-16 · grants	tokelang	tokelang	28	15	46.43%	9.24 ms	11.54 ms
Exec Meeting Summary pb-04 · operations	tokelang	tokelang	31	19	38.71%	9.17 ms	11.30 ms
Contract Exception Memo pb-08 · legal	tokelang	tokelang	53	33	37.74%	8.92 ms	10.54 ms
Security Alert Triage pb-14 · security	tokelang	tokelang	42	27	35.71%	9.48 ms	10.64 ms
Lesson Plan pb-10 · education	tokelang	tokelang	36	24	33.33%	11.57 ms	19.27 ms
User Research Themes pb-15 · user-research	tokelang	tokelang	36	24	33.33%	9.71 ms	10.61 ms
Sales Follow-Up Strategy pb-06 · sales	tokelang	tokelang	35	24	31.43%	9.54 ms	10.48 ms
Incident Response Narrative pb-02 · incident-response	tokelang	tokelang	54	41	24.07%	7.94 ms	9.23 ms
Exact Translation pb-17 · translation	passthrough	passthrough	25	25	0.00%	9.09 ms	10.15 ms
Exact Symbolic Math pb-21 · math	passthrough	passthrough	42	42	0.00%	8.08 ms	9.18 ms