Fastest LLM API provider: time to first token and complete-response latency
What the benchmark found. Baseten was the fastest complete-response LLM API at end-to-end p99 of 3.23 seconds. Its time to first token p95 was 1.16 seconds, and its operational failure rate was 0.0% across 600 submissions. Measured across 10 LLM inference providers serving GLM 5.3 Flash, 600 requests each, Sep 3, 2026.
Providers measured. Baseten, DeepInfra, Fireworks AI, Modal, Nebius, Novita AI, Parasail, Telnyx, Together AI, and Z.AI, each serving GLM 5.3 Flash behind an OpenAI-compatible chat-completions endpoint.
What this page compares. For an application calling an LLM API, the useful finish line is often not the first streamed character but the complete JSON result that unlocks the next step. This comparison measures that API round trip across identical lookup, routing, and extraction prompts.
How to read the result. The table keeps API startup and completion in view together: time to first token p95 shows when visible output begins, while end-to-end p95 and p99 show when the application has the complete answer. Every row links to the API documentation and model key used in the run.
Workload and configuration. Every provider received the same 600 deliberately easy structured tasks: 200 meeting-note lookups, 200 ticket-triage classifications, and 200 contract-term extractions. GLM 5.3 Flash ran with streaming enabled, reasoning low, temperature 0, top-p 1, a 256-token output ceiling, one attempt, a 20-second timeout, and per-provider concurrency one.
APIs used. The runner called each provider's OpenAI-compatible chat-completions surface with the model identifiers below; the links open the provider documentation used to verify each adapter.
- Baseten:
zai-org/GLM-5.3-Flash· API docs ↗ - Modal:
zai-org/GLM-5.3-Flash· API docs ↗ - Telnyx:
zai-org/GLM-5.3-Flash· API docs ↗ - Novita AI:
zai-org/glm-5.3-flash· API docs ↗ - Z.AI:
glm-5.3-flash· API docs ↗ - Fireworks AI:
accounts/fireworks/models/glm-5p3-flash· API docs ↗ - Parasail:
zai-org/GLM-5.3-Flash· API docs ↗ - Together AI:
zai-org/GLM-5.3-Flash· API docs ↗ - Nebius:
zai-org/GLM-5.3-Flash· API docs ↗ - DeepInfra:
zai-org/GLM-5.3-Flash· API docs ↗
Which LLM API provider has the lowest time to first token and end-to-end p99?
The table is API-focused: model key, documentation, time to first token tail, complete-answer tail, and operational failures. It does not repeat the main benchmark's full streaming matrix.
| Provider / API | End-to-end p95 | End-to-end p99 | Δ vs fastest p99 | TTFT p95 | Task success | Failure rate |
|---|---|---|---|---|---|---|
| Basetenzai-org/GLM-5.3-Flash ↗ | 2.29 s | 3.23 s | 0 ms | 1.16 s | 95.8% | 0.0% |
| Modalzai-org/GLM-5.3-Flash ↗ | 1.90 s | 3.28 s | +50 ms | 1.09 s | 99.3% | 0.0% |
| Telnyxzai-org/GLM-5.3-Flash ↗ | 2.33 s | 3.48 s | +250 ms | 1.34 s | 99.5% | 0.0% |
| Novita AIzai-org/glm-5.3-flash ↗ | 5.00 s | 7.25 s | +4020 ms | 3.41 s | 94.2% | 4.0% |
| Z.AIglm-5.3-flash ↗ | 4.83 s | 8.59 s | +5362 ms | 3.14 s | 97.5% | 0.2% |
| Fireworks AIaccounts/fireworks/models/glm-5p3-flash ↗ | 6.72 s | 10.5 s | +7259 ms | 5.83 s | 99.5% | 0.0% |
| Parasailzai-org/GLM-5.3-Flash ↗ | 7.74 s | 11.5 s | +8283 ms | 5.54 s | 98.3% | 1.3% |
| Together AIzai-org/GLM-5.3-Flash ↗ | 3.66 s | 12.4 s | +9221 ms | 2.26 s | 98.5% | 1.0% |
| Nebiuszai-org/GLM-5.3-Flash ↗ | 7.45 s | 15.0 s | +11742 ms | 7.11 s | 84.8% | 14.8% |
| DeepInfrazai-org/GLM-5.3-Flash ↗ | 11.0 s | 17.1 s | +13842 ms | 6.41 s | 95.5% | 4.0% |
Provider by provider, in the order above
- Baseten: 331 ms median, 3.23 seconds end-to-end p99, time to first token p95 1.16 seconds, 240.5 tokens per second, 95.8% task success, 0.0% failures.
- Modal: 574 ms median, 3.28 seconds end-to-end p99, time to first token p95 1.09 seconds, 222.2 tokens per second, 99.3% task success, 0.0% failures.
- Telnyx: 669 ms median, 3.48 seconds end-to-end p99, time to first token p95 1.34 seconds, 184.3 tokens per second, 99.5% task success, 0.0% failures.
- Novita AI: 1.74 seconds median, 7.25 seconds end-to-end p99, time to first token p95 3.41 seconds, 60.6 tokens per second, 94.2% task success, 4.0% failures.
- Z.AI: 1.75 seconds median, 8.59 seconds end-to-end p99, time to first token p95 3.14 seconds, 56.3 tokens per second, 97.5% task success, 0.2% failures.
- Fireworks AI: 1.95 seconds median, 10.49 seconds end-to-end p99, time to first token p95 5.83 seconds, 58.2 tokens per second, 99.5% task success, 0.0% failures.
- Parasail: 1.59 seconds median, 11.51 seconds end-to-end p99, time to first token p95 5.54 seconds, 52.4 tokens per second, 98.3% task success, 1.3% failures.
- Together AI: 609 ms median, 12.45 seconds end-to-end p99, time to first token p95 2.26 seconds, 151.6 tokens per second, 98.5% task success, 1.0% failures.
- Nebius: 1.45 seconds median, 14.97 seconds end-to-end p99, time to first token p95 7.11 seconds, 269.3 tokens per second, 84.8% task success, 14.8% failures.
- DeepInfra: 1.83 seconds median, 17.07 seconds end-to-end p99, time to first token p95 6.41 seconds, 69.4 tokens per second, 95.5% task success, 4.0% failures.
When does LLM API completion latency matter more than the first token?
- Tool-result generation. Choose on end-to-end p95 when an agent cannot make its next tool call until a schema-valid JSON result is complete.
- Live support routing. Read time to first token only if the text is shown to an operator; read end-to-end latency when category and priority feed an automated queue.
- Inline document extraction. Use end-to-end p99 and task success together when a user is waiting for contract fields to populate a form.
How is LLM inference provider latency and accuracy measured here?
- Same model. Every endpoint serves GLM 5.3 Flash; the provider layer is the only variable.
- Same request. One-turn, non-tool chat completion in JSON-object mode, streaming on, temperature 0, top-p 1, reasoning low, 256-token ceiling, 20-second timeout, concurrency one.
- Client clock, pinned tokenizer. Time to first token, end-to-end latency and tokens per second come from the client; visible answer tokens are counted with a pinned
tiktoken o200k_basetokenizer, never provider-reported usage. - Exact task success. An answer counts only when it is schema-valid and exactly correct under the task rubric. Timeouts, HTTP errors and malformed responses stay in the denominator.
- Serverless endpoints only. Dedicated capacity is outside the comparison. Runner and per-request records: openbenchmarks-labs/inference ↗.
Choosing a fast LLM API: measured answers
What was the fastest LLM API for a complete structured response?
Baseten was the fastest complete-response LLM API at end-to-end p99 of 3.23 seconds. Its time to first token p95 was 1.16 seconds, and its operational failure rate was 0.0% across 600 submissions.
Which API model identifiers were used for the LLM latency test?
Baseten: zai-org/GLM-5.3-Flash; Modal: zai-org/GLM-5.3-Flash; Telnyx: zai-org/GLM-5.3-Flash; Novita AI: zai-org/glm-5.3-flash; Z.AI: glm-5.3-flash; Fireworks AI: accounts/fireworks/models/glm-5p3-flash; Parasail: zai-org/GLM-5.3-Flash; Together AI: zai-org/GLM-5.3-Flash; Nebius: zai-org/GLM-5.3-Flash; DeepInfra: zai-org/GLM-5.3-Flash. Each identifier links to the provider documentation used for this comparison where available.
Should an agent workflow optimize time to first token or end-to-end latency?
Optimize end-to-end latency when the next agent step requires the complete JSON object. Optimize time to first token when visible text is rendered immediately and the user benefits before the structured response finishes.
Same run, other questions
- LLM inference provider benchmark: complete scorecard and methodology →
- Fastest LLM inference provider: time to first token, tokens per second and end-to-end latency →
- Lowest-latency LLM inference provider: p99 tail analysis →
- Fastest LLM inference API by tokens per second →
- Lowest-latency LLM API for synchronous workflows with an SLA →
- Best LLM inference provider: speed, accuracy and failures compared →
- Fireworks AI vs Together AI →
Read the complete LLM inference provider benchmark
Metric definitions, request scheduling, task rubrics, failure handling, and the complete provider scorecard live on the LLM Inference Provider Benchmark →