Fastest inference provider
What the benchmark found. Baseten recorded the lowest E2E P99 at 3.23 seconds on this run, with E2E P50 of 331 ms, task success of 95.8%, and failure rate of 0.0%.
What this page compares. This page asks one narrow question: which provider finishes a short inference request fastest for a single waiting user? It ranks the same GLM 5.3 Flash workload by measured end-to-end latency rather than advertised accelerator throughput.
How to read the result. E2E P99 is the primary decision metric because it captures the complete visible answer at the slow edge of traffic. Median latency is included to distinguish a consistently fast provider from one whose typical result is fast but whose tail stretches out.
Workload and configuration. Every provider received the same 600 deliberately easy structured tasks: 200 meeting-note lookups, 200 ticket-triage classifications, and 200 contract-term extractions. GLM 5.3 Flash ran with streaming enabled, reasoning low, temperature 0, top-p 1, a 256-token output ceiling, one attempt, a 20-second timeout, and per-provider concurrency one.
APIs used. The runner called each provider's OpenAI-compatible chat-completions surface. Baseten, Modal, Telnyx, Novita AI, Z.AI, Fireworks AI, Parasail, Together AI, Nebius, and DeepInfra used the model identifiers below; the links open the provider documentation used to verify each adapter.
- Baseten:
zai-org/GLM-5.3-Flash· API docs ↗ - Modal:
zai-org/GLM-5.3-Flash· API docs ↗ - Telnyx:
zai-org/GLM-5.3-Flash· API docs ↗ - Novita AI:
zai-org/glm-5.3-flash· API docs ↗ - Z.AI:
glm-5.3-flash· API docs ↗ - Fireworks AI:
accounts/fireworks/models/glm-5p3-flash· API docs ↗ - Parasail:
zai-org/GLM-5.3-Flash· API docs ↗ - Together AI:
zai-org/GLM-5.3-Flash· API docs ↗ - Nebius:
zai-org/GLM-5.3-Flash· API docs ↗ - DeepInfra:
zai-org/GLM-5.3-Flash· API docs ↗
Inference providers sorted by complete-response latency
Sorted by E2E P99, lower is better. The delta column is measured against the fastest P99 result on this run; task success and failure remain beside latency so missing work is not mistaken for speed.
| Provider | E2E P50 | E2E P95 | E2E P99 | Δ vs fastest P99 | Task success | Failure rate |
|---|---|---|---|---|---|---|
| Basetenzai-org/GLM-5.3-Flash ↗ | 331 ms | 2.29 s | 3.23 s | 0 ms | 95.8% | 0.0% |
| Modalzai-org/GLM-5.3-Flash ↗ | 574 ms | 1.90 s | 3.28 s | +50 ms | 99.3% | 0.0% |
| Telnyxzai-org/GLM-5.3-Flash ↗ | 669 ms | 2.33 s | 3.48 s | +250 ms | 99.5% | 0.0% |
| Novita AIzai-org/glm-5.3-flash ↗ | 1.74 s | 5.00 s | 7.25 s | +4020 ms | 94.2% | 4.0% |
| Z.AIglm-5.3-flash ↗ | 1.75 s | 4.83 s | 8.59 s | +5362 ms | 97.5% | 0.2% |
| Fireworks AIaccounts/fireworks/models/glm-5p3-flash ↗ | 1.95 s | 6.72 s | 10.5 s | +7259 ms | 99.5% | 0.0% |
| Parasailzai-org/GLM-5.3-Flash ↗ | 1.59 s | 7.74 s | 11.5 s | +8283 ms | 98.3% | 1.3% |
| Together AIzai-org/GLM-5.3-Flash ↗ | 609 ms | 3.66 s | 12.4 s | +9221 ms | 98.5% | 1.0% |
| Nebiuszai-org/GLM-5.3-Flash ↗ | 1.45 s | 7.45 s | 15.0 s | +11742 ms | 84.8% | 14.8% |
| DeepInfrazai-org/GLM-5.3-Flash ↗ | 1.83 s | 11.0 s | 17.1 s | +13842 ms | 95.5% | 4.0% |
Where an E2E latency ranking is useful
- Synchronous classification. Use E2E P95 when a router, policy check, or ticket-triage step blocks the next action and needs predictably complete JSON.
- Interactive assistants. Use TTFA P95 for the visible response start, then E2E P95 for the point at which downstream code can consume the complete object.
- Strict user-facing SLAs. Use E2E P99 with failure rate. A fast successful sample does not compensate for a request that timed out or returned an HTTP error.
Fast inference providers — questions this page answers
Which inference provider had the lowest E2E P99 in this benchmark?
Baseten recorded the lowest E2E P99 at 3.23 seconds on this run, with E2E P50 of 331 ms, task success of 95.8%, and failure rate of 0.0%.
Does the fastest median inference provider also win at the tail?
Baseten had the lowest E2E P50 at 331 ms, while Baseten had the lowest E2E P99 at 3.23 seconds. The same provider led both points of the distribution.
Why are failed inference calls excluded from latency percentiles?
Timeouts, HTTP errors, and transport failures have no completed visible answer and therefore no honest E2E latency value. They stay in the submitted-request denominator for failure rate and task success, which must be read beside the latency columns.
Read the complete inference benchmark
Metric definitions, request scheduling, task rubrics, failure handling, and the complete provider scorecard live on the Inference Benchmark →