DeepInfra vs Together AI
What the benchmark found. Together AI had the lower end-to-end p99 latency at 12.45 seconds. Together AI had the higher task success rate at 98.5%. Both figures come from 600 submitted requests per provider. Measured across 10 LLM inference providers serving GLM 5.3 Flash, 600 requests each, Sep 3, 2026.
Providers measured. Baseten, DeepInfra, Fireworks AI, Modal, Nebius, Novita AI, Parasail, Telnyx, Together AI, and Z.AI, each serving GLM 5.3 Flash behind an OpenAI-compatible chat-completions endpoint.
What this page compares. DeepInfra and Together AI both serve GLM 5.3 Flash behind OpenAI-compatible endpoints and are compared here on the same 600 questions in the same rounds. Each axis reports the measured leader and the size of the gap.
How to read the result. Both providers received identical selected items in the same temporal rounds. Each axis below is computed independently; the faster complete response is not automatically the more accurate or reliable API.
Workload and configuration. Every provider received the same 600 deliberately easy structured tasks: 200 meeting-note lookups, 200 ticket-triage classifications, and 200 contract-term extractions. GLM 5.3 Flash ran with streaming enabled, reasoning low, temperature 0, top-p 1, a 256-token output ceiling, one attempt, a 20-second timeout, and per-provider concurrency one.
APIs used. The runner called each provider's OpenAI-compatible chat-completions surface with the model identifiers below; the links open the provider documentation used to verify each adapter.
- DeepInfra:
zai-org/GLM-5.3-Flash· API docs ↗ - Together AI:
zai-org/GLM-5.3-Flash· API docs ↗
DeepInfra vs Together AI: which is faster, and which is more accurate?
Lower wins for latency and failure rate; higher wins for tokens per second and task success. The gap is shown in the native unit of each metric.
| Metric | DeepInfra | Together AI | Measured leader | Absolute gap |
|---|---|---|---|---|
| End-to-end p99 | 17.1 s | 12.4 s | Together AI | 4621 ms |
| End-to-end p95 | 11.0 s | 3.66 s | Together AI | 7345 ms |
| Time to first token p95 | 6.41 s | 2.26 s | Together AI | 4146 ms |
| Tokens per second p50 | 69.4 tok/s | 151.6 tok/s | Together AI | 82.2 tok/s |
| Tokens per second p95 | 9.0 tok/s | 34.0 tok/s | Together AI | 25.1 tok/s |
| Task success | 95.5% | 98.5% | Together AI | 3.0 pp |
| Failure rate | 4.0% | 1.0% | Together AI | 3.0 pp |
Provider by provider, in the order above
- DeepInfra: 1.83 seconds median, 17.07 seconds end-to-end p99, time to first token p95 6.41 seconds, 69.4 tokens per second, 95.5% task success, 4.0% failures.
- Together AI: 609 ms median, 12.45 seconds end-to-end p99, time to first token p95 2.26 seconds, 151.6 tokens per second, 98.5% task success, 1.0% failures.
Where should I compare DeepInfra with Together AI?
- Cost-sensitive structured pipelines. Start with task success, then compare end-to-end p95 and p99 because downstream code waits for the complete structured result.
- Streamed assistant responses. Compare time to first token p95 for response start and tokens per second p95 for the slow throughput tail.
- Blocking classification calls. Use end-to-end p99 with failure rate when this model call sits directly on a user-visible critical path.
How is LLM inference provider latency and accuracy measured here?
- Same model. Every endpoint serves GLM 5.3 Flash; the provider layer is the only variable.
- Same request. One-turn, non-tool chat completion in JSON-object mode, streaming on, temperature 0, top-p 1, reasoning low, 256-token ceiling, 20-second timeout, concurrency one.
- Client clock, pinned tokenizer. Time to first token, end-to-end latency and tokens per second come from the client; visible answer tokens are counted with a pinned
tiktoken o200k_basetokenizer, never provider-reported usage. - Exact task success. An answer counts only when it is schema-valid and exactly correct under the task rubric. Timeouts, HTTP errors and malformed responses stay in the denominator.
- Serverless endpoints only. Dedicated capacity is outside the comparison. Runner and per-request records: openbenchmarks-labs/inference ↗.
DeepInfra vs Together AI: direct questions
Is DeepInfra faster than Together AI at end-to-end p99?
Together AI had the lower end-to-end p99 latency at 12.45 seconds. Together AI had the higher task success rate at 98.5%. Both figures come from 600 submitted requests per provider.
Which of DeepInfra and Together AI returned more exact structured answers?
DeepInfra task success was 95.5% versus 98.5% for Together AI. Operational failure rates were 4.0% and 1.0% respectively.
How were the DeepInfra and Together AI endpoints configured for this comparison?
DeepInfra used zai-org/GLM-5.3-Flash; Together AI used zai-org/GLM-5.3-Flash. Both received the logical low-reasoning setting, streaming JSON-object output, temperature 0, and the same 600 questions.
Same run, other questions
Read the complete LLM inference provider benchmark
Metric definitions, request scheduling, task rubrics, failure handling, and the complete provider scorecard live on the LLM Inference Provider Benchmark →