--- title: "DeepInfra vs Together AI 2026: Inference Compared | Openbenchmarks" description: "DeepInfra vs Together AI for LLM inference: end-to-end p99, tokens per second, task success and failures on the same 600 requests. Open source." url: https://openbenchmarks.com/inference/deepinfra-vs-together-ai --- [](https://openbenchmarks.com/) [← LLM inference provider benchmark](https://openbenchmarks.com/inference) [benchmarks](https://openbenchmarks.com/) / [inference](https://openbenchmarks.com/inference) / deepinfra vs together ai LLM inference API · measured head-to-head · measured Sep 3, 2026 # DeepInfra vs Together AI **What the benchmark found.** Together AI had the lower end-to-end p99 latency at 12.45 seconds. Together AI had the higher task success rate at 98.5%. Both figures come from 600 submitted requests per provider. Measured across 10 LLM inference providers serving GLM 5.3 Flash, 600 requests each, Sep 3, 2026. **Providers measured.** Baseten, DeepInfra, Fireworks AI, Modal, Nebius, Novita AI, Parasail, Telnyx, Together AI, and Z.AI, each serving GLM 5.3 Flash behind an OpenAI-compatible chat-completions endpoint. **What this page compares.** DeepInfra and Together AI both serve GLM 5.3 Flash behind OpenAI-compatible endpoints and are compared here on the same 600 questions in the same rounds. Each axis reports the measured leader and the size of the gap. **How to read the result.** Both providers received identical selected items in the same temporal rounds. Each axis below is computed independently; the faster complete response is not automatically the more accurate or reliable API. **Workload and configuration.** Every provider received the same 600 deliberately easy structured tasks: 200 meeting-note lookups, 200 ticket-triage classifications, and 200 contract-term extractions. GLM 5.3 Flash ran with streaming enabled, reasoning low, temperature 0, top-p 1, a 256-token output ceiling, one attempt, a 20-second timeout, and per-provider concurrency one. **APIs used.** The runner called each provider's OpenAI-compatible chat-completions surface with the model identifiers below; the links open the provider documentation used to verify each adapter. * **DeepInfra:** `zai-org/GLM-5.3-Flash` · [API docs ↗](https://docs.deepinfra.com/api-reference/chat-completions/openai-chat-completions) * **Together AI:** `zai-org/GLM-5.3-Flash` · [API docs ↗](https://docs.together.ai/docs/inference/chat/overview) \[01\] measured comparison ## DeepInfra vs Together AI: which is faster, and which is more accurate? Lower wins for latency and failure rate; higher wins for tokens per second and task success. The gap is shown in the native unit of each metric. benchmarks/inference/deepinfra-vs-together-ai reviewed run __DeepInfra and Together AI measured head to head__ | Metric | DeepInfra | Together AI | Measured leader | Absolute gap | | --- | --- | --- | --- | --- | | End-to-end p99 | 17.1 s | 12.4 s | Together AI | 4621 ms | | End-to-end p95 | 11.0 s | 3.66 s | Together AI | 7345 ms | | Time to first token p95 | 6.41 s | 2.26 s | Together AI | 4146 ms | | Tokens per second p50 | 69.4 tok/s | 151.6 tok/s | Together AI | 82.2 tok/s | | Tokens per second p95 | 9.0 tok/s | 34.0 tok/s | Together AI | 25.1 tok/s | | Task success | 95.5% | 98.5% | Together AI | 3.0 pp | | Failure rate | 4.0% | 1.0% | Together AI | 3.0 pp | ### Provider by provider, in the order above * DeepInfra: 1.83 seconds median, 17.07 seconds end-to-end p99, time to first token p95 6.41 seconds, 69.4 tokens per second, 95.5% task success, 4.0% failures. * Together AI: 609 ms median, 12.45 seconds end-to-end p99, time to first token p95 2.26 seconds, 151.6 tokens per second, 98.5% task success, 1.0% failures. \[02\] workflow fit ## Where should I compare DeepInfra with Together AI? * **Cost-sensitive structured pipelines.** Start with task success, then compare end-to-end p95 and p99 because downstream code waits for the complete structured result. * **Streamed assistant responses.** Compare time to first token p95 for response start and tokens per second p95 for the slow throughput tail. * **Blocking classification calls.** Use end-to-end p99 with failure rate when this model call sits directly on a user-visible critical path. \[03\] how it was measured ## How is LLM inference provider latency and accuracy measured here? * **Same model.** Every endpoint serves GLM 5.3 Flash; the provider layer is the only variable. * **Same request.** One-turn, non-tool chat completion in JSON-object mode, streaming on, temperature 0, top-p 1, reasoning low, 256-token ceiling, 20-second timeout, concurrency one. * **Client clock, pinned tokenizer.** Time to first token, end-to-end latency and tokens per second come from the client; visible answer tokens are counted with a pinned `tiktoken o200k_base` tokenizer, never provider-reported usage. * **Exact task success.** An answer counts only when it is schema-valid and exactly correct under the task rubric. Timeouts, HTTP errors and malformed responses stay in the denominator. * **Serverless endpoints only.** Dedicated capacity is outside the comparison. Runner and per-request records: [openbenchmarks-labs/inference ↗](https://github.com/openbenchmarks-labs/inference). \[04\] faq ## DeepInfra vs Together AI: direct questions ### Is DeepInfra faster than Together AI at end-to-end p99? Together AI had the lower end-to-end p99 latency at 12.45 seconds. Together AI had the higher task success rate at 98.5%. Both figures come from 600 submitted requests per provider. ### Which of DeepInfra and Together AI returned more exact structured answers? DeepInfra task success was 95.5% versus 98.5% for Together AI. Operational failure rates were 4.0% and 1.0% respectively. ### How were the DeepInfra and Together AI endpoints configured for this comparison? DeepInfra used zai-org/GLM-5.3-Flash; Together AI used zai-org/GLM-5.3-Flash. Both received the logical low-reasoning setting, streaming JSON-object output, temperature 0, and the same 600 questions. \[05\] related comparisons ## Same run, other questions * [LLM inference provider benchmark: complete scorecard and methodology →](https://openbenchmarks.com/inference) * [Baseten vs Fireworks AI →](https://openbenchmarks.com/inference/baseten-vs-fireworks-ai) * [Baseten vs Together AI →](https://openbenchmarks.com/inference/baseten-vs-together-ai) * [Fireworks AI vs Together AI →](https://openbenchmarks.com/inference/fireworks-ai-vs-together-ai) * [Baseten alternatives, benchmarked →](https://openbenchmarks.com/inference/baseten-alternatives) * [Together AI alternatives, benchmarked →](https://openbenchmarks.com/inference/together-ai-alternatives) * [Fireworks AI alternatives, benchmarked →](https://openbenchmarks.com/inference/fireworks-ai-alternatives) \[06\] source benchmark ## Read the complete LLM inference provider benchmark Metric definitions, request scheduling, task rubrics, failure handling, and the complete provider scorecard live on the [LLM Inference Provider Benchmark →](https://openbenchmarks.com/inference) Openbenchmarks [inference benchmark](https://openbenchmarks.com/inference) [all benchmarks](https://openbenchmarks.com/) © 2026 Openbenchmarks