Baseten vs Together AI
What the benchmark found. Baseten had the lower E2E P99 at 3.23 seconds. Together AI had the higher task success rate at 98.5%. Both figures come from 600 submitted requests per provider.
What this page compares. Baseten and Together AI are compared as delivery layers for the same GLM model. Completion latency, first visible answer, token throughput, task success, and failures are calculated from their shared 600-question schedule.
How to read the result. Both providers received identical selected items in the same temporal rounds. Each axis below is computed independently; the faster complete response is not automatically the more accurate or reliable API.
Workload and configuration. Every provider received the same 600 deliberately easy structured tasks: 200 meeting-note lookups, 200 ticket-triage classifications, and 200 contract-term extractions. GLM 5.3 Flash ran with streaming enabled, reasoning low, temperature 0, top-p 1, a 256-token output ceiling, one attempt, a 20-second timeout, and per-provider concurrency one.
APIs used. The runner called each provider's OpenAI-compatible chat-completions surface. Baseten and Together AI used the model identifiers below; the links open the provider documentation used to verify each adapter.
- Baseten:
zai-org/GLM-5.3-Flash· API docs ↗ - Together AI:
zai-org/GLM-5.3-Flash· API docs ↗
Baseten and Together AI, axis by axis
Lower wins for latency and failure rate; higher wins for token generation speed and task success. The gap is shown in the native unit of each metric.
| Metric | Baseten | Together AI | Measured leader | Absolute gap |
|---|---|---|---|---|
| E2E P99 | 3.23 s | 12.4 s | Baseten | 9221 ms |
| E2E P95 | 2.29 s | 3.66 s | Baseten | 1377 ms |
| TTFA P95 | 1.16 s | 2.26 s | Baseten | 1099 ms |
| Generation P50 | 240.5 tok/s | 151.6 tok/s | Baseten | 88.8 tok/s |
| Generation P95 | 61.6 tok/s | 34.0 tok/s | Baseten | 27.6 tok/s |
| Task success | 95.8% | 98.5% | Together AI | 2.7 pp |
| Failure rate | 0.0% | 1.0% | Baseten | 1.0 pp |
Where to compare Baseten with Together AI
- Agent tool-result generation. Start with task success, then compare E2E P95/P99 because downstream code waits for the complete structured result.
- Live ticket classification. Compare TTFA P95 for response start and generation P95 for the slow throughput tail.
- Meeting-note answer lookup. Use E2E P99 with failure rate when this model call sits directly on a user-visible critical path.
Baseten vs Together AI — direct questions
Baseten vs Together AI: which had the lower complete-answer tail?
Baseten had the lower E2E P99 at 3.23 seconds. Together AI had the higher task success rate at 98.5%. Both figures come from 600 submitted requests per provider.
Did Together AI or Baseten start visible answers sooner?
Baseten had the lower TTFA P95 at 1.16 seconds. The head-to-head table keeps E2E tail latency beside it because a fast visible start does not guarantee a fast complete answer.
What should an agent builder inspect beyond the Baseten–Together median?
Beyond median latency, inspect E2E P99 for rare long waits, task success for usable structured results, and failure rate for requests that never completed normally.
Read the complete inference benchmark
Metric definitions, request scheduling, task rubrics, failure handling, and the complete provider scorecard live on the Inference Benchmark →