Best inference provider for RAG and document lookup
What the benchmark found. Fireworks AI, Modal, Telnyx, and Z.AI tied for the task-success lead at 100.0%. The leading score represents approximately 200 exact successful results from 200 requests; operational failure rates were Fireworks AI 0.0%; Modal 0.0%; Telnyx 0.0%; Z.AI 0.0%. This is an answer-extraction result with relevant context already supplied, not an end-to-end retrieval benchmark.
What this page compares. The benchmark does not run retrieval infrastructure. It tests the inference step after a relevant document is already supplied: each provider reads a meeting note and returns one requested grounded fact. This page isolates those 200 lookup questions rather than claiming to rank embeddings, vector databases, or full RAG systems.
How to read the result. Use task success to compare answer extraction from provided context, then use E2E latency when that extraction blocks a user or agent. Do not use this table to choose a retriever: retrieval recall, reranking, citations, and context assembly are outside the measured path.
Workload and configuration. Every provider received the same 600 deliberately easy structured tasks: 200 meeting-note lookups, 200 ticket-triage classifications, and 200 contract-term extractions. GLM 5.3 Flash ran with streaming enabled, reasoning low, temperature 0, top-p 1, a 256-token output ceiling, one attempt, a 20-second timeout, and per-provider concurrency one.
Task-specific table. The table below uses only the published meeting-notes-lookup aggregate rather than the three-family pooled summary. Each provider therefore contributes 200 submitted requests to this view.
APIs used. The runner called each provider's OpenAI-compatible chat-completions surface. Fireworks AI, Modal, Telnyx, Z.AI, Baseten, Parasail, Together AI, DeepInfra, Novita AI, and Nebius used the model identifiers below; the links open the provider documentation used to verify each adapter.
- Fireworks AI:
accounts/fireworks/models/glm-5p3-flash· API docs ↗ - Modal:
zai-org/GLM-5.3-Flash· API docs ↗ - Telnyx:
zai-org/GLM-5.3-Flash· API docs ↗ - Z.AI:
glm-5.3-flash· API docs ↗ - Baseten:
zai-org/GLM-5.3-Flash· API docs ↗ - Parasail:
zai-org/GLM-5.3-Flash· API docs ↗ - Together AI:
zai-org/GLM-5.3-Flash· API docs ↗ - DeepInfra:
zai-org/GLM-5.3-Flash· API docs ↗ - Novita AI:
zai-org/glm-5.3-flash· API docs ↗ - Nebius:
zai-org/GLM-5.3-Flash· API docs ↗
Providers sorted by supplied-document lookup success
Only meeting-note lookup aggregates appear here. The table adds exact-success counts, failure rate, and E2E P99 to show answer yield and wait for 200 provided-context questions per provider.
| Provider / API | Task success | Exact successes | Failure rate | E2E P99 |
|---|---|---|---|---|
| Fireworks AIaccounts/fireworks/models/glm-5p3-flash ↗ | 100.0% | 200 / 200 | 0.0% | 7.54 s |
| Modalzai-org/GLM-5.3-Flash ↗ | 100.0% | 200 / 200 | 0.0% | 1.63 s |
| Telnyxzai-org/GLM-5.3-Flash ↗ | 100.0% | 200 / 200 | 0.0% | 2.92 s |
| Z.AIglm-5.3-flash ↗ | 100.0% | 200 / 200 | 0.0% | 7.57 s |
| Basetenzai-org/GLM-5.3-Flash ↗ | 99.5% | 199 / 200 | 0.0% | 1.43 s |
| Parasailzai-org/GLM-5.3-Flash ↗ | 99.0% | 198 / 200 | 1.0% | 11.1 s |
| Together AIzai-org/GLM-5.3-Flash ↗ | 98.5% | 197 / 200 | 1.5% | 12.8 s |
| DeepInfrazai-org/GLM-5.3-Flash ↗ | 98.0% | 196 / 200 | 2.0% | 15.7 s |
| Novita AIzai-org/glm-5.3-flash ↗ | 94.5% | 189 / 200 | 5.5% | 6.13 s |
| Nebiuszai-org/GLM-5.3-Flash ↗ | 84.5% | 169 / 200 | 15.5% | 18.6 s |
RAG stages this evidence can and cannot inform
- Answer extraction after retrieval. Use the result when your system has already selected a compact relevant passage and needs one exact fact.
- Document-grounded agent steps. Compare task success and completion latency when a supplied record controls the agent's next action.
- Not retrieval quality. Run a separate retrieval evaluation for embeddings, chunking, search recall, reranking, and citation coverage.
RAG inference and document-lookup boundaries
Which provider extracted the most correct facts from supplied meeting notes?
Fireworks AI, Modal, Telnyx, and Z.AI tied for the task-success lead at 100.0%. The leading score represents approximately 200 exact successful results from 200 requests; operational failure rates were Fireworks AI 0.0%; Modal 0.0%; Telnyx 0.0%; Z.AI 0.0%. This is an answer-extraction result with relevant context already supplied, not an end-to-end retrieval benchmark.
How does the benchmark normalize a document-lookup answer?
The lookup evaluator reconstructs the complete visible answer, parses the answer string, trims and collapses whitespace, and compares case-insensitively with the canonical answer and permitted aliases.
Does this page benchmark retrieval, embeddings, or a complete RAG pipeline?
This task supplies the relevant document directly in the prompt. It measures answer extraction after retrieval, not embedding quality, vector search recall, reranking, citation faithfulness, or an end-to-end RAG stack.
Read the complete inference benchmark
Metric definitions, request scheduling, task rubrics, failure handling, and the complete provider scorecard live on the Inference Benchmark →