# Openbenchmarks

> Independent benchmarks for build-vs-buy decisions on AI agent tooling. Verifiable, reproducible and open source.

Last updated: 2026-10-15T03:00:00.000Z
Canonical: https://openbenchmarks.com

## What this is

Openbenchmarks is a public benchmark hub. Methodology is public; every metric is externally validated; there is no internal "ground truth" or cohort-consensus voting that could reward conformity.

## Live benchmarks

Winners per axis, as shown on the homepage. Every page has a markdown copy at the same URL plus `.md` (or send `Accept: text/markdown`).

### Web Search for AI

- Page: https://openbenchmarks.com/web-search

Web search APIs benchmarked across three tasks: factual lookup, hard retrieval, and multi-hop search.

| Setup | Winner on | Provider | Result |
| --- | --- | --- | --- |
| factual lookup | Most accurate | Exa fast | 99.3% |
| factual lookup | Fastest | Parallel turbo | 348 ms |
| factual lookup | Lowest cost | TinyFish | Free |
| hard retrieval · search only | Most tasks | Perplexity | 77.3% |
| hard retrieval · search only | Fastest | Perplexity | 18.3 s |
| hard retrieval · search only | Lowest cost | TinyFish | $0.035 |
| hard retrieval · search + fetch | Most tasks | Exa deep | 83.0% |
| hard retrieval · search + fetch | Fastest | Perplexity | 21.5 s |
| hard retrieval · search + fetch | Lowest cost | TinyFish | $0.050 |
| multi-hop · search only | Leads F1 | Parallel basic | 46.5 |
| multi-hop · search only | Fastest | Brave Search | 43.5 s |
| multi-hop · search only | Lowest cost | TinyFish | $0.212 |
| multi-hop · search + fetch | Leads F1 | Exa deep | 48.2 |
| multi-hop · search + fetch | Fastest | Brave Search | 45.1 s |
| multi-hop · search + fetch | Lowest cost | TinyFish | $0.219 |

### Multi Turn Company Search

- Page: https://openbenchmarks.com/multi-turn-company-search

Search APIs tested as tools for a research agent finding complete company sets across three- and four-part constraints. Ranked on precision, recall, F1, exact-set accuracy, latency, and cost.

| Setup | Winner on | Provider | Result |
| --- | --- | --- | --- |
| search only | Leads F1 | Parallel basic | 46.5 |
| search only | Fastest | Brave Search | 43.5 s |
| search only | Lowest cost | TinyFish | $0.212 |
| search + fetch | Leads F1 | Exa deep | 48.2 |
| search + fetch | Fastest | Brave Search | 45.1 s |
| search + fetch | Lowest cost | TinyFish | $0.219 |

### Company News

- Page: https://openbenchmarks.com/company-news

Which API answers questions about recent company events. Web search APIs against dedicated news indexes, same questions.

| Winner on | Provider | Result |
| --- | --- | --- |
| Most accurate | Exa (type=fast) | 99.3% |
| Fastest | Parallel (mode=turbo) | 348 ms |
| Lowest cost | TinyFish | Free |

### Web Search for Coding Agents

- Page: https://openbenchmarks.com/web-search-for-coding-agents

Which search API helps a coding agent find an opaque detail in official Enterprise SaaS docs and ground the patch in a URL it actually retrieved.

| Setup | Winner on | Provider | Result |
| --- | --- | --- | --- |
| search only | Most tasks | Perplexity | 77.3% |
| search only | Fastest | Perplexity | 18.3 s |
| search only | Lowest cost | TinyFish | $0.035 |
| search + fetch | Most tasks | Exa deep | 83.0% |
| search + fetch | Fastest | Perplexity | 21.5 s |
| search + fetch | Lowest cost | TinyFish | $0.050 |

### Document Processing Benchmark

- Page: https://openbenchmarks.com/document-processing

Specialised document parsers benchmarked on heavily redlined contracts, delivered as text-layer PDFs and as scanned images.

| Setup | Winner on | Provider | Result |
| --- | --- | --- | --- |
| text-layer PDF | Most accurate | LlamaParse agentic | 80.0% |
| text-layer PDF | Fastest | Reducto | 7s |
| text-layer PDF | Parser $ / 1k correct | Mistral OCR | $14.04 |
| scanned PDF · image only | Most accurate | Datalab track changes | 79.7% |
| scanned PDF · image only | Fastest | Reducto | 8s |
| scanned PDF · image only | Parser $ / 1k correct | Datalab track changes | $19.13 |

### Company Lookalikes

- Page: https://openbenchmarks.com/lookalikes
- JSON API: https://openbenchmarks.com/api/benchmarks/lookalikes

How well company-search and lookalike APIs turn a seed company domain or description into a ranked list of genuinely similar businesses, for account discovery, prospecting, market mapping, and TAM expansion.

| Winner on | Provider | Result |
| --- | --- | --- |
| Most precise | Seltz | 63.4% |
| Fastest | Discolike | 1.2 s |
| Relevant / $1 | Seltz | 12,683 |

### Company Firmographic Enrichment

- Page: https://openbenchmarks.com/company-enrichment
- JSON API: https://openbenchmarks.com/api/benchmarks/company-enrichment

282 company domains × 11 APIs, normalized into the same seven-field scoring contract. Enrichment success rate, field accuracy, coverage, company match rate, latency, and cost.

| Winner on | Provider | Result |
| --- | --- | --- |
| Correct field yield | People Data Labs | 89.0% |
| Fastest | People Data Labs | 274 ms |
| Lowest cost | Nimble lite | $0.31 |

### Company Funding Data Enrichment

- Page: https://openbenchmarks.com/company-funding
- JSON API: https://openbenchmarks.com/api/benchmarks/company-funding

17 funding vendors, judged on latest funding stage against human-labelled ground truth.

| Setup | Winner on | Provider | Result |
| --- | --- | --- | --- |
| enrichment · older rounds | Correct stage yield | Firecrawl | 92.3% |
| enrichment · older rounds | Fastest | People Data Labs | 276 ms |
| enrichment · older rounds | Lowest cost | Ocean.io | $1.35 |
| freshness · last 30 days | Correct stage yield | Firecrawl | 100.0% |
| freshness · last 30 days | Fastest | Apollo | 320 ms |
| freshness · last 30 days | Lowest cost | Nimble lite | $0.23 |

### LLM Inference Providers

- Page: https://openbenchmarks.com/inference
- JSON API: https://openbenchmarks.com/api/benchmarks/inference

Serverless inference providers serve the same GLM 5.3 Flash model on 600 latency-sensitive lookup, classification, and extraction requests each. Response latency, streaming speed, task success, and failures, without collapsing them into one overall rank.

| Winner on | Provider | Result |
| --- | --- | --- |
| Fastest P99 | Baseten | 3.2 s |
| Most reliable | Fireworks AI | 99.5% |
| Fastest streaming | Baseten | 240 tok/s |

### Voice Agents Latency

- Page: https://openbenchmarks.com/voice-agent-latency
- JSON API: https://openbenchmarks.com/api/benchmarks/voice-agent-latency

TTFAB (Time To First Audio Byte): how long a caller waits before a voice AI agent starts speaking, measured over real phone calls from the call's own audio, never platform-reported timestamps.

| Winner on | Provider | Result |
| --- | --- | --- |
| Fastest median | Telnyx | 1,296 ms |
| Fastest P95 | ElevenLabs | 1,768 ms |
| Lowest cost | Telnyx | $0.0500/min |

### Text-to-Speech (by Coval)

- Page: https://openbenchmarks.com/text-to-speech-benchmark-by-coval
- JSON API: https://openbenchmarks.com/api/benchmarks/text-to-speech-benchmark-by-coval

31 text-to-speech models on Word Error Rate and Time to First Audio (TTFA), measured by Coval under production-realistic conditions and mirrored with attribution.

| Winner on | Provider | Result |
| --- | --- | --- |
| Lowest WER | Cartesia sonic-3.6 | 1.3% |
| Fastest | Fluxions vui | 40 ms |

### Live Speech-to-Text (by Coval)

- Page: https://openbenchmarks.com/speech-to-text-benchmark-by-coval
- JSON API: https://openbenchmarks.com/api/benchmarks/speech-to-text-benchmark-by-coval

31 speech-to-text models on Word Error Rate and Time to Final Segment (TTFS), measured by Coval under production-realistic conditions and mirrored with attribution.

| Winner on | Provider | Result |
| --- | --- | --- |
| Lowest WER | AssemblyAI universal-3.6-pro | 2.1% |
| Fastest | Baseten qwen3-asr-1.7b | 21 ms |

### Structured Speech-to-Text

- Page: https://openbenchmarks.com/structured-speech-to-text-benchmark

300 human-recorded workplace utterances × 17 ASR systems, checked for exact recovery of 1,482 structured values (emails, phone numbers, CLI flags, file paths, IDs). Task success = recordings with every value correct.

| Setup | Winner on | Provider | Result |
| --- | --- | --- | --- |
| batch | Task success | Deepgram Nova-3 | 71.0% |
| batch | Value accuracy | ElevenLabs Scribe v2 | 91.8% |
| batch | Lowest WER | AssemblyAI Universal 3.5 Pro | 3.9% |
| streaming | Task success | Deepgram Nova-3 | 61.0% |
| streaming | Value accuracy | OpenAI GPT Realtime (Whisper) | 88.7% |
| streaming | Lowest WER | ElevenLabs Scribe v2 Realtime | 5.1% |

## How to use this from an agent

### API (no auth)
```bash
curl https://openbenchmarks.com/api/benchmarks
curl https://openbenchmarks.com/api/benchmarks/lookalikes
```

### Discovery

- OpenAPI spec: https://openbenchmarks.com/openapi.json
- llms.txt: https://openbenchmarks.com/llms.txt
- llms-full.txt (with inlined snapshot): https://openbenchmarks.com/llms-full.txt
- A2A agent card: https://openbenchmarks.com/.well-known/agent-card.json
- Agent skills index: https://openbenchmarks.com/.well-known/agent-skills/index.json
- API catalog (RFC 9727): https://openbenchmarks.com/.well-known/api-catalog

## Constraints

- Data is point-in-time against a specific dataset; re-check before major decisions.
- Data refreshes hourly. Honor `Cache-Control`, `ETag`, and `Last-Modified` to stay polite.
- License: CC-BY-4.0. Attribute "Openbenchmarks" with a link back.

## Pricing

Openbenchmarks is free to query (website, API) for humans and agents. See https://openbenchmarks.com/pricing.md for the full machine-readable pricing.

## Trust

- Methodology, scoring rules, and pricing assumptions are documented on every benchmark page.
- Raw inputs and personally identifiable data are not part of the public surface.
- Terms: https://openbenchmarks.com/legal/terms · Privacy: https://openbenchmarks.com/legal/privacy · Security: https://openbenchmarks.com/.well-known/security.txt