Web Search Benchmark
Why we built this. To evaluate web search APIs. The task is to search for recent company events such as layoffs, product launches, acquisitions, funding, leadership changes, expansions as those are factual lookups with a factual ground truth grounded in an official wire or newsroom URL. We also compare those general web search APIs against specialized news indexes. The benchmark shows how each endpoint balances accuracy, cost, and latency through retrieval (AR@1, AR@5).
How it is measured. Every endpoint gets the same user query and can retrieve max 10 results. An LLM in the harness (gpt-5.6-terra, medium) reads title + excerpt only and extracts an answer. A separate post-hoc judge from independent model family (claude-opus-5 via Amazon Bedrock) scores accuracy and AR@K against human labelled ground truth: the full ground truth must be present. Accuracy is extract correctness. AR is answer recall. AR@1 is the first snippet. AR@5 is any of the top 5.
Vendors benchmarked. AI search APIs, search engines, and news indexes.
- AI-specific web search. Exa, Parallel, Linkup, Tavily, and Firecrawl. Search APIs built for RAG. Exa, Parallel, and Linkup each have two endpoints benchmarked, a Fast option and a Deep option, so you can see the accuracy, cost, and latency tradeoff inside one vendor.
- Search engine. Brave Search and SERP. Search engines: a query in, and 10 results with links and snippets returned.
- News index. Seltz and PredictLeads. Specialized APIs that index news, included so general web search can be compared with a news-specific index.
How to pick a provider. There is no single best web search API. It is a tradeoff between accuracy, latency, cost, and retrieval quality.
- Fastest for a realtime loop. Read
Latency. Prefer the Fast row of a pair. - Most accurate for a critical path. Read
Accuracy. Prefer the Deep row if the extra wait is acceptable. - Best retrieval. AR is answer recall. Read
AR@1if the first hit is used. ReadAR@5if it scans the top 5. - Cheapest at volume. Read
Cost. List price for this run, not free tiers or volume discounts.
Benchmark Data
Rows are alphabetical by vendor.
| Vendor | Endpoint & configuration | Accuracy | AR@1 | AR@5 | Latency | Snippet tokens | Total $ | Official docs |
|---|---|---|---|---|---|---|---|---|
| Brave Searchweb search | GET /res/v1/web/searchcount=10 · result_filter=web | 95.0% | 85.7% | 95.0% | 697ms | 727 | $0.67 | Official docs |
| Exaweb search | POST /searchtype=deep | 99.2% | 98.3% | 99.2% | 2.95s | 1,293 | $1.51 | Official docs |
| Exaweb search | POST /searchtype=instant | 99.2% | 84.9% | 97.5% | 444ms | 2,053 | $0.96 | Official docs |
| Firecrawlweb search | POST /v2/search | 94.1% | 77.3% | 93.3% | 1.10s | 672 | $0.67 | Official docs |
| Linkupweb search | POST /v1/searchdepth=fast · outputType=searchResults | 96.6% | 89.1% | 96.6% | 1.84s | 2,846 | $0.76 | Official docs |
| Linkupweb search | POST /v1/searchdepth=standard · outputType=searchResults | 97.5% | 84.9% | 95.0% | 2.34s | 2,936 | $0.76 | Official docs |
| Parallelweb search | POST /v1/searchmode=advanced | 98.3% | 63.0% | 98.3% | 3.33s | 2,143 | $0.74 | Official docs |
| Parallelweb search | POST /v1/searchmode=basic | 97.5% | 68.9% | 93.3% | 1.40s | 2,413 | $0.75 | Official docs |
| PredictLeadscompany news events | GET /api/v3/companies/{domain}/news_events | 63.0% | 47.9% | 63.0% | 668ms | 411 | $4.81 | Official docs |
| Seltznews index | POST /v1/searchscope=news | 68.9% | 31.9% | 64.7% | 321ms | 2,913 | $0.76 | Official docs |
| SERPgoogle search | GET google-search74.p.rapidapi.comlimit=10 | 94.1% | 76.5% | 93.3% | 2.14s | 468 | $0.42 | Official docs |
| Tavilyweb search | POST /searchsearch_depth=advanced · chunks_per_source=3 | 95.0% | 84.0% | 95.0% | 4.34s | 3,035 | $2.08 | Official docs |
[02] methodology and metric definitions+
How the benchmark is built
We measure the web search API. Everything around it is held constant: the input query, the model that reads the results and extracts an answer (gpt-5.6-terra, medium), and the judge (claude-opus-5 via Amazon Bedrock). The only thing that changes is which search API is called.
- Collect official wires and newsrooms. Fix ground truth by human labelling before calling any vendors. Set of 119 search queries in v1.
- Send every endpoint the same natural-language question, one request, max 10 results. The query is the user question, unchanged.
- Persist the raw HTTP envelope, normalized hits (url, title, snippet), and latency for every call.
- An LLM in the harness (
gpt-5.6-terra, medium) reads title + excerpt only and extracts an answer. A separate post-hoc judge from independent model family (claude-opus-5via Amazon Bedrock) scores accuracy and AR@K against human labelled ground truth. It never fetches the live page. No vendor APIs are recalled for extract or score. - Score accuracy, AR@1, AR@5, snippet tokens, latency, and list-price cost. Rank by accuracy.
Out of scope for this benchmark
This is an atomic fact lookup benchmark using search APIs. The following are not measured yet through this benchmark.
- Query rewriting and multi-hop searches. The user query is sent as-is to all APIs as input and is a single fact-lookup query.
- Varying search depth. Every endpoint returns the top 10 results. We measure retrieval quality at that cutoff; we do not run higher-k searches or tune depth.
What each metric means
Accuracy·gpt-5.6-terra(medium) writes an answer from the returned titles and snippets. The judge scores that written answer against human labelled ground truth. The headline correctness metric.AR@K· answer recall at K. Whether any of the top K vendor snippets already contained the human labelled ground truth. No extract. The judge scores the snippets themselves.AR@1· the first snippet already contained the human labelled ground truth.AR@5· whether any one of the top 5 snippets already contained the full human labelled ground truth. AR@5 can be higher than accuracy: accuracy scores the written extract, which reads all 10 snippets. If a top-5 snippet already has the full fact but later hits in the top 10 conflict, the extract can return empty or say the fact was not found. AR@5 is then 1 and accuracy is 0.Latency· speed. Mean wall time of the search request, not the extract or score calls.Snippet tokens· mean approximate tokens in the returned web search results per question (~4 characters / token).Total $· search list price for the run as of 16 Aug 2026. Judge tokens are not in this column.Endpoint & configuration· the HTTP path we called, then the vendor setting that tells two rows of the same vendor apart (type=instantvstype=deep,mode=basicvsmode=advanced,depth=fastvsdepth=standard). Single-endpoint vendors show their scope when the call has one. The vendor column uses the company name on every arm.Official docs· vendor API documentation for that product.
Most accurate, fastest, and cheapest search API
What is the most accurate search API?
Exa (type=deep) leads extracted-answer accuracy at 99.2% on 119 lookup questions.
What is the fastest search API?
Seltz has the lowest mean latency at 321ms on this run.
What is the cheapest search API?
SERP (RapidAPI) has the lowest list-price search cost on this run ($0.42). Free tiers and volume discounts are not included.
Is this benchmark independent?
Yes. No vendor pays for inclusion, ranking, or removal. The input query, the extract model (gpt-5.6-terra), and the judge (claude-opus-5) are held constant. The only thing that changes is which search API is called.
What does the Web Search Benchmark not measure?
Query rewriting and multi-hop searches: the user query is sent as-is to all APIs and is a single fact-lookup query. Varying search depth: every endpoint returns the top 10 results; we do not run higher-k searches or tune depth.
Choosing a search API
Most accurate search API, ranked by accuracy → · Fastest search API, measured latency → · Cheapest search API, list price beside accuracy →