BrowseComp vs LiveBrowseComp vs multi-turn company search
What the benchmark found. Current output of the method: Parallel basic (parallel-basic) leads F1 at 46.5% across 16 configurations on 45 questions.
What this page compares. Six fanouts in the probe searched for LiveBrowseComp and landed on arXiv. This page places the three benchmarks side by side: what is held constant, what is scored, and what training-data contamination does to each.
How to read it. BrowseComp and LiveBrowseComp evaluate agents; this benchmark evaluates search APIs with the agent held constant. They answer different questions and are not competing leaderboards.
Method in one paragraph. The same agent, prompt, turn budget and tool schema run against every web search API on 45 questions that each define a company set by three or four constraints. Every returned company is resolved to a canonical identity and compared with a human-reviewed gold set; precision, recall, F1 and exact-set are averaged over three runs per question. The model-only baseline is 0.
Most accurate · Fastest · Cheapest · Find companies by criteria · Search + fetch · Full benchmark
What each term means here
Why not just run BrowseComp against each search API?
Because agents answer up to 44.5% of BrowseComp with no search tool at all (LiveBrowseComp, Fan et al., 2026). A benchmark the model can partly answer from memory cannot isolate the search API's contribution.
Is LiveBrowseComp a substitute?
It fixes contamination by using fresh facts, but it still evaluates the browsing agent end to end. To compare search APIs you also need the agent fixed and the questions built so the model-only baseline is zero, which is what this board does.
Can I compare scores across the three?
No. Different tasks, different scoring, different units. Use BrowseComp-style sets to compare agents and this board to compare the search API behind one agent.
Ranked by F1, search-only
| Provider | Endpoint & configuration | F1 | Precision | Recall | Exact set | Median time | Mean turns | Median task cost | API list price |
|---|---|---|---|---|---|---|---|---|---|
| Parallel | POST /v1/search mode=basicparallel-basic | 46.5 ± 1.9 | 88.7 ± 0.9 | 34.4 ± 2.0 | 3.7 ± 2.6 | 67.6 s | 7.4 | $1.120$0.070 search$1.050 token cost | $0.005 / search |
| Exa | POST /search type=deepexa-deep | 45.4 ± 2.0 | 83.2 ± 3.0 | 33.7 ± 1.4 | 1.5 ± 1.3 | 89.5 s | 7.4 | $0.717$0.156 search$0.556 token cost | $0.012 / search |
| Parallel | POST /v1/search mode=advancedparallel-advanced | 44.2 ± 1.4 | 87.6 ± 3.4 | 32.0 ± 1.6 | 2.2 ± 0.0 | 83.4 s | 7.3 | $0.625$0.070 search$0.565 token cost | $0.005 / search |
| Exa | POST /search type=instantexa-instant | 43.3 ± 1.0 | 82.6 ± 3.8 | 32.2 ± 0.5 | 3.0 ± 1.3 | 49.8 s | 7.3 | $0.652$0.091 search$0.559 token cost | $0.007 / search |
| Linkup | POST /v1/search depth=fastlinkup-fast | 41.1 ± 1.7 | 82.8 ± 1.0 | 30.3 ± 1.5 | 0.7 ± 1.3 | 64.2 s | 7.4 | $0.923$0.070 search$0.853 token cost | $0.005 / search |
| Tavily | POST /search search_depth=advancedtavily-advanced | 41.1 ± 2.3 | 83.7 ± 4.2 | 29.8 ± 1.7 | 2.2 ± 0.0 | 92.2 s | 7.4 | $1.029$0.224 search$0.808 token cost | $0.016 / search |
| Linkup | POST /v1/search depth=standardlinkup-standard | 40.6 ± 0.9 | 84.0 ± 7.9 | 29.7 ± 1.4 | 1.5 ± 2.6 | 72.3 s | 7.3 | $0.937$0.070 search$0.867 token cost | $0.005 / search |
| Parallel | POST /v1/search mode=fastparallel-fast | 38.0 ± 2.0 | 79.9 ± 4.6 | 27.5 ± 1.4 | 1.5 ± 1.3 | 53.9 s | 7.6 | $0.460$0.014 search$0.447 token cost | $0.001 / search |
| Perplexity | POST /searchsearch_context_size=low | 37.8 ± 2.1 | 79.3 ± 5.7 | 26.8 ± 1.4 | 2.2 ± 2.2 | 48.9 s | 7.7 | $0.334$0.070 search$0.264 token cost | $0.005 / search |
| Parallel | POST /v1/search mode=turboparallel-turbo | 34.7 ± 2.4 | 80.0 ± 5.2 | 24.8 ± 2.4 | 1.5 ± 1.3 | 46.4 s | 7.6 | $0.419$0.014 search$0.405 token cost | $0.001 / search |
| You | POST /v1/searchyou | 33.1 ± 2.4 | 75.7 ± 3.1 | 23.1 ± 1.9 | 1.5 ± 1.3 | 46.2 s | 7.7 | $0.477$0.070 search$0.408 token cost | $0.005 / search |
| Firecrawl | POST /v2/searchfirecrawl | 30.4 ± 1.1 | 77.3 ± 4.4 | 20.7 ± 0.7 | 2.2 ± 0.0 | 75.0 s | 7.9 | $0.282$0.070 search$0.212 token cost | $0.005 / search |
| Brave Search | GET /res/v1/web/searchbrave | 28.0 ± 1.7 | 66.9 ± 8.5 | 19.3 ± 0.5 | 0.7 ± 1.3 | 43.5 s | 8.0 | $0.268$0.070 search$0.200 token cost | $0.005 / search |
| TinyFish | GET api.search.tinyfish.aitinyfish | 26.6 ± 1.3 | 64.7 ± 1.5 | 17.9 ± 0.8 | 1.5 ± 1.3 | 64.2 s | 7.9 | $0.212$0.000 search$0.212 token cost | $0 / search |
| Seltz | POST /v1/search scope=companiesseltz-companies | 14.5 ± 0.9 | 40.0 ± 3.1 | 9.4 ± 0.7 | 0.0 ± 0.0 | 55.2 s | 7.3 | $1.751$0.070 search$1.681 token cost | $0.005 / search |
| SERP (RapidAPI) | GET google-search74.p.rapidapi.comserp | 0.4 ± 0.6 | 0.7 ± 1.3 | 0.3 ± 0.4 | 0.0 ± 0.0 | 31.4 s | 7.8 | $0.103$0.036 search$0.065 token cost | $0.003 / search |
F1, precision, recall, and exact-set accuracy are percentages reported as mean ± sample SD across three independent runs; each run aggregates all 45 questions. SD is measured in percentage points. Median time is the median end-to-end time across all runs for each vendor. Median task cost is the median of LLM $ plus search/fetch API $ per agent run. API list price is the PAYG unit rate of the search (and fetch) endpoint the harness calls.
Same sort, with a page-reading tool enabled
| Provider | Endpoint & configuration | F1 | Precision | Recall | Exact set | Median time | Mean turns | Median task cost | API list price |
|---|---|---|---|---|---|---|---|---|---|
| Exa | POST /search type=deepexa-deep | 48.2 ± 2.1 | 89.4 ± 1.2 | 36.0 ± 2.2 | 2.2 ± 2.2 | 95.8 s | 7.5 | $0.683$0.156 search+fetch$0.526 token cost | $0.012 / search$0.001 / fetch |
| Perplexity | POST /searchsearch_context_size=high | 46.6 ± 2.0 | 87.7 ± 5.9 | 34.7 ± 1.1 | 2.2 ± 2.2 | 53.9 s | 7.5 | $0.504$0.070 search+fetch$0.441 token cost | $0.005 / search |
| Exa | POST /search type=instantexa-instant | 44.9 ± 0.9 | 85.9 ± 2.5 | 33.5 ± 0.9 | 5.2 ± 1.3 | 52.5 s | 7.5 | $0.653$0.091 search+fetch$0.557 token cost | $0.007 / search$0.001 / fetch |
| Parallel | POST /v1/search mode=basicparallel-basic | 42.3 ± 1.1 | 81.3 ± 2.8 | 31.3 ± 1.1 | 3.0 ± 1.3 | 68.6 s | 7.5 | $1.089$0.061 search+fetch$1.033 token cost | $0.005 / search$0.001 / fetch |
| Parallel | POST /v1/search mode=advancedparallel-advanced | 42.2 ± 1.1 | 87.6 ± 0.3 | 30.1 ± 1.1 | 2.2 ± 0.0 | 80.9 s | 7.4 | $0.599$0.061 search+fetch$0.538 token cost | $0.005 / search$0.001 / fetch |
| Linkup | POST /v1/search depth=standardlinkup-standard | 42.0 ± 1.8 | 90.7 ± 2.0 | 30.5 ± 2.0 | 3.0 ± 3.4 | 81.0 s | 7.5 | $0.911$0.070 search+fetch$0.848 token cost | $0.005 / search$0.001 / fetch |
| Tavily | POST /search search_depth=advancedtavily-advanced | 41.0 ± 1.3 | 89.4 ± 6.0 | 29.1 ± 0.7 | 2.2 ± 0.0 | 92.6 s | 7.3 | $0.898$0.195 search+fetch$0.685 token cost | $0.016 / search$0.0032 / fetch |
| Linkup | POST /v1/search depth=fastlinkup-fast | 39.9 ± 1.3 | 85.3 ± 3.6 | 28.6 ± 1.3 | 0.7 ± 1.3 | 68.0 s | 7.3 | $0.903$0.061 search+fetch$0.837 token cost | $0.005 / search$0.001 / fetch |
| Parallel | POST /v1/search mode=fastparallel-fast | 39.3 ± 3.3 | 82.3 ± 6.6 | 28.2 ± 2.0 | 2.2 ± 0.0 | 55.1 s | 7.6 | $0.441$0.014 search+fetch$0.427 token cost | $0.001 / search$0.001 / fetch |
| Parallel | POST /v1/search mode=turboparallel-turbo | 36.0 ± 3.5 | 83.6 ± 4.0 | 25.0 ± 2.6 | 0.0 ± 0.0 | 48.1 s | 7.7 | $0.414$0.013 search+fetch$0.400 token cost | $0.001 / search$0.001 / fetch |
| You | POST /v1/searchyou | 34.0 ± 0.9 | 78.8 ± 0.6 | 23.8 ± 1.2 | 3.0 ± 1.3 | 47.4 s | 7.8 | $0.489$0.065 search+fetch$0.427 token cost | $0.005 / search$0.001 / fetch |
| Firecrawl | POST /v2/searchfirecrawl | 33.2 ± 2.1 | 83.3 ± 0.8 | 22.7 ± 1.8 | 1.5 ± 1.3 | 82.1 s | 7.9 | $0.295$0.063 search+fetch$0.230 token cost | $0.005 / search$0.0025 / fetch |
| TinyFish | GET api.search.tinyfish.aitinyfish | 30.2 ± 3.5 | 70.9 ± 11.3 | 20.9 ± 2.5 | 0.0 ± 0.0 | 63.4 s | 7.9 | $0.219$0.000 search+fetch$0.219 token cost | $0 / search$0 / fetch |
| Brave Search | GET /res/v1/web/searchbrave | 29.4 ± 1.6 | 73.5 ± 5.8 | 20.4 ± 0.9 | 1.5 ± 1.3 | 45.1 s | 8.0 | $0.285$0.060 search+fetch$0.222 token cost | $0.005 / search |
| Seltz | POST /v1/search scope=companiesseltz-companies | 16.3 ± 1.5 | 49.5 ± 2.4 | 10.2 ± 1.2 | 0.0 ± 0.0 | 60.1 s | 7.5 | $1.741$0.070 search+fetch$1.671 token cost | $0.005 / search |
| SERP (RapidAPI) | GET google-search74.p.rapidapi.comserp | 0.0 ± 0.0 | 0.0 ± 0.0 | 0.0 ± 0.0 | 0.0 ± 0.0 | 33.4 s | 7.8 | $0.102$0.039 search+fetch$0.066 token cost | $0.003 / search |
F1, precision, recall, and exact-set accuracy are percentages reported as mean ± sample SD across three independent runs; each run aggregates all 45 questions. SD is measured in percentage points. Median time is the median end-to-end time across all runs for each vendor. Median task cost is the median of LLM $ plus search/fetch API $ per agent run. API list price is the PAYG unit rate of the search (and fetch) endpoint the harness calls.
How to evaluate a web search API for AI agents → · Precision vs recall vs F1 for web search APIs →