BrowseComp vs LiveBrowseComp vs multi-turn company search
What the benchmark found. Current output of the method: Parallel basic (parallel-basic) leads F1 at 46.5% across 20 configurations on 45 questions.
What this page compares. Six fanouts in the probe searched for LiveBrowseComp and landed on arXiv. This page places the three benchmarks side by side: what is held constant, what is scored, and what training-data contamination does to each.
How to read it. BrowseComp and LiveBrowseComp evaluate agents; this benchmark evaluates search APIs with the agent held constant. They answer different questions and are not competing leaderboards.
Method in one paragraph. The same agent, prompt, turn budget and tool schema run against every web search API on 45 questions that each define a company set by three or four constraints. Every returned company is resolved to a canonical identity and compared with a human-reviewed gold set; precision, recall, F1 and exact-set are averaged over three runs per question. The model-only baseline is 0.
Most accurate · Fastest · Cheapest · Find companies by criteria · Search + fetch · Full benchmark
What each term means here
Why not just run BrowseComp against each search API?
Because agents answer up to 44.5% of BrowseComp with no search tool at all (LiveBrowseComp, Fan et al., 2026). A benchmark the model can partly answer from memory cannot isolate the search API's contribution.
Is LiveBrowseComp a substitute?
It fixes contamination by using fresh facts, but it still evaluates the browsing agent end to end. To compare search APIs you also need the agent fixed and the questions built so the model-only baseline is zero, which is what this board does.
Can I compare scores across the three?
No. Different tasks, different scoring, different units. Use BrowseComp-style sets to compare agents and this board to compare the search API behind one agent.
Ranked by F1, search-only
| Provider | Endpoint & configuration | F1 | Precision | Recall | Exact set | Median time | Mean turns | Median task cost | API list price |
|---|---|---|---|---|---|---|---|---|---|
| Parallel basic | POST /v1/search mode=basicparallel-basic | 46.5 ± 1.9 | 88.7 ± 0.9 | 34.4 ± 2.0 | 3.7 ± 2.6 | 67.6 s | 7.4 | $1.120$0.070 search$1.050 token cost | $0.005 / search |
| Exa deep | POST /search type=deepexa-deep | 45.4 ± 2.0 | 83.2 ± 3.0 | 33.7 ± 1.4 | 1.5 ± 1.3 | 89.5 s | 7.4 | $0.717$0.156 search$0.556 token cost | $0.012 / search |
| Parallel advanced | POST /v1/search mode=advancedparallel-advanced | 44.2 ± 1.4 | 87.6 ± 3.4 | 32.0 ± 1.6 | 2.2 ± 0.0 | 83.4 s | 7.3 | $0.625$0.070 search$0.565 token cost | $0.005 / search |
| Exa instant | POST /search type=instantexa-instant | 43.3 ± 1.0 | 82.6 ± 3.8 | 32.2 ± 0.5 | 3.0 ± 1.3 | 49.8 s | 7.3 | $0.652$0.091 search$0.559 token cost | $0.007 / search |
| Linkup fast | POST /v1/search depth=fastlinkup-fast | 41.1 ± 1.7 | 82.8 ± 1.0 | 30.3 ± 1.5 | 0.7 ± 1.3 | 64.2 s | 7.4 | $0.923$0.070 search$0.853 token cost | $0.005 / search |
| Tavily advanced | POST /search search_depth=advancedtavily-advanced | 41.1 ± 2.3 | 83.7 ± 4.2 | 29.8 ± 1.7 | 2.2 ± 0.0 | 92.2 s | 7.4 | $1.029$0.224 search$0.808 token cost | $0.016 / search |
| Linkup standard | POST /v1/search depth=standardlinkup-standard | 40.6 ± 0.9 | 84.0 ± 7.9 | 29.7 ± 1.4 | 1.5 ± 2.6 | 72.3 s | 7.3 | $0.937$0.070 search$0.867 token cost | $0.005 / search |
| You | POST /v1/searchextraction_mode=highlights | 38.6 ± 0.8 | 79.3 ± 4.9 | 28.3 ± 0.9 | 4.4 ± 0.0 | 48.2 s | 7.5 | $0.971$0.070 search$0.901 token cost | $0.005 / search |
| You | POST /v1/searchextraction_mode=highlights · knowledge=core | 38.1 ± 0.6 | 80.9 ± 1.6 | 27.4 ± 0.9 | 3.0 ± 1.3 | 47.6 s | 7.5 | $0.895$0.070 search$0.827 token cost | $0.005 / search |
| Parallel fast | POST /v1/search mode=fastparallel-fast | 38.0 ± 2.0 | 79.9 ± 4.6 | 27.5 ± 1.4 | 1.5 ± 1.3 | 53.9 s | 7.6 | $0.460$0.014 search$0.447 token cost | $0.001 / search |
| Perplexity | POST /searchsearch_context_size=low | 37.8 ± 2.1 | 79.3 ± 5.7 | 26.8 ± 1.4 | 2.2 ± 2.2 | 48.9 s | 7.7 | $0.334$0.070 search$0.264 token cost | $0.005 / search |
| Tavily basic | POST /search search_depth=basictavily-basic | 36.5 ± 3.2 | 82.5 ± 1.9 | 25.7 ± 2.7 | 1.5 ± 1.3 | 63.0 s | 7.7 | $0.615$0.112 search$0.504 token cost | $0.008 / search |
| Parallel turbo | POST /v1/search mode=turboparallel-turbo | 34.7 ± 2.4 | 80.0 ± 5.2 | 24.8 ± 2.4 | 1.5 ± 1.3 | 46.4 s | 7.6 | $0.419$0.014 search$0.405 token cost | $0.001 / search |
| Nimble | POST /v2/searchsearch_depth=standard · full_content=false · focus=general | 30.7 ± 1.4 | 70.3 ± 3.3 | 21.3 ± 1.1 | 0.7 ± 1.3 | 53.0 s | 7.7 | $0.467$0.070 search$0.398 token cost | $0.005 / search |
| Firecrawl | POST /v2/searchfirecrawl | 30.4 ± 1.1 | 77.3 ± 4.4 | 20.7 ± 0.7 | 2.2 ± 0.0 | 75.0 s | 7.9 | $0.282$0.070 search$0.212 token cost | $0.005 / search |
| Brave Search | GET /res/v1/web/searchbrave | 28.0 ± 1.7 | 66.9 ± 8.5 | 19.3 ± 0.5 | 0.7 ± 1.3 | 43.5 s | 8.0 | $0.268$0.070 search$0.200 token cost | $0.005 / search |
| TinyFish | GET api.search.tinyfish.aitinyfish | 26.6 ± 1.3 | 64.7 ± 1.5 | 17.9 ± 0.8 | 1.5 ± 1.3 | 64.2 s | 7.9 | $0.212$0.000 search$0.212 token cost | $0 / search |
| Nimble | POST /v2/searchsearch_depth=lite · full_content=false · focus=general | 25.3 ± 1.1 | 62.8 ± 1.9 | 17.6 ± 1.2 | 1.5 ± 1.3 | 67.9 s | 7.9 | $0.234$0.015 search$0.220 token cost | $0.0011 / search |
| Seltz | POST /v1/search scope=companiesseltz-companies | 14.5 ± 0.9 | 40.0 ± 3.1 | 9.4 ± 0.7 | 0.0 ± 0.0 | 55.2 s | 7.3 | $1.751$0.070 search$1.681 token cost | $0.005 / search |
| SERP | GET google-search74.p.rapidapi.comserp | 0.4 ± 0.6 | 0.7 ± 1.3 | 0.3 ± 0.4 | 0.0 ± 0.0 | 31.4 s | 7.8 | $0.103$0.036 search$0.065 token cost | $0.003 / search |
F1, precision, recall, and exact-set accuracy are percentages reported as mean ± sample SD across three independent runs; each run aggregates all 45 questions. SD is measured in percentage points. Median time is the median end-to-end time across all runs for each vendor. Median task cost is the median of LLM $ plus search API $ per agent run. API list price is the PAYG unit rate of the search endpoint the harness calls.
Same sort, with a page-reading tool enabled
| Provider | Endpoint & configuration | F1 | Precision | Recall | Exact set | Median time | Mean turns | Median task cost | API list price |
|---|---|---|---|---|---|---|---|---|---|
| Exa deep | POST /search type=deepexa-deep | 48.2 ± 2.1 | 89.4 ± 1.2 | 36.0 ± 2.2 | 2.2 ± 2.2 | 95.8 s | 7.5 | $0.683$0.156 search+fetch$0.526 token cost | $0.012 / search$0.001 / fetch |
| Perplexity | POST /searchsearch_context_size=high | 46.6 ± 2.0 | 87.7 ± 5.9 | 34.7 ± 1.1 | 2.2 ± 2.2 | 53.9 s | 7.5 | $0.504$0.070 search+fetch$0.441 token cost | $0.005 / search |
| Exa instant | POST /search type=instantexa-instant | 44.9 ± 0.9 | 85.9 ± 2.5 | 33.5 ± 0.9 | 5.2 ± 1.3 | 52.5 s | 7.5 | $0.653$0.091 search+fetch$0.557 token cost | $0.007 / search$0.001 / fetch |
| Parallel basic | POST /v1/search mode=basicparallel-basic | 42.3 ± 1.1 | 81.3 ± 2.8 | 31.3 ± 1.1 | 3.0 ± 1.3 | 68.6 s | 7.5 | $1.089$0.061 search+fetch$1.033 token cost | $0.005 / search$0.001 / fetch |
| Parallel advanced | POST /v1/search mode=advancedparallel-advanced | 42.2 ± 1.1 | 87.6 ± 0.3 | 30.1 ± 1.1 | 2.2 ± 0.0 | 80.9 s | 7.4 | $0.599$0.061 search+fetch$0.538 token cost | $0.005 / search$0.001 / fetch |
| Linkup standard | POST /v1/search depth=standardlinkup-standard | 42.0 ± 1.8 | 90.7 ± 2.0 | 30.5 ± 2.0 | 3.0 ± 3.4 | 81.0 s | 7.5 | $0.911$0.070 search+fetch$0.848 token cost | $0.005 / search$0.001 / fetch |
| Tavily advanced | POST /search search_depth=advancedtavily-advanced | 41.0 ± 1.3 | 89.4 ± 6.0 | 29.1 ± 0.7 | 2.2 ± 0.0 | 92.6 s | 7.3 | $0.898$0.195 search+fetch$0.685 token cost | $0.016 / search$0.0032 / fetch |
| Linkup fast | POST /v1/search depth=fastlinkup-fast | 39.9 ± 1.3 | 85.3 ± 3.6 | 28.6 ± 1.3 | 0.7 ± 1.3 | 68.0 s | 7.3 | $0.903$0.061 search+fetch$0.837 token cost | $0.005 / search$0.001 / fetch |
| Parallel fast | POST /v1/search mode=fastparallel-fast | 39.3 ± 3.3 | 82.3 ± 6.6 | 28.2 ± 2.0 | 2.2 ± 0.0 | 55.1 s | 7.6 | $0.441$0.014 search+fetch$0.427 token cost | $0.001 / search$0.001 / fetch |
| You | POST /v1/searchextraction_mode=highlights | 38.5 ± 1.6 | 85.2 ± 2.7 | 26.9 ± 1.7 | 0.7 ± 1.3 | 45.6 s | 7.4 | $0.864$0.065 search+fetch$0.795 token cost | $0.005 / search$0.001 / fetch |
| You | POST /v1/searchextraction_mode=highlights · knowledge=core | 38.5 ± 2.4 | 78.2 ± 2.5 | 28.3 ± 3.0 | 3.7 ± 1.3 | 48.5 s | 7.4 | $0.894$0.065 search+fetch$0.830 token cost | $0.005 / search$0.001 / fetch |
| Tavily basic | POST /search search_depth=basictavily-basic | 38.1 ± 1.4 | 81.2 ± 5.9 | 27.8 ± 0.7 | 2.2 ± 0.0 | 62.0 s | 7.8 | $0.610$0.112 search+fetch$0.507 token cost | $0.008 / search$0.0032 / fetch |
| Parallel turbo | POST /v1/search mode=turboparallel-turbo | 36.0 ± 3.5 | 83.6 ± 4.0 | 25.0 ± 2.6 | 0.0 ± 0.0 | 48.1 s | 7.7 | $0.414$0.013 search+fetch$0.400 token cost | $0.001 / search$0.001 / fetch |
| Firecrawl | POST /v2/searchfirecrawl | 33.2 ± 2.1 | 83.3 ± 0.8 | 22.7 ± 1.8 | 1.5 ± 1.3 | 82.1 s | 7.9 | $0.295$0.063 search+fetch$0.230 token cost | $0.005 / search$0.0025 / fetch |
| Nimble | POST /v2/searchsearch_depth=standard · full_content=false · focus=general · extract formats=[markdown] | 31.3 ± 1.8 | 73.2 ± 1.9 | 21.7 ± 1.6 | 1.5 ± 1.3 | 59.2 s | 7.9 | $0.424$0.424 token cost | $0.005 / search |
| TinyFish | GET api.search.tinyfish.aitinyfish | 30.2 ± 3.5 | 70.9 ± 11.3 | 20.9 ± 2.5 | 0.0 ± 0.0 | 63.4 s | 7.9 | $0.219$0.000 search+fetch$0.219 token cost | $0 / search$0 / fetch |
| Brave Search | GET /res/v1/web/searchbrave | 29.4 ± 1.6 | 73.5 ± 5.8 | 20.4 ± 0.9 | 1.5 ± 1.3 | 45.1 s | 8.0 | $0.285$0.060 search+fetch$0.222 token cost | $0.005 / search |
| Nimble | POST /v2/searchsearch_depth=lite · full_content=false · focus=general · extract formats=[markdown] | 27.1 ± 1.5 | 68.5 ± 4.4 | 18.7 ± 0.9 | 2.2 ± 0.0 | 72.0 s | 8.0 | $0.237$0.237 token cost | $0.0011 / search |
| Seltz | POST /v1/search scope=companiesseltz-companies | 16.3 ± 1.5 | 49.5 ± 2.4 | 10.2 ± 1.2 | 0.0 ± 0.0 | 60.1 s | 7.5 | $1.741$0.070 search+fetch$1.671 token cost | $0.005 / search |
| SERP | GET google-search74.p.rapidapi.comserp | 0.0 ± 0.0 | 0.0 ± 0.0 | 0.0 ± 0.0 | 0.0 ± 0.0 | 33.4 s | 7.8 | $0.102$0.039 search+fetch$0.066 token cost | $0.003 / search |
F1, precision, recall, and exact-set accuracy are percentages reported as mean ± sample SD across three independent runs; each run aggregates all 45 questions. SD is measured in percentage points. Median time is the median end-to-end time across all runs for each vendor. Median task cost is the median of LLM $ plus search/fetch API $ per agent run. API list price is the PAYG unit rate of the search (and fetch) endpoint the harness calls.
How to evaluate a web search API for AI agents → · Precision vs recall vs F1 for web search APIs →