benchmarks/web-search/multi-turn company search/find companies by criteria
45 questions · multi-hop · 20 configurations

Best search API to find companies by investor, location and funding stage

Ranked on the multi-hop search task of the web search benchmark. Each question combines three or four constraints.

Parallel basic (parallel-basic) leads F1 at 46.5% on this run.

Every question here is a company set defined by three or four constraints (investor backing, headquarters geography, accelerator, funding stage, sector). F1 rewards the API that lets the agent recover the complete set without padding. This is search-driven discovery, not a company database lookup.

Most accurate · Fastest · Cheapest · Find companies by criteria · Search + fetch · Full benchmark

open runner + judgePublic in openbenchmarks-labs/multi-turn-company-search.github →

Ranking, sorted by f1

Search-only, ranked by f1
ProviderEndpoint & configurationF1PrecisionRecallExact setMedian timeMean turnsMedian task costAPI list price
Parallel basicPOST /v1/search mode=basicparallel-basic46.5 ± 1.988.7 ± 0.934.4 ± 2.03.7 ± 2.667.6 s7.4$1.120$0.070 search$1.050 token cost$0.005 / search
Exa deepPOST /search type=deepexa-deep45.4 ± 2.083.2 ± 3.033.7 ± 1.41.5 ± 1.389.5 s7.4$0.717$0.156 search$0.556 token cost$0.012 / search
Parallel advancedPOST /v1/search mode=advancedparallel-advanced44.2 ± 1.487.6 ± 3.432.0 ± 1.62.2 ± 0.083.4 s7.3$0.625$0.070 search$0.565 token cost$0.005 / search
Exa instantPOST /search type=instantexa-instant43.3 ± 1.082.6 ± 3.832.2 ± 0.53.0 ± 1.349.8 s7.3$0.652$0.091 search$0.559 token cost$0.007 / search
Linkup fastPOST /v1/search depth=fastlinkup-fast41.1 ± 1.782.8 ± 1.030.3 ± 1.50.7 ± 1.364.2 s7.4$0.923$0.070 search$0.853 token cost$0.005 / search
Tavily advancedPOST /search search_depth=advancedtavily-advanced41.1 ± 2.383.7 ± 4.229.8 ± 1.72.2 ± 0.092.2 s7.4$1.029$0.224 search$0.808 token cost$0.016 / search
Linkup standardPOST /v1/search depth=standardlinkup-standard40.6 ± 0.984.0 ± 7.929.7 ± 1.41.5 ± 2.672.3 s7.3$0.937$0.070 search$0.867 token cost$0.005 / search
YouPOST /v1/searchextraction_mode=highlights38.6 ± 0.879.3 ± 4.928.3 ± 0.94.4 ± 0.048.2 s7.5$0.971$0.070 search$0.901 token cost$0.005 / search
YouPOST /v1/searchextraction_mode=highlights · knowledge=core38.1 ± 0.680.9 ± 1.627.4 ± 0.93.0 ± 1.347.6 s7.5$0.895$0.070 search$0.827 token cost$0.005 / search
Parallel fastPOST /v1/search mode=fastparallel-fast38.0 ± 2.079.9 ± 4.627.5 ± 1.41.5 ± 1.353.9 s7.6$0.460$0.014 search$0.447 token cost$0.001 / search
PerplexityPOST /searchsearch_context_size=low37.8 ± 2.179.3 ± 5.726.8 ± 1.42.2 ± 2.248.9 s7.7$0.334$0.070 search$0.264 token cost$0.005 / search
Tavily basicPOST /search search_depth=basictavily-basic36.5 ± 3.282.5 ± 1.925.7 ± 2.71.5 ± 1.363.0 s7.7$0.615$0.112 search$0.504 token cost$0.008 / search
Parallel turboPOST /v1/search mode=turboparallel-turbo34.7 ± 2.480.0 ± 5.224.8 ± 2.41.5 ± 1.346.4 s7.6$0.419$0.014 search$0.405 token cost$0.001 / search
NimblePOST /v2/searchsearch_depth=standard · full_content=false · focus=general30.7 ± 1.470.3 ± 3.321.3 ± 1.10.7 ± 1.353.0 s7.7$0.467$0.070 search$0.398 token cost$0.005 / search
FirecrawlPOST /v2/searchfirecrawl30.4 ± 1.177.3 ± 4.420.7 ± 0.72.2 ± 0.075.0 s7.9$0.282$0.070 search$0.212 token cost$0.005 / search
Brave SearchGET /res/v1/web/searchbrave28.0 ± 1.766.9 ± 8.519.3 ± 0.50.7 ± 1.343.5 s8.0$0.268$0.070 search$0.200 token cost$0.005 / search
TinyFishGET api.search.tinyfish.aitinyfish26.6 ± 1.364.7 ± 1.517.9 ± 0.81.5 ± 1.364.2 s7.9$0.212$0.000 search$0.212 token cost$0 / search
NimblePOST /v2/searchsearch_depth=lite · full_content=false · focus=general25.3 ± 1.162.8 ± 1.917.6 ± 1.21.5 ± 1.367.9 s7.9$0.234$0.015 search$0.220 token cost$0.0011 / search
SeltzPOST /v1/search scope=companiesseltz-companies14.5 ± 0.940.0 ± 3.19.4 ± 0.70.0 ± 0.055.2 s7.3$1.751$0.070 search$1.681 token cost$0.005 / search
SERPGET google-search74.p.rapidapi.comserp0.4 ± 0.60.7 ± 1.30.3 ± 0.40.0 ± 0.031.4 s7.8$0.103$0.036 search$0.065 token cost$0.003 / search

F1, precision, recall, and exact-set accuracy are percentages reported as mean ± sample SD across three independent runs; each run aggregates all 45 questions. SD is measured in percentage points. Median time is the median end-to-end time across all runs for each vendor. Median task cost is the median of LLM $ plus search API $ per agent run. API list price is the PAYG unit rate of the search endpoint the harness calls.

Search plus fetch, same sort

Search plus fetch, ranked by f1
ProviderEndpoint & configurationF1PrecisionRecallExact setMedian timeMean turnsMedian task costAPI list price
Exa deepPOST /search type=deepexa-deep48.2 ± 2.189.4 ± 1.236.0 ± 2.22.2 ± 2.295.8 s7.5$0.683$0.156 search+fetch$0.526 token cost$0.012 / search$0.001 / fetch
PerplexityPOST /searchsearch_context_size=high46.6 ± 2.087.7 ± 5.934.7 ± 1.12.2 ± 2.253.9 s7.5$0.504$0.070 search+fetch$0.441 token cost$0.005 / search
Exa instantPOST /search type=instantexa-instant44.9 ± 0.985.9 ± 2.533.5 ± 0.95.2 ± 1.352.5 s7.5$0.653$0.091 search+fetch$0.557 token cost$0.007 / search$0.001 / fetch
Parallel basicPOST /v1/search mode=basicparallel-basic42.3 ± 1.181.3 ± 2.831.3 ± 1.13.0 ± 1.368.6 s7.5$1.089$0.061 search+fetch$1.033 token cost$0.005 / search$0.001 / fetch
Parallel advancedPOST /v1/search mode=advancedparallel-advanced42.2 ± 1.187.6 ± 0.330.1 ± 1.12.2 ± 0.080.9 s7.4$0.599$0.061 search+fetch$0.538 token cost$0.005 / search$0.001 / fetch
Linkup standardPOST /v1/search depth=standardlinkup-standard42.0 ± 1.890.7 ± 2.030.5 ± 2.03.0 ± 3.481.0 s7.5$0.911$0.070 search+fetch$0.848 token cost$0.005 / search$0.001 / fetch
Tavily advancedPOST /search search_depth=advancedtavily-advanced41.0 ± 1.389.4 ± 6.029.1 ± 0.72.2 ± 0.092.6 s7.3$0.898$0.195 search+fetch$0.685 token cost$0.016 / search$0.0032 / fetch
Linkup fastPOST /v1/search depth=fastlinkup-fast39.9 ± 1.385.3 ± 3.628.6 ± 1.30.7 ± 1.368.0 s7.3$0.903$0.061 search+fetch$0.837 token cost$0.005 / search$0.001 / fetch
Parallel fastPOST /v1/search mode=fastparallel-fast39.3 ± 3.382.3 ± 6.628.2 ± 2.02.2 ± 0.055.1 s7.6$0.441$0.014 search+fetch$0.427 token cost$0.001 / search$0.001 / fetch
YouPOST /v1/searchextraction_mode=highlights38.5 ± 1.685.2 ± 2.726.9 ± 1.70.7 ± 1.345.6 s7.4$0.864$0.065 search+fetch$0.795 token cost$0.005 / search$0.001 / fetch
YouPOST /v1/searchextraction_mode=highlights · knowledge=core38.5 ± 2.478.2 ± 2.528.3 ± 3.03.7 ± 1.348.5 s7.4$0.894$0.065 search+fetch$0.830 token cost$0.005 / search$0.001 / fetch
Tavily basicPOST /search search_depth=basictavily-basic38.1 ± 1.481.2 ± 5.927.8 ± 0.72.2 ± 0.062.0 s7.8$0.610$0.112 search+fetch$0.507 token cost$0.008 / search$0.0032 / fetch
Parallel turboPOST /v1/search mode=turboparallel-turbo36.0 ± 3.583.6 ± 4.025.0 ± 2.60.0 ± 0.048.1 s7.7$0.414$0.013 search+fetch$0.400 token cost$0.001 / search$0.001 / fetch
FirecrawlPOST /v2/searchfirecrawl33.2 ± 2.183.3 ± 0.822.7 ± 1.81.5 ± 1.382.1 s7.9$0.295$0.063 search+fetch$0.230 token cost$0.005 / search$0.0025 / fetch
NimblePOST /v2/searchsearch_depth=standard · full_content=false · focus=general · extract formats=[markdown]31.3 ± 1.873.2 ± 1.921.7 ± 1.61.5 ± 1.359.2 s7.9$0.424$0.424 token cost$0.005 / search
TinyFishGET api.search.tinyfish.aitinyfish30.2 ± 3.570.9 ± 11.320.9 ± 2.50.0 ± 0.063.4 s7.9$0.219$0.000 search+fetch$0.219 token cost$0 / search$0 / fetch
Brave SearchGET /res/v1/web/searchbrave29.4 ± 1.673.5 ± 5.820.4 ± 0.91.5 ± 1.345.1 s8.0$0.285$0.060 search+fetch$0.222 token cost$0.005 / search
NimblePOST /v2/searchsearch_depth=lite · full_content=false · focus=general · extract formats=[markdown]27.1 ± 1.568.5 ± 4.418.7 ± 0.92.2 ± 0.072.0 s8.0$0.237$0.237 token cost$0.0011 / search
SeltzPOST /v1/search scope=companiesseltz-companies16.3 ± 1.549.5 ± 2.410.2 ± 1.20.0 ± 0.060.1 s7.5$1.741$0.070 search+fetch$1.671 token cost$0.005 / search
SERPGET google-search74.p.rapidapi.comserp0.0 ± 0.00.0 ± 0.00.0 ± 0.00.0 ± 0.033.4 s7.8$0.102$0.039 search+fetch$0.066 token cost$0.003 / search

F1, precision, recall, and exact-set accuracy are percentages reported as mean ± sample SD across three independent runs; each run aggregates all 45 questions. SD is measured in percentage points. Median time is the median end-to-end time across all runs for each vendor. Median task cost is the median of LLM $ plus search/fetch API $ per agent run. API list price is the PAYG unit rate of the search (and fetch) endpoint the harness calls.

Common questions

Which API can find a list of companies that match several criteria?

Parallel basic (parallel-basic) leads F1 at 46.5% on 45 questions that each define a company set by three or four constraints, such as investor, headquarters location, accelerator and funding stage. The agent is held constant; only the web search API changes.

Is this a company database API or a web search API?

Web search. A company database (enrichment or lookup API) answers from a stored record and cannot assemble a set defined by criteria it does not index. This benchmark measures whether a research agent can build that set from live search results. The model-only baseline is 0: none of these sets is recoverable from training data.

Can I use this to build account lists for GTM?

That is the workflow it approximates: automated account-list building, investor or accelerator portfolio discovery, and geography-scoped market maps. For similar-companies (lookalike) list building, which is a different job, see the lookalikes benchmark.