Precision vs recall vs F1 for web search APIs
What the benchmark found. Current output of the method: Parallel basic (parallel-basic) leads F1 at 46.5% across 16 configurations on 45 questions.
What this page compares. Three fanout strings in the probe asked for a precision and recall comparison of web search APIs and found no page. This is that page: the definitions as used on this board, and the board.
How to read it. Precision: of the companies the agent returned, the share that satisfy every constraint. Recall: of the gold set, the share the agent recovered. F1: their harmonic mean. Exact-set: the share of questions where the returned set matched the gold set exactly. High precision with low recall is a cautious API; the reverse is a padding one.
Method in one paragraph. The same agent, prompt, turn budget and tool schema run against every web search API on 45 questions that each define a company set by three or four constraints. Every returned company is resolved to a canonical identity and compared with a human-reviewed gold set; precision, recall, F1 and exact-set are averaged over three runs per question. The model-only baseline is 0.
Most accurate · Fastest · Cheapest · Find companies by criteria · Search + fetch · Full benchmark
What each term means here
Which matters more for a research agent, precision or recall?
It depends on what happens downstream. If a human reviews the list, recall matters more because a missed company is never seen. If the list feeds automation, precision matters more because every wrong company costs an action. F1 is the neutral ranking; the board shows both so the reader can weight them.
How is a returned company matched to the gold set?
Every returned company is resolved to a canonical identity (domain and LinkedIn where available) before comparison, so name variants and subsidiaries do not create false negatives or false positives.
Ranked by F1, search-only
| Provider | Endpoint & configuration | F1 | Precision | Recall | Exact set | Median time | Mean turns | Median task cost | API list price |
|---|---|---|---|---|---|---|---|---|---|
| Parallel | POST /v1/search mode=basicparallel-basic | 46.5 ± 1.9 | 88.7 ± 0.9 | 34.4 ± 2.0 | 3.7 ± 2.6 | 67.6 s | 7.4 | $1.120$0.070 search$1.050 token cost | $0.005 / search |
| Exa | POST /search type=deepexa-deep | 45.4 ± 2.0 | 83.2 ± 3.0 | 33.7 ± 1.4 | 1.5 ± 1.3 | 89.5 s | 7.4 | $0.717$0.156 search$0.556 token cost | $0.012 / search |
| Parallel | POST /v1/search mode=advancedparallel-advanced | 44.2 ± 1.4 | 87.6 ± 3.4 | 32.0 ± 1.6 | 2.2 ± 0.0 | 83.4 s | 7.3 | $0.625$0.070 search$0.565 token cost | $0.005 / search |
| Exa | POST /search type=instantexa-instant | 43.3 ± 1.0 | 82.6 ± 3.8 | 32.2 ± 0.5 | 3.0 ± 1.3 | 49.8 s | 7.3 | $0.652$0.091 search$0.559 token cost | $0.007 / search |
| Linkup | POST /v1/search depth=fastlinkup-fast | 41.1 ± 1.7 | 82.8 ± 1.0 | 30.3 ± 1.5 | 0.7 ± 1.3 | 64.2 s | 7.4 | $0.923$0.070 search$0.853 token cost | $0.005 / search |
| Tavily | POST /search search_depth=advancedtavily-advanced | 41.1 ± 2.3 | 83.7 ± 4.2 | 29.8 ± 1.7 | 2.2 ± 0.0 | 92.2 s | 7.4 | $1.029$0.224 search$0.808 token cost | $0.016 / search |
| Linkup | POST /v1/search depth=standardlinkup-standard | 40.6 ± 0.9 | 84.0 ± 7.9 | 29.7 ± 1.4 | 1.5 ± 2.6 | 72.3 s | 7.3 | $0.937$0.070 search$0.867 token cost | $0.005 / search |
| Parallel | POST /v1/search mode=fastparallel-fast | 38.0 ± 2.0 | 79.9 ± 4.6 | 27.5 ± 1.4 | 1.5 ± 1.3 | 53.9 s | 7.6 | $0.460$0.014 search$0.447 token cost | $0.001 / search |
| Perplexity | POST /searchsearch_context_size=low | 37.8 ± 2.1 | 79.3 ± 5.7 | 26.8 ± 1.4 | 2.2 ± 2.2 | 48.9 s | 7.7 | $0.334$0.070 search$0.264 token cost | $0.005 / search |
| Parallel | POST /v1/search mode=turboparallel-turbo | 34.7 ± 2.4 | 80.0 ± 5.2 | 24.8 ± 2.4 | 1.5 ± 1.3 | 46.4 s | 7.6 | $0.419$0.014 search$0.405 token cost | $0.001 / search |
| You | POST /v1/searchyou | 33.1 ± 2.4 | 75.7 ± 3.1 | 23.1 ± 1.9 | 1.5 ± 1.3 | 46.2 s | 7.7 | $0.477$0.070 search$0.408 token cost | $0.005 / search |
| Firecrawl | POST /v2/searchfirecrawl | 30.4 ± 1.1 | 77.3 ± 4.4 | 20.7 ± 0.7 | 2.2 ± 0.0 | 75.0 s | 7.9 | $0.282$0.070 search$0.212 token cost | $0.005 / search |
| Brave Search | GET /res/v1/web/searchbrave | 28.0 ± 1.7 | 66.9 ± 8.5 | 19.3 ± 0.5 | 0.7 ± 1.3 | 43.5 s | 8.0 | $0.268$0.070 search$0.200 token cost | $0.005 / search |
| TinyFish | GET api.search.tinyfish.aitinyfish | 26.6 ± 1.3 | 64.7 ± 1.5 | 17.9 ± 0.8 | 1.5 ± 1.3 | 64.2 s | 7.9 | $0.212$0.000 search$0.212 token cost | $0 / search |
| Seltz | POST /v1/search scope=companiesseltz-companies | 14.5 ± 0.9 | 40.0 ± 3.1 | 9.4 ± 0.7 | 0.0 ± 0.0 | 55.2 s | 7.3 | $1.751$0.070 search$1.681 token cost | $0.005 / search |
| SERP (RapidAPI) | GET google-search74.p.rapidapi.comserp | 0.4 ± 0.6 | 0.7 ± 1.3 | 0.3 ± 0.4 | 0.0 ± 0.0 | 31.4 s | 7.8 | $0.103$0.036 search$0.065 token cost | $0.003 / search |
F1, precision, recall, and exact-set accuracy are percentages reported as mean ± sample SD across three independent runs; each run aggregates all 45 questions. SD is measured in percentage points. Median time is the median end-to-end time across all runs for each vendor. Median task cost is the median of LLM $ plus search/fetch API $ per agent run. API list price is the PAYG unit rate of the search (and fetch) endpoint the harness calls.
Same sort, with a page-reading tool enabled
| Provider | Endpoint & configuration | F1 | Precision | Recall | Exact set | Median time | Mean turns | Median task cost | API list price |
|---|---|---|---|---|---|---|---|---|---|
| Exa | POST /search type=deepexa-deep | 48.2 ± 2.1 | 89.4 ± 1.2 | 36.0 ± 2.2 | 2.2 ± 2.2 | 95.8 s | 7.5 | $0.683$0.156 search+fetch$0.526 token cost | $0.012 / search$0.001 / fetch |
| Perplexity | POST /searchsearch_context_size=high | 46.6 ± 2.0 | 87.7 ± 5.9 | 34.7 ± 1.1 | 2.2 ± 2.2 | 53.9 s | 7.5 | $0.504$0.070 search+fetch$0.441 token cost | $0.005 / search |
| Exa | POST /search type=instantexa-instant | 44.9 ± 0.9 | 85.9 ± 2.5 | 33.5 ± 0.9 | 5.2 ± 1.3 | 52.5 s | 7.5 | $0.653$0.091 search+fetch$0.557 token cost | $0.007 / search$0.001 / fetch |
| Parallel | POST /v1/search mode=basicparallel-basic | 42.3 ± 1.1 | 81.3 ± 2.8 | 31.3 ± 1.1 | 3.0 ± 1.3 | 68.6 s | 7.5 | $1.089$0.061 search+fetch$1.033 token cost | $0.005 / search$0.001 / fetch |
| Parallel | POST /v1/search mode=advancedparallel-advanced | 42.2 ± 1.1 | 87.6 ± 0.3 | 30.1 ± 1.1 | 2.2 ± 0.0 | 80.9 s | 7.4 | $0.599$0.061 search+fetch$0.538 token cost | $0.005 / search$0.001 / fetch |
| Linkup | POST /v1/search depth=standardlinkup-standard | 42.0 ± 1.8 | 90.7 ± 2.0 | 30.5 ± 2.0 | 3.0 ± 3.4 | 81.0 s | 7.5 | $0.911$0.070 search+fetch$0.848 token cost | $0.005 / search$0.001 / fetch |
| Tavily | POST /search search_depth=advancedtavily-advanced | 41.0 ± 1.3 | 89.4 ± 6.0 | 29.1 ± 0.7 | 2.2 ± 0.0 | 92.6 s | 7.3 | $0.898$0.195 search+fetch$0.685 token cost | $0.016 / search$0.0032 / fetch |
| Linkup | POST /v1/search depth=fastlinkup-fast | 39.9 ± 1.3 | 85.3 ± 3.6 | 28.6 ± 1.3 | 0.7 ± 1.3 | 68.0 s | 7.3 | $0.903$0.061 search+fetch$0.837 token cost | $0.005 / search$0.001 / fetch |
| Parallel | POST /v1/search mode=fastparallel-fast | 39.3 ± 3.3 | 82.3 ± 6.6 | 28.2 ± 2.0 | 2.2 ± 0.0 | 55.1 s | 7.6 | $0.441$0.014 search+fetch$0.427 token cost | $0.001 / search$0.001 / fetch |
| Parallel | POST /v1/search mode=turboparallel-turbo | 36.0 ± 3.5 | 83.6 ± 4.0 | 25.0 ± 2.6 | 0.0 ± 0.0 | 48.1 s | 7.7 | $0.414$0.013 search+fetch$0.400 token cost | $0.001 / search$0.001 / fetch |
| You | POST /v1/searchyou | 34.0 ± 0.9 | 78.8 ± 0.6 | 23.8 ± 1.2 | 3.0 ± 1.3 | 47.4 s | 7.8 | $0.489$0.065 search+fetch$0.427 token cost | $0.005 / search$0.001 / fetch |
| Firecrawl | POST /v2/searchfirecrawl | 33.2 ± 2.1 | 83.3 ± 0.8 | 22.7 ± 1.8 | 1.5 ± 1.3 | 82.1 s | 7.9 | $0.295$0.063 search+fetch$0.230 token cost | $0.005 / search$0.0025 / fetch |
| TinyFish | GET api.search.tinyfish.aitinyfish | 30.2 ± 3.5 | 70.9 ± 11.3 | 20.9 ± 2.5 | 0.0 ± 0.0 | 63.4 s | 7.9 | $0.219$0.000 search+fetch$0.219 token cost | $0 / search$0 / fetch |
| Brave Search | GET /res/v1/web/searchbrave | 29.4 ± 1.6 | 73.5 ± 5.8 | 20.4 ± 0.9 | 1.5 ± 1.3 | 45.1 s | 8.0 | $0.285$0.060 search+fetch$0.222 token cost | $0.005 / search |
| Seltz | POST /v1/search scope=companiesseltz-companies | 16.3 ± 1.5 | 49.5 ± 2.4 | 10.2 ± 1.2 | 0.0 ± 0.0 | 60.1 s | 7.5 | $1.741$0.070 search+fetch$1.671 token cost | $0.005 / search |
| SERP (RapidAPI) | GET google-search74.p.rapidapi.comserp | 0.0 ± 0.0 | 0.0 ± 0.0 | 0.0 ± 0.0 | 0.0 ± 0.0 | 33.4 s | 7.8 | $0.102$0.039 search+fetch$0.066 token cost | $0.003 / search |
F1, precision, recall, and exact-set accuracy are percentages reported as mean ± sample SD across three independent runs; each run aggregates all 45 questions. SD is measured in percentage points. Median time is the median end-to-end time across all runs for each vendor. Median task cost is the median of LLM $ plus search/fetch API $ per agent run. API list price is the PAYG unit rate of the search (and fetch) endpoint the harness calls.
How to evaluate a web search API for AI agents → · BrowseComp vs LiveBrowseComp vs multi-turn company search →