Agentic Search Tooling Benchmarks for Leading Local Models

Last updated: August 31, 2026

Designed and run by Scott Sanchez (X: @ScottSanchez · LinkedIn). The goal: help people understand which agentic search providers work best with the local models they actually run, measured on the attributes that matter: accuracy, cost, and speed.

Models tested:

DeepSeek V4 Flash 0731 (2x DGX Spark)

GLM 5.3 Flash (2x DGX Spark)

Qwen 3.8 27B (1x DGX Spark)

Qwen 3.8 Flash Next (2x DGX Spark)

Search providers tested:

Tavily

Best overall combination

Qwen 3.8 27B + Parallel turbo mode

This combination got 50 out of 50 questions correct, at 5.7 avg seconds per answer, and an avg cost of $1.26 per 1,000 questions. Six other pairings also aced the test, but none with this combination of speed and cost. Runner-up: DeepSeek V4 Flash + Serper, at 7.2 seconds per answer and $1.30 per 1,000 questions.

Search provider leaderboard

Accuracy

Parallel turbo

197/200 across all models

Speed

Parallel turbo

7.1s avg per answer (0.48s per search call)

Cost

Parallel turbo

$1.26 avg per 1,000 answers

Each award pools all four models' graded answers on the same questions.

Local Search Bench: Accuracy vs Cost vs Speed

Every search provider, pooled across all four local models. Up and to the left wins: accuracy on the vertical, effective search fees per 1,000 answers on the horizontal (log scale), bigger dot = faster answers. Cost is not the provider's list price: it is computed from this benchmark's measured usage, the actual number of searches each model chose to run on each question (typically 1 to 2), billed at the provider's per-search list rate. It answers "what would 1,000 answers actually have cost me," which is why every provider plots above its raw per-search price. Model-only baseline (5.5% accuracy, $0) is off the chart.

Credit: x.com/ScottSanchez

Accuracy vs cost vs speed scatter chart: Parallel turbo leads at 98.5% accuracy, 7.1s per answer, $1.26 per 1,000 answers

Download this chart as a PNG (free to share)

Sort providers by click the chips to set your own priority order · default: accuracy, then cost, then speed

accuracy = correct answers out of 50 time = full wall clock: searching + page reading + model inference tokens: big number = input read; small line = output written search cost = list-price fees for the searches actually used, scaled to 1,000 answers

Results by question

Show the full question-by-question grid50 questions, every provider, all four models. Click to expand

50 questions, grouped by question type. Provider columns run left to right by overall rank: best accuracy first, ties broken by lowest cost, then fastest, matching the head-to-head default; "model only" sits last as the no-search baseline.

correct wrong answer didn't answer G = GLM 5.3 Flash (2x DGX Spark) · Q = Qwen 3.8 27B (1x DGX Spark) · F = Qwen 3.8 Flash Next (2x DGX Spark) · D = DeepSeek V4 Flash 0731 (2x DGX Spark)

Wall time per task (all 50 questions)

GLM 5.3 Flash (2x DGX Spark)Qwen 3.8 27B (1x DGX Spark)Qwen 3.8 Flash Next (2x DGX Spark)DeepSeek V4 Flash 0731 (2x DGX Spark)

Key observations: models and provider combinations.

The short answer: for most local AI users, Qwen 3.8 27B + Parallel turbo likely wins with the best blend of accuracy, cost, and speed: all 50 questions answered correctly, fastest of any perfect pairing (5.7s per answer), at $1.26 per 1,000 questions in search fees.
Any of these search APIs makes a local model genuinely good. Without search, the four models answer 1 to 4 of 50 correctly. With search, even the single worst pairing tested scored 44/50, and DeepSeek never dropped below 48/50 with any provider. You are choosing among good options, not dodging bad ones; the leaderboard is a margins game.
Expect to pay more than the provider's list price, because models search more than once per task. Every model averaged between about 1 and 1.7 searches per question regardless of provider, so budget roughly 1.3x the per-search rate for real workloads: a $1/1k tier lands near $1.30 per 1,000 answers, a $16/1k tier near $20. Accuracy did not track price across providers: the $1/1k tiers took first (Parallel turbo, 197/200) and fourth (Serper, 193) while the $16/1k tier bought 195. One caveat: within a single provider's lineup, cheaper is not always better (Exa's cheapest mode was its least accurate). Across providers, spending more bought no accuracy.
The timing data points at payload size, not API latency. Brave's search calls average 0.67 seconds, yet Brave answers take 12.8 seconds, because the local model spends the time reading and reasoning over the larger result payloads. Parallel turbo is fast twice over: quick calls (0.48s each), and lean excerpts that roughly halve the model's own reading-and-thinking time (6.5s vs 10-12s on richer payloads). In our timings the model's reading time, driven by how much text the provider returns, dominates wall clock; API latency is a minor term. If your agent feels slow, measure payload size before blaming the API.
A crisp search result can make up for a weaker model. Qwen Flash Next, the weakest model here, scored 49/50 on Parallel turbo's lean excerpts but as low as 44 on providers that hand it rich payloads; on Brave it rambled to 1,223 output tokens and 22.7s per answer, nearly four times its turbo speed. The stronger models never needed the help (Qwen 27B and DeepSeek scored 47-50 with every provider), so the weaker your model, the more a tight, focused payload buys you.
These rankings reflect out-of-the-box behavior, not each product's ceiling. Every model and every search provider has tuning knobs that could plausibly shift these results: sampling and prompt settings on the model side; result counts, content depth, freshness and livecrawl policies, query shaping, and payload sizing on the provider side. We deliberately ran each one at the defaults or the settings its own docs recommend for agent use, because that is what most people will actually deploy. Treat the rankings as "how these products behave as shipped," not as the best any of them could do.
When the answer changed recently, index freshness decides. The most-missed lookup outside date math (the Mac Studio's current top chip) came down to crawl recency: Brave gave all four models the current answer, while Exa's fast and auto modes fed three of the four stale excerpts naming the superseded chip.

How this benchmark works.

The question being answered: if you run an AI model on your own hardware and give it web search, which search API (and which speed/quality tier of it) gives you the best answers, at the best price, fastest? Every result on this page is a (local model × search provider) pairing measured on those three things.
The setup: four open-weight models running on NVIDIA DGX Spark machines are each asked the same 50 questions through ten search configurations across five providers (Brave, Exa, Parallel, Serper, Tavily, including their fast/cheap and premium tiers). For each question the model gets three tools: web_search (wired to the provider being tested), web_fetch (one shared, provider-neutral page fetcher, identical for every config, so no provider gets fetch credit), and finish to submit a short answer. Temperature 0, at most 5 searches and 10 turns per question. Each provider's native response is passed to the model with only UI chrome stripped; how rich or lean their results are is part of what's being tested.
The control: every model also answers all 50 questions with no tools at all ("model only"). Those baselines score 1–4 out of 50, which is the proof that the scores here measure search, not memorization.
The questions: 50 total, grouped into five types (current-state lookups, recent-event facts, multi-hop chains, search-plus-arithmetic, and static trivia). The set deliberately leans toward fresh 2026-era facts, since that is the workload where search choice matters. Every gold answer was verified against at least two independent sources before any run; "what is X right now" answers are re-verified at grading time.
Grading: answers matching the gold or an accepted alias are scored mechanically. Anything needing judgment is decided by Claude (Fable 5) acting as an independent judge; the local models never grade themselves. Infrastructure failures (a local endpoint hiccup) are retried and never scored against a provider. Every graded task is published in full: results_full.csv (2,200 rows: every question, model, config, answer, grade, and timing). As a cross-check, an unaffiliated second model (xAI's Grok 4.6) independently audited that file: it recomputed every published number, re-verified all mechanical grades, and reviewed the judged grades for errors, finding none.
Ranking: accuracy first, then search cost, then speed. Cost or speed differences inside 15% are treated as ties that fall through to the next measure, because at these scales the raw numbers are too small and too noisy to rank on honestly: the costs being compared differ by fractions of a cent per answer, and per-query latency varies run to run by more than that margin. A provider should only win a tiebreak on a difference big enough to survive the noise. Cost is the provider's list-price search fees only; model tokens are free on your own hardware. Thinking/reasoning is disabled on every model ("agent mode"): enable_thinking: false for GLM and the Qwens, reasoning_effort: "none" for DeepSeek; the same lever position everywhere.
Fine print: endpoints and parameters verified against each provider's current docs as of Aug 30, 2026 (config settings: Brave web search with extra_snippets; Brave LLM Context /res/v1/llm/context; Exa type=instant/fast/auto with highlights; Parallel /v1/search mode=turbo/advanced; Tavily search_depth=basic/advanced; Serper standard). Runs executed Aug 30–31, 2026. Speed differences between providers are fair comparisons (same model, same hardware); speed differences between models are not apples-to-apples, since Qwen 3.8 27B is a much smaller model that runs comfortably on a single DGX Spark (even that is overkill) while the other three models need two. Search fees use each provider's entry-tier pay-as-you-go list price, verified Aug 31, 2026: Brave $5/1k, Brave LLM Context $5/1k, Exa $7/1k, Parallel turbo $1/1k, Parallel advanced $5/1k, Tavily basic $8/1k, Tavily advanced $16/1k, Serper $1/1k (Serper falls to $0.30/1k at its largest volume pack; entry tiers are used consistently for all providers).

Have feedback or suggestions?

Please reach out to me on X: @ScottSanchez

Disclaimer: I am a marketing guy, not a professional benchmarker. I did my best here, and if I got something wrong, please cut me some slack. I ran these numbers for my own curiosity, using public data, public docs, public API endpoints, and public model weights, as of August 31, 2026. Things change fast and your workload is not my workload, so run your own benchmarks before relying on anything here. If you spot an error or something I could improve for next time, please reach out on X (@ScottSanchez).