Skip to content
Preferium

Blog

Similar web-research scores do not mean identical AI answers

BrowseComp scores for leading models are close. That does not establish equal citation behavior, or explain why one assistant names a client and another does not.

Answers, mentions and sources
All articles
Updated 2 min read · Preferium team
  • AI search
  • AI visibility
Three horses, one white, one grey, one black, gallop side by side across a dusty plain, all exactly level with each other.

The BrowseComp results reported around the GPT-6 Astra launch are close: 91.5% for Astra, 90.8% for Claude Opus 5 and 90.4% for GPT-5.6 Sol. It is tempting to conclude that search quality no longer explains differences between assistants. The benchmark does not support that conclusion.

What the comparison measures

BrowseComp tests difficult web-research questions. It measures success on those tasks under a particular configuration. OpenAI’s release report provides the launch context; the earlier comparison in this article drew on the tables compiled by Vellum.

Before comparing scores, check tool access, effort settings and the evaluation method. Similar totals do not mean the models solved the same questions or used the same sources. They also do not establish that a consumer assistant runs with the benchmark’s configuration.

The scores can inform an assessment of research capability. They cannot tell an agency how often a particular client will be named in an answer.

Why citation results can still differ

An assistant’s answer depends on more than its underlying model. Search providers, indexes, query generation, access rules and product settings all influence what it can retrieve. The system then has to select and present sources.

A citation gap can therefore have several causes. It may reflect a blocked page or outdated content. It may also reflect different prompts, different source selection or normal variation between runs. A high benchmark score does not let you rule out any of those explanations.

Our earlier wording claimed model quality no longer explained missing citations. That went beyond the evidence. It also compared different computer-use benchmarks as if they measured the same thing. Those comparisons have been removed.

Investigate the actual client queries

Start with the question the client cares about. Record the assistant, visible model or mode, date, language and cited URLs. Repeat the query under comparable conditions before deciding the result represents a pattern.

If one assistant consistently omits the client, inspect access to the relevant pages and check the information they contain. The JavaScript-link experiment gives an example of a specific access problem to test; it is not a universal explanation for every missing citation.

Keep the finding proportionate to the evidence. “This assistant did not cite the client in these runs” is a usable observation. Explaining why requires a separate investigation.

White label

Offer this under your brand.

Your brand, Stripe, packages and prices. Partner access opens by invitation from the waitlist.

All articles