Methodology

Version 1.0 · 2026-Q3 · Perplexity Sonar · 2026-07-20

This page exists so anyone can check our work, disagree with it, or reproduce it. Where a number is uncertain we say so and publish the bound rather than a cleaner-looking point estimate.

What the report measures

Two different things, from the same API responses. They are reported separately and never blended.

Answer Share — whether a law firm is named in the text of the AI’s answer. This is the headline. It is what a prospective client actually reads.

Citation Share — whether a domain is cited as a source beneath that answer. A firm can be named without its website being cited, and a website can be cited without the firm being named. Those are different phenomena and conflating them would be an error.

The prompt set

540 prompts, constructed by crossing three fixed dimensions:

• 6 practice areas — criminal defense, employment law, family law, immigration, medical malpractice, personal injury.
• 30 US metros across 23 states, selected by legal-market size.
• 3 templates, held constant:
  1. top {practice area} law firms in {city} {state}
  2. who should I hire for a {practice area} case in {city} {state}
  3. best {practice area} attorneys in {city} {state}

The template set is frozen. Changing the wording of a template creates a new template ID rather than editing an existing one, because period-over-period comparison is meaningless if the questions drift.

One template was tested and excluded. A pilot form — best {practice area} lawyer in {city} {state}, singular — returned only 1–4 sources per answer against 14–18 for the templates above. It was dropped before the main collection for low yield, not because of the results it produced.

How the data was collected

Each of the 540 prompts was submitted to Perplexity Sonar 3 times, giving 1,620 API calls. All 1,620 succeeded. The full answer text and the complete source payload were retained for every call, so every derived figure can be recomputed without re-querying the API.

Sonar is non-deterministic: the same prompt returns different answers on different runs. This is the single most important fact about the dataset and it drives the metric design below.

How Answer Share is computed

The headline metric is a consistency threshold, not a percentage.

For each prompt, firm names are extracted from the answer text, then a firm counts as consistently named if it appears in at least 2 of the 3 runs.

We publish a threshold instead of a share because a share would mostly encode noise. Across the corpus, roughly half of all firms named appear in only one of three runs. Reporting “Firm X holds 12.4% answer share” would be reporting which run we happened to catch. The underlying run counts are carried on every row, so anyone can recompute at a stricter or looser threshold.

Firm-name extraction

Names are extracted by combining two signals:

1. Emphasis — Sonar bolds firm names in its answers.
2. Firm shape — the candidate must contain a law-firm marker (LLP, P.C., Law, Attorneys, and similar).

Requiring both is deliberate. Emphasis alone is unusable: Sonar also bolds section headers and topic phrases, and an emphasis-only extractor scored a stability of 0.14 against 0.38 for the combined approach.

Extracted names are then alias-collapsed by surname overlap, so Salvi, Schostok & Pritchard P.C. and Schostok & Pritchard resolve to one firm rather than two. Collapsing requires at least two shared surnames; single-surname names merge only on exact match, because a looser rule merged the distinct firms Modern Family Law and Modern Law Group.

This is pattern matching, not a curated firm registry. Some name variants will still split, and some non-firm entities will survive the filter.

How Citation Share is computed

Every source URL is reduced to its domain and classified. Counting is fully deterministic; no language processing is involved.

Source classification — and why it is published as a bound

Classification is the most attackable part of this report, so here is exactly how it works. Domains are assigned to one of: directory, firm, media, review platform, gov, edu, other.

• Known legal directories, review platforms, news and reference sites are classified from a curated list, matched on subdomains as well (lawyers.findlaw.com is a directory, not a separate entity).
• A domain is classified as a firm only on positive evidence: its registrable name must encode the surnames of a firm the AI itself named, requiring at least two surnames (or one of six-plus characters) covering at least 60% of the domain label.
• Directory-shaped names (top…, best…, rank…, find…) are classified as directories before the firm test, so bestlawyers.us.com is not mistaken for a firm.
• Anything else stays other. Unreviewed is never assumed to be a firm.

The strict evidence bar buys precision at the cost of recall. 36% of citation volume remains unclassified, so the firm-versus-directory split is published as a bound:

Directories are at least 51.8% of all citations. Law firms are at most 36%, and the true figure is lower, because the unclassified bucket still contains directories.

We would rather publish a wide honest bound than a narrow number we cannot defend. These are the errors the bar was built to prevent — all real, all caught during development:

Five ambiguous high-frequency domains were resolved by fetching the live site and reading its title (counselindex.com, toplawdog.com, donthireblind.com, legalrank.co, getcodeshealth.com); each is recorded with that provenance.

Known limitations

1. Single engine. Perplexity Sonar only; ChatGPT, Gemini and Google AI Overviews are not measured in this period and may differ materially.
2. Three runs per prompt. Sonar is non-deterministic; roughly half of all firms named appear in only one of three runs. That is why the published metric is a consistency threshold rather than a percentage share.
3. Source classification is incomplete. 36.0% of citation volume sits on domains not yet classified as firm or directory, so the firm-versus-directory split is published as a bound.
4. Firm names are extracted from answer prose by pattern matching and alias collapsing, not by a curated firm registry. Name variants may still split.
5. Metros, not full state coverage. 30 metros across 23 states; results do not generalise to rural markets.
6. No causal claim. Being cited is not the same as being hired, and this report measures visibility only.

Reproducibility and corrections

The prompt frame, the run-level raw responses, the extractor and the classifier are all retained. Every published figure is derived from those raw runs by an offline transform, so improving the extractor recomputes the entire history without new API calls.

Corrections. If you find an error — a misclassified domain, a firm named incorrectly, a figure that does not reconcile — tell us and we will correct it and note the correction here.

← Back to the report