AI search experiments: we test what actually moves AI answers.
GEO advice is full of confident claims and very little evidence. So we run controlled experiments on ChatGPT, Perplexity, Gemini, Copilot and Google's AI features — with control pages, fixed prompt sets and repeated sampling — to separate what changes citations from what just sounds clever.
on this page
sampled
for 98+ industries
Opinions are cheap. Controlled tests aren't.
AI answer engines change fast, publish little about how they choose sources, and give different answers to the same question. That's a perfect environment for myths.
Some platforms say plainly that no special tricks are needed. Google's own documentation states there are "no additional requirements to appear in AI Overviews or AI Mode" and that you don't need "new machine readable files, AI text files, or markup" to appear in them. Source: Google Search Central, 2025 Yet agencies sell exactly those things as GEO.
Our position is simple: if we recommend it to a client, we should be able to show evidence for it — either from peer-reviewed research or from our own controlled tests. The experiments on this page are how we earn that right, and they directly shape our citation optimisation and ChatGPT SEO playbooks.
- We publish designs before results so we can't move the goalposts.
- We report null results — "no measurable effect" is a useful finding.
- We never present a test result as a guarantee for your site.
How we keep AI search tests honest.
AI answers are noisy. Without these safeguards, almost any change can look like it "worked".
Control groups
Every test pairs changed pages with matched, unchanged control pages — similar topic, age, authority and traffic. We compare the difference in movement, not raw before/after numbers.
Prompt variance
Each target page is tested against a cluster of prompts — several phrasings of the same intent — so a result isn't an artefact of one lucky wording.
Repeated sampling
Each prompt is run multiple times per engine per measurement window, from clean sessions. We report rates and ranges, never single screenshots.
Pre-registration
Hypothesis, metrics, sample and duration are written down before the change ships. No adding metrics afterwards to find something that moved.
Time windows
Baseline and test windows of equal length, long enough to cover crawl and index delays — and logged against known model or platform updates.
One variable at a time
A test page changes one thing. If we restructure, add statistics and update the date together, we've learned nothing about any of them.
Why repeated sampling matters so much
SparkToro and Gumshoe's January 2026 study (2,961 runs by 600 volunteers across ChatGPT, Claude and Google's AI) found less than a 1-in-100 chance that two responses from ChatGPT or Google's AI list the same brands. Any experiment that compares one "before" answer with one "after" answer is measuring noise. Source: SparkToro, 2026
What on the page makes AI quote it?
The first group tests content changes — the levers you control most directly. Each card shows the design; findings are shared with subscribers and clients as each test completes.
E1 · Answer-first vs narrative introductions
Hypothesis: pages that answer the target question in the first 40–60 words are cited more often than pages that build up to the answer.
Method: matched pairs of informational pages; test pages get a direct-answer opening paragraph, controls keep their narrative intro. Body content otherwise unchanged.
We measure: citation rate per page across a prompt cluster; whether the cited passage is the new opening.
Why it matters: if confirmed, it's the cheapest GEO fix there is — a rewrite of the first paragraph on existing pages.
E2 · Adding statistics and expert quotations
Hypothesis: adding sourced statistics and attributed quotations increases how much of a page is used in AI answers — replicating, on live engines and commercial topics, what the Princeton-led GEO paper found in a benchmark.
Method: three arms — statistics added, quotations added, control — with the same word count change across arms.
We measure: citation rate, share of answer text attributable to the page, mention position.
Why it matters: it tests whether a peer-reviewed lab result holds up for real businesses. See the research box below.
E3 · Comparison tables vs prose comparisons
Hypothesis: "X vs Y" and "best X" pages with a structured HTML comparison table are cited more often for comparison prompts than equivalent prose.
Method: identical information presented as a table on test pages and as paragraphs on controls; prompts focus on "compare", "difference between" and "which is better".
We measure: citation rate on comparison prompts; accuracy of attributes repeated in answers.
Why it matters: comparison prompts are where buying decisions happen — see our prompt library.
E4 · Genuine content refresh vs date-only updates
Hypothesis: substantive updates with a visible "last updated" date improve citation on time-sensitive prompts; changing the date without changing content does not.
Method: three arms — real refresh + new date, date change only, untouched control — on pages targeting "2026", "latest" and pricing prompts.
We measure: citation rate on time-sensitive prompts; recrawl timing from server logs.
Why it matters: separating real freshness from cosmetic "date bumping" protects clients from a tactic that can erode trust.
E5 · Multimedia: transcripts, image alt text and video
Hypothesis: pages with an embedded video plus full transcript and descriptive alt text earn more citations — and more video citations — than text-only equivalents.
Method: add a short explainer video with transcript and chapters to test pages; controls stay text-only.
We measure: citation rate; whether answers cite the page, the video platform URL, or both.
Why it matters: Pew's 2025 analysis found Wikipedia, YouTube and Reddit among the most-cited sources in Google's AI summaries. Source: Pew Research Center, 2025
Want your site in a test cohort?
We recruit a small number of sites per experiment, matched by industry and authority. You get early access to findings and a free AI visibility baseline. Your brand is never named in what we publish.
Apply to joinDon't wait for the findings to start winning.
Our audits apply what the evidence already supports — today. Get a free AI visibility audit with prioritised, evidence-backed fixes.
Crawlers, structured data and the wider web.
The second group tests technical access and off-site signals — the claims most often sold as GEO "must-haves" with the least evidence behind them.
E6 · Schema markup vs no schema
Hypothesis: accurate FAQPage, Product, Organization and Article schema has a measurable effect on citation or factual accuracy in at least some engines — even though Google says no special schema is required for its AI features.
Method: matched page pairs; test pages get complete, valid schema matching visible content; controls have none. Run separately per engine.
We measure: citation rate and accuracy of facts (prices, hours, ratings) in answers.
Why it matters: schema is cheap, but clients deserve to know whether it helps visibility, accuracy, both — or neither.
E7 · Does llms.txt change anything?
Hypothesis (null): adding an llms.txt file has no measurable effect on citations in major consumer AI search engines.
Method: test sites publish a curated /llms.txt; matched controls don't. We also check server logs for which agents, if any, request the file.
We measure: requests to /llms.txt by user agent; citation rate vs controls.
Why it matters: llms.txt is a proposal by Jeremy Howard (September 2024), not an adopted standard. Source: llmstxt.org, 2024 Clients should know whether it's worth the effort.
E8 · IndexNow and time-to-citation
Hypothesis: submitting new and updated URLs via IndexNow shortens the time until they're cited in Bing-grounded experiences such as Microsoft Copilot.
Method: new pages published in pairs; one submitted via IndexNow immediately, the other left to normal discovery.
We measure: days from publish to first crawl (logs), first Bing index, and first citation per engine.
Why it matters: for launches and news, speed to citation is the whole game. Related: Copilot SEO.
E9 · robots.txt rules for AI crawlers
Hypothesis: blocking search-specific AI crawlers (e.g. OAI-SearchBot, PerplexityBot) removes pages from citations, while blocking training-only agents (e.g. GPTBot, Google-Extended) does not affect live citations.
Method: on consenting test sections, toggle specific user-agent rules and monitor over fixed windows; logs confirm crawler behaviour.
We measure: crawler hits by agent, citation rate per engine before/after.
Why it matters: blanket "block all AI bots" rules are common — and can quietly remove a brand from AI search. See Perplexity's crawler docs.
E10 · Ethical Reddit & community participation
Hypothesis: genuine, disclosed expert participation in relevant community threads increases a brand's mention rate on prompts where those threads are cited.
Method: brand experts answer real questions in their own name, clearly disclosing affiliation, following each community's rules. No fake accounts, no vote manipulation, no astroturfing.
We measure: mention rate and sentiment on prompts where community threads are cited; thread citations over time.
Why it matters: Reddit content is licensed to AI companies — OpenAI announced a partnership in 2024. Source: OpenAI, 2024 The ethical way in is to be genuinely useful.
What we will never test
- Hidden text or prompt injection aimed at AI crawlers
- Fake reviews, sockpuppet accounts or vote manipulation
- Mass AI-generated "listicles" that rank ourselves first
- Cloaking — showing bots different content than people
These may move a number briefly. They also put real brands at risk. We don't do them for clients, so we don't test them either.
What the peer-reviewed evidence says so far.
Our experiments build on the best public research available. Here's a summary — all verifiable at the source. More in our GEO statistics library.
The GEO paper (Aggarwal et al., KDD 2024)
Researchers from Princeton University and IIT Delhi, with independent co-authors, coined "generative engine optimization" and built GEO-bench, a benchmark of 10,000 queries across 25 domains. They tested nine content-editing methods and measured how visible a source became in generated answers. Headline: GEO methods boosted visibility by up to 40%, and results were validated on Perplexity.ai. Effectiveness varied by domain.
Scores below: position-adjusted word count (overall) from the paper's main results table — higher means more of the answer drew on the source. Bars are scaled for comparison.
Source: Aggarwal et al., "GEO: Generative Engine Optimization", KDD 2024
Our read: evidence-rich content (quotes, statistics, sources) helped; keyword stuffing — a classic SEO habit — scored below the unoptimised baseline. That's why E2 replicates this on live engines with commercial queries.
Share of Google visits with a click on a traditional result, with vs without an AI summary (March 2025 browsing data, 900 US adults).
Lower average CTR for position-one pages when an AI Overview appears, across 300,000 keywords.
Chance that ChatGPT or Google's AI returns the same brand list in two responses to the same prompt.
special schema, AI text files or markup required to appear in Google AI Overviews or AI Mode, per Google.
Hypotheses queued for the next round.
Ideas come from client audits, platform changes and questions we hear every week. These are next in line once current tests complete — tell us which you'd like prioritised.
Author bylines & credentials
Do pages with named, credentialed authors and linked Person schema get cited more on expertise-heavy prompts (health, finance, legal) than unattributed pages?
Original data vs aggregated stats
Is a page publishing first-party survey data cited more — and by more third parties — than a page curating other people's statistics?
FAQ blocks on commercial pages
Does adding concise, genuine Q&A sections to service and product pages change citation on long-tail conversational prompts?
Server-side vs client-side rendering
Do pages whose key content only appears after JavaScript execution get cited less often by AI search crawlers than server-rendered equivalents?
Review-platform freshness
Does a steady flow of recent, genuine reviews on third-party platforms shift sentiment and mention rate on "is [brand] good?" prompts?
Regional language pages
For Indian and other multilingual markets, do native-language pages earn citations on local-language prompts that English pages miss?
From hypothesis to playbook in six stages.
Design & pre-register
Hypothesis, single variable, metrics, sample size and duration written down and shared internally.
Select & match pages
Test and control pages matched on topic, authority, age and baseline citation rate.
Baseline window
Prompt clusters run repeatedly across engines until baseline rates are stable.
Intervention & monitoring
Change ships on test pages only. Server logs confirm recrawls; platform updates are logged.
Analysis
Difference-in-differences between test and control, with ranges — plus a plain-English verdict.
Playbook update
Confirmed effects enter client playbooks; null results retire tactics we stop recommending.
The metrics behind every experiment.
| Metric | Definition | Used in |
|---|---|---|
| Citation rate | % of runs where the page appears as a cited source | E1–E9 |
| Mention rate | % of runs where the brand is named in the answer | E2, E6, E10 |
| Answer share | How much of the generated answer draws on the page | E1, E2, E3 |
| Fact accuracy | % of answers that state tested facts correctly | E3, E6 |
| Time-to-crawl / time-to-citation | Days from publish or change to first crawl and first citation | E4, E8, E9 |
| Crawler hits by agent | Requests per AI user agent from server logs | E7, E8, E9 |
| Sentiment | Positive / neutral / negative framing of the brand | E10 |
These are the same metrics we report in client AI visibility tracking — so test findings translate directly into your dashboard.
Get the findings first — or become a test site.
Two ways to benefit from the programme.
Findings briefing
Ask to be added to our findings briefing: a short summary of each experiment's verdict and what to do about it. No fluff, no spam.
Test cohort
Volunteer pages on your site for an experiment. You get a free baseline, early results and a private debrief. Your brand is never published.
Mention "experiments" in the audit form message and we'll route you to the right team.
An agency that tests before it sells.
Since 2008 we've watched search tactics come and go. The ones that last are the ones backed by evidence. We apply the same discipline to AI search.
- Evidence-led playbooks. What we recommend is tied to published research or our own controlled tests — and we say which.
- Multi-engine coverage. Tests span Perplexity, Gemini, ChatGPT, Copilot and Google AI Overviews.
- Full-stack execution. Around 300 specialists across SEO, content, digital PR and development.
- Honesty by policy. No guarantees of AI "rankings", no manipulative tactics, ever.
What clients get from the programme
Put evidence behind your AI search strategy.
Start with a free AI visibility audit. We'll show you where you stand across five engines and which evidence-backed fixes to make first.
AI search experiment questions, answered.
What is an AI search experiment?
It's a controlled test that changes one thing on a set of pages — such as the introduction, schema or crawler access — and compares how often those pages are cited or mentioned by AI answer engines against matched control pages, using repeated runs of a fixed prompt set.
Why don't you publish results on this page?
We only publish findings once an experiment has completed its pre-registered window and been reviewed. Until then, the designs are shown here so the methodology is transparent. Findings are shared first with clients and briefing subscribers.
Does adding statistics and quotes really help AI visibility?
The peer-reviewed GEO paper by Aggarwal et al. (KDD 2024) found that adding quotations and statistics were among the most effective methods in its benchmark, with visibility gains of up to 40%. Our experiment E2 tests whether that holds on live engines and commercial topics.
Does llms.txt help with ChatGPT or Google?
llms.txt is a proposed convention, not an adopted standard, and Google states that no AI text files are needed to appear in AI Overviews or AI Mode. Our experiment E7 checks whether any agents request the file and whether it has any measurable effect.
Should I block AI crawlers in robots.txt?
It depends on your goals. Search-focused crawlers such as OAI-SearchBot and PerplexityBot are used to surface and cite pages; training-focused agents such as GPTBot and Google-Extended relate to model training. Blocking the former can remove you from AI search citations. We review this in every audit.
How do you deal with AI answers changing every time?
Through repeated sampling and control groups. We run each prompt multiple times per engine per window from clean sessions, and compare the change in test pages against the change in control pages over the same period.
Can my company take part in an experiment?
Yes. We recruit a small number of sites per test, matched by industry and authority. You receive a free baseline, early findings and a private debrief, and your brand is never named publicly. Use the audit form and mention "experiments".
Will experiment findings apply to my website?
Findings show what tends to work under controlled conditions; your results depend on your competition, authority and content. That's why we pair research with a site-specific audit and ongoing measurement.
Keep exploring generative engine optimization
Every AI platform rewards slightly different signals. Dive into the playbook for each one — or let our GEO team run all of them for you.
Find out what ChatGPT, Gemini & Google AI say about you today.
Tell us your website and top competitors. Within 3 business days a GEO strategist sends you a personal AI visibility report — no generic PDF, no sales script.
- Share-of-voice snapshot across ChatGPT, Gemini, Perplexity, Copilot and Google AI Overviews
- Prompt-level gaps where competitors are named and you aren't
- Citation sources each AI engine leans on in your category
- A prioritised 90-day GEO plan you can run in-house or with us