The AI Crawler Controls You Actually Have (And the Trap Hidden in One of Them)

Google has started rolling a site-level opt-out from AI Overviews, AI Mode and Discover AI features beyond its initial UK launch, and Search Console now surfaces some reporting on AI-driven visibility. Search Engine Journal's write-up of the change is worth reading in full if you manage a site.

Most coverage framed this as "you can now opt out of AI search". That framing has already caused damage, because the controls are not one switch and one of them does something quite different from what its name suggests.

Here is what each one actually does.

The four controls, and what each costs you

Google-Extended governs whether your content is used for Gemini model training. It does not affect whether you appear in Google Search, and it does not remove you from AI Overviews. It is the closest thing to a clean "don't train on me" signal, and for most service businesses it is the least consequential of the four.

The Search Console site-level opt-out removes you from AI Overviews, AI Mode and Discover AI features. This is the new control, and it is the one people mean when they say they have opted out.

noindex removes you from search entirely. Nobody reaches for this by accident, but it is worth stating for completeness: it is not an AI control.

nosnippet is the trap.

The nosnippet trap

nosnippet will keep your content out of AI Overviews. It is a legitimate way to do it, and it has been around far longer than any of this.

It also removes your traditional search snippet. The description under your link in ordinary Google results. Gone.

There is no partial setting. It is all or nothing: either Google may quote your page, in both AI Overviews and normal results, or it may quote it in neither.

For a service business this is close to the worst possible trade. You would be giving up the two lines of text that persuade someone to click your ordinary organic listing, in exchange for not appearing in a summary you probably wanted to appear in anyway. If you take one thing from this article, take that.

The reporting gap nobody mentions

Search Console's generative AI performance reporting shows impressions only. No clicks. No click-through rate.

So you can see that you appeared, and you cannot see whether it did anything. There is no way to answer the obvious follow-up question: is being cited in AI Overviews actually sending me work, or just showing my name to people who never arrive?

This matters for how you interpret anything you read on this subject, including confident claims about AI search "traffic". At the moment, for Google's AI surfaces, the click data to support those claims is not being published. Treat precise assertions about AI-driven traffic with the scepticism they have earned.

Should a service business opt out at all?

Almost certainly not, and the reasoning is the same one I keep coming back to: you do not sell page views.

A publisher whose revenue comes from ads against article traffic has a genuine argument for blocking. Every question the assistant answers is a visit that did not happen and a page view that did not monetise. Their content is the product, and giving it away for free to a system that reprints it is a real loss.

You sell work. An assistant summarising your service page and naming you as the firm to call has not stolen anything. It has done your sales prospecting for free.

Opting out means choosing not to appear in the answer. When the answer is increasingly where the shortlist gets built, that is a strange thing to choose.

Where the calculation is genuinely harder

There are cases where I would not be so quick, and it is worth being honest about them.

If you have published genuinely proprietary research — original survey data, a pricing benchmark, a methodology nobody else has — then having it summarised and attributed loosely, with your name appearing as one source among five, is a poor return on what it cost to produce. That is a real grievance and the arguments on both sides are reasonable.

If your business is content itself, the publisher logic applies to you directly.

For everyone else, the sums are not close.

What to do instead of opting out

Leave the crawlers alone and compete for the citation. The work that gets you named is the work that would have made you rank anyway: specific, complete, current information that is safe to quote.

Set Google-Extended deliberately. Decide whether you are comfortable with your content training Gemini. It is a philosophical call more than a commercial one, and it does not affect your visibility either way. Just make it a decision rather than a default.

Never use nosnippet on a commercial page. See above.

Track impressions in Search Console, but do not mistake them for demand. Pair them with branded search and enquiry volume, which are the numbers that actually move when recommendations turn into work.

What to do this week

1. Check whether anyone has already set nosnippet on your site. It is more common than you would expect, usually added years ago by someone worried about content scraping, and it is quietly costing you snippets right now.
2. Look at your robots.txt and confirm you know what every directive there is doing. AI crawler directives get added by plugins and hosts without anyone deciding.
3. Decide your Google-Extended position on purpose.
4. Leave the AI Overviews opt-out alone unless you sell content directly.

The bottom line

The new controls are real and useful, and most of the businesses reaching for them should not be. The publisher argument for blocking is sound, specific to publishers, and does not transfer.

The one thing worth acting on immediately is the nosnippet check, because that setting can already be live on your site, costing you ordinary search snippets, with nobody having decided it should be.

Want to know how you currently appear across search and AI answers? Request a visibility check.