noindex for AI Pages
Google Wants Throwaway Pages Out of Its Index

Key takeaways

• New Google guidance: On October 26, 2026, Google expanded its guide on preventing user-generated spam.
• Not everything belongs in search: According to Google, pages meant only for sharing, collaboration or as a utility should be kept out of search, with noindex (an instruction telling Google not to include the page in search results) or with robots.txt.
• AI as the trigger: The German trade publication SEO Südwest sees the trigger in AI tools that produce small planners, calculators and assistants in minutes, many of them never meant for the public.
• Aimed at platforms: Google addresses services where users build pages; applying it to company websites is my own conclusion, not a statement by Google.
• Check Search Console: Google advises regularly checking its free Search Console tool for sudden jumps in the number of indexed pages.

Contents

1. What Google changed
2. Why AI pages are the trigger
3. What this means for company websites
4. noindex or robots.txt: the difference
5. Search Console: how to check your website
6. What you can do now
7. Frequently asked questions

Not every page belongs in Google Search. On October 26, 2026, Google expanded its guidance against spam on platforms: pages meant only for sharing, collaboration or as a utility should not end up in the index, the directory Google draws its search results from. SEO Südwest sees the trigger in AI tools that create such pages in minutes. I call them throwaway pages: built for one purpose, not for search.

What Google changed

Google added a section to its guide “Prevent user-generated spam on your site and platform” in the Search Central documentation. It’s called “Only index content meant for Search”.

The key sentence reads, word for word: “When pages are created for direct sharing, collaboration, or utility rather than public search discovery, apply the noindex rule (using a robots meta tag or X-Robots-Tag HTTP response header) or block crawling in robots.txt by default.” (Google Search Central, as of Oct 26, 2026). In plain terms: if a page is there to be shared directly, to work on together or as a tool, and not to be found in search, it should get a noindex by default or be blocked from crawling in robots.txt.

In its changelog of October 26, 2026, Google gives the reason: platforms should keep pages that aren’t meant for Search out of the index and detect spam earlier. This refers to services where users create pages themselves; Google calls this user-generated spam.

Why AI pages are the trigger

Google’s guidance (as of Oct 26, 2026) doesn’t mention AI. The German trade publication SEO Südwest, however, put the change in context on October 27, 2026: with AI tools, pages such as appointment planners, language courses or small assistants are created in minutes. As an example, SEO Südwest names ChatGPT Sites, which generates such pages and apps directly.

According to SEO Südwest, many of these pages are made for personal use. If they end up in Google’s index anyway, they can lower the quality of search results, in that assessment. The expanded guidance fits this exactly: what is only shared by link should not show up in search.

What this means for company websites

To be honest: Google’s guidance is aimed at platforms where users create pages themselves. Google doesn’t say it also applies to a regular company website. That is my own conclusion.

I still think the conclusion makes sense. The basic idea applies to every website: a page nobody should find via Google doesn’t need to be in the index. On a company website, this can look like the following. A made-up example: a machine manufacturer has 80 pages that it published on purpose. Search Console, however, shows 240 indexed pages. Among them are old campaign pages, thank-you pages after the contact form, an agency’s test pages and half-finished AI drafts that someone put online “just for a moment”.

None of these pages help a buyer looking for the company. Still, Google fetches them as long as robots.txt doesn’t forbid it, as Google describes in its introduction to robots.txt; this fetching is called crawling. Anyone building pages, calculators or landing pages quickly with AI should therefore decide beforehand which of them belong in Google at all. I explain wasted crawls in the glossary under Crawl Waste and Index Budget.

noindex or robots.txt: the difference

Google names both options in the guidance. They work differently, though, as Google describes in its own documentation on noindex and robots.txt. A noindex is an instruction in the page’s source code: Google may read the page but doesn’t include it in search results. The robots.txt is a file in the root directory of a website that tells search engines which URLs not to fetch at all.

For a page that’s already in the index, noindex is usually the right choice. Google itself writes that robots.txt is “not a mechanism for keeping a web page out of Google”. The order matters. According to Google, a noindex only works if the page isn’t also blocked in robots.txt. If it is blocked, Google doesn’t fetch the page and never sees the noindex. The URL can then still appear in search, for example if other pages link to it. So use only one of the two per page, and noindex for pages that are already indexed. More on this in Technical SEO explained. I check for such mistakes as part of my SEO audit.

Search Console: how to check your website

Search Console is a free Google tool that shows how Google sees your website. In its expanded guidance of October 26, 2026, Google advises verifying your website there and regularly checking two reports: the Page indexing report (which pages are in the index) and the Performance report (for which searches your pages appear). Watch for spikes, meaning sudden jumps in the number of pages or queries. If your website isn’t verified there yet, add it at search.google.com/search-console and verify it; usually whoever maintains your website handles this. The menu names below are from the English interface and may change.

1. Open the report: In Search Console under Indexing, Pages, you see how many pages are indexed.
2. Compare the number: If it’s well above the number of pages you published on purpose, take a closer look.
3. Look at the URLs: Under “View data about indexed pages” you’ll find the list. Look for folders like /test/, /thank-you/ or old campaign names.
4. Check the trend: Was there a sudden jump even though you didn’t publish new content? Then it’s worth checking which pages were added.

What you can do now

1. Count your pages: Roughly estimate how many pages you published on purpose, for example based on the menu and subpages. Compare that number with Search Console.
2. Sort them out: In the list, mark every indexed page that no buyer needs to find via Google.
3. Pass the list on: Whoever maintains your website technically sets these pages to noindex. In WordPress, for example, common SEO plugins let you do this per page with a checkbox. Pages nobody needs anymore can also be deleted; if one of them has links from other sites, add a redirect to a suitable page.
4. Set a rule: From now on, new AI pages, calculators and test pages go online with noindex until someone deliberately decides they should be in search.

Next step

Google now says it itself: not every page belongs in search. If you want to know whether more of your site is indexed than you thought, book a free consultation. You’ll get an initial assessment of your website, with no contract commitment.

Frequently asked questions

Do I need to change anything on my company website now?

It’s not mandatory; Google’s guidance is aimed at platforms with user-generated pages. A look at Search Console still makes sense. If you find far more indexed pages than expected, a cleanup is worth it.

What is the difference between noindex and robots.txt?

With noindex, Google may read the page but won’t show it in search. robots.txt forbids fetching, but according to Google it doesn’t reliably keep the URL from appearing in search anyway.

Do many unnecessary pages hurt my rankings?

The expanded guidance of October 26, 2026, contains no fixed rule on this. My assessment: the more superfluous pages are in the index, the harder it is for Google and for you to see which pages really should be found.

Which pages should typically get a noindex?

My recommendation: thank-you pages after a form, test pages, expired campaigns, internal tools and pages that are only shared by link with individual people. The deciding question is whether a buyer should find the page via Google.

Where can I see how many pages of my website are indexed?

In Google Search Console under Indexing, Pages. There you see the number of indexed pages, the trend and the individual URLs.

Sources

1. Google Search Central: “Updated guidance on preventing user-generated spam”, changelog, Oct 26, 2026, developers.google.com, accessed Oct 29, 2026
2. Google Search Central: “Prevent user-generated spam on your site and platform”, as of Oct 26, 2026, developers.google.com, accessed Oct 29, 2026
3. Google Search Central: “Introduction to robots.txt”, developers.google.com, accessed Oct 29, 2026
4. Google Search Central: “Block Search indexing with noindex”, developers.google.com, accessed Oct 29, 2026
5. Christian Kunz: “KI-Content: Google verschärft Spam-Richtlinien für nutzergenerierte Inhalte” (German), SEO Südwest, Oct 27, 2026, seo-suedwest.de, accessed Oct 29, 2026