Hide and Seek

Pages can be indexed and quoted

directives · Access · weight 25 of 110

What we check

A page can carry instructions for search engines, either in a <meta name="robots"> tag in its HTML or in an X-Robots-Tag header from the server. We read both on your home page and the sample of pages we check.

  • noindex (or none, which includes it) scores 0: the page is kept out of the index.
  • nosnippet or max-snippet:0 scores 0.2: the page is indexed, but nothing on it may be quoted.
  • noarchive on its own scores 0.8.
  • noai and noimageai are reported for your information and cost nothing.

The score is the worst page’s. An instruction aimed at one bot in the header, like googlebot: noindex, still counts.

Why it matters

An assistant answers by finding a page and quoting it. A noindex page can’t be found. A nosnippet page can be found but not quoted, and an assistant that can’t quote you can’t cite you either.

These tags are often left behind by accident: a staging setting that shipped, an SEO plugin’s default, or a template that marks every page the same way.

How to pass

  1. View the source of your home page and look for a robots meta tag in the head:
    <meta name="robots" content="noindex, nofollow">
  2. Remove it, or change it to allow indexing. If you want to limit how much is quoted, use a positive number instead of zero:
    <meta name="robots" content="max-snippet:160">
  3. Ask your developer to check the server’s headers too, since an X-Robots-Tag doesn’t show in the page source:
    curl -sI https://example.com/ | grep -i x-robots-tag
  4. In WordPress and similar platforms, look for a “discourage search engines” setting and turn it off.

Back to the rules