noindex Detected in robots meta False Positive GSC Fix

The Problem

Search Console fired a coverage alert for 412 pages on a client's site last week. The status was "Excluded by 'noindex' tag", the inspector showed noindex detected in robots meta, and the live URL inspection confirmed the rendered HTML contained:

<meta name="robots" content="noindex" />

The issue was that none of those pages should have had a noindex tag. The CMS marked them as published, the App Router metadata returned index: true, and curl on the same URL from my machine returned <meta name="robots" content="index, follow">. Somehow Googlebot was seeing a different version of the page from the one I could fetch.

I ran into this on a Next.js 16.2 site that had migrated to the new cookies() async API two weeks earlier. The pattern is now common enough that I have hit it on three projects this quarter. The metadata function was reading a cookie to decide whether to show a draft preview, and when the cookie was absent (every Googlebot fetch) it fell into a branch that returned noindex for "unauthenticated draft preview" mode that should never have fired on a published page.

Why It Happens

Three things stacked on top of each other to produce the false positive.

First, the metadata function was written defensively. It read a preview-mode cookie, and if absent treated the request as anonymous, which the previous developer had wired to return noindex so unpublished previews would not leak into search. That logic was correct for the /draft/[slug] route it was originally written for. Someone later refactored the shared generateMetadata helper into the published /blog/[slug] route as well, and the noindex branch came with it.

Second, the bug only showed up on production. Cookies in dev mode came through with a default preview mode cookie from a stale localhost session, so the helper returned index: true on my machine. Googlebot has no cookies. Every Googlebot fetch hit the anonymous branch, every page got noindex, and the live URL test in Search Console reproduced it perfectly because that test runs an anonymous fetch as well.

Third, the robots meta directive is read once per fetch, not blended with sitemap signals. If the rendered HTML says noindex, Google honours it regardless of what your sitemap, internal links, or canonicals claim. There is no override. The only signal that matters is what comes back in the HTML.

The Next.js metadata API made this easier to break because generateMetadata runs in the same request context as the page, has access to cookies and headers, and silently returns whatever value the code path produces. A typo or stale conditional in that function changes what Googlebot sees without changing what your team sees in dev.

The Fix

Three steps. The first stops the bleed. The other two stop it from happening again.

Step 1: Make the metadata function default to indexable. Audit every generateMetadata in the App Router and make noindex an explicit opt-in, not an implicit fallback. The corrected helper for the blog route:

// app/blog/[slug]/page.tsx
import type { Metadata } from 'next'
import { cookies } from 'next/headers'
import { getPostBySlug } from '@/lib/cms'

export async function generateMetadata(
  { params }: { params: Promise<{ slug: string }> }
): Promise<Metadata> {
  const { slug } = await params
  const post = await getPostBySlug(slug)

  if (!post) {
    return { title: 'Not found', robots: { index: false, follow: false } }
  }

  const cookieStore = await cookies()
  const isPreview = cookieStore.get('preview-mode')?.value === 'on'
  const isDraft = post.status === 'draft'

  // Only block indexing when the post is genuinely a draft preview.
  const shouldBlockIndexing = isDraft && isPreview

  return {
    title: post.seoTitle ?? post.title,
    description: post.seoDescription,
    robots: {
      index: !shouldBlockIndexing,
      follow: true,
    },
    alternates: {
      canonical: `https://example.com/blog/${slug}`,
    },
  }
}

The default branch is now index: true. The only way to get a noindex is to be both a draft and in preview mode, which is the right policy for a published blog. Deploy this and the rendered HTML on the next Googlebot fetch will be <meta name="robots" content="index, follow">.

Step 2: Verify with the exact fetch Googlebot uses. Search Console's URL Inspection tool runs an anonymous fetch with the Googlebot user agent. Reproduce it from the command line to confirm the fix before requesting re-indexing:

curl -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
  -s https://example.com/blog/your-slug | grep -i 'name="robots"'

The output should be <meta name="robots" content="index, follow"/>. If it still says noindex, the metadata function has another branch the change missed. Search the route's imports for any helper that returns robots: { index: false } and trace where it triggers.

Step 3: Request reindexing in bulk. Once curl confirms the fix on a sample of affected URLs, submit the canonical sitemap in Search Console under Sitemaps, then use the URL Inspection tool on the highest-traffic 10 to 20 URLs to request indexing manually. The remaining pages will be recrawled on Google's own schedule, usually within seven to fourteen days. Watch the "Excluded by 'noindex' tag" count in the Pages report drop over the next two weeks.

Add a regression guard. Drop a tiny check into your end-to-end test suite that fetches a sample of indexable routes with no cookies and asserts the meta tag is correct. Playwright works for this:

import { test, expect } from '@playwright/test'

const indexableRoutes = ['/', '/blog/example-published-post', '/services']

for (const route of indexableRoutes) {
  test(`${route} is indexable`, async ({ page }) => {
    await page.goto(`https://example.com${route}`)
    const robots = await page.locator('meta[name="robots"]').getAttribute('content')
    expect(robots ?? '').not.toContain('noindex')
  })
}

Run that in CI and the next refactor that quietly flips a metadata branch fails the build instead of taking your traffic down for two weeks.

The Lesson

If Search Console says noindex is in the rendered HTML, the rendered HTML really does contain noindex. Run an anonymous fetch with the Googlebot user agent and check. App Router metadata functions are easy to write in a way that flips to noindex when cookies are missing, which is exactly what Googlebot sees on every fetch. Default to indexable, opt out explicitly, and add a CI check.

If your traffic dropped after a Next.js metadata refactor and the cause is buried in a shared helper, that is the kind of audit I run on App Router sites every week. See my services. For a related Search Console issue, read Crawled, currently not indexed in GSC fix.

Lost rankings after a metadata refactor? Get it audited.