SEO Tools

Robots Meta Tag Generator

Build per-page robots meta tags and X-Robots-Tag headers with noindex, nofollow, max-snippet, and per-bot directives. Live preview, nginx and Apache snippets.

Quick presets

Apply a common configuration to the active bot. The AI training opt-out preset adds per-bot tags for GPTBot and Google-Extended.

Crawlers

Use robots for a tag that applies to every crawler. Add a specific bot to override the default for that crawler only.

Add a known crawler

Custom user-agent

Active crawler name

The directives below apply only to this user-agent. Use lowercase letters, numbers, hyphens, underscores, and dots.

HTML meta tags

Place the tag (or tags) inside the <head> element of each page they should apply to.

X-Robots-Tag HTTP header

Use the HTTP header form to apply the same directives to non-HTML files such as PDFs, images, and JSON responses. Per-bot variants place the user-agent before the directives.

X-Robots-Tag: all

nginx

# Add to a server or location block.
# add_header may be repeated for multiple X-Robots-Tag values.
add_header X-Robots-Tag "all" always;

Apache / .htaccess

# Place in a <Files> block, .htaccess, or virtual host configuration.
# Header append allows multiple X-Robots-Tag values to coexist.
Header append X-Robots-Tag "all"

Express / Node.js

// Express / Node.js middleware
app.use((req, res, next) => {
  res.append("X-Robots-Tag", "all");
  next();
});

When to use which

HTML meta tag: the page is a normal HTML document. Place it in the <head> before any tag that loads external content.
X-Robots-Tag header: the resource is not HTML (PDF, image, JSON, plain text) or you want to keep the directive in the server configuration rather than in the document.
robots.txt: prevents the crawler from fetching a URL in the first place. If the crawler cannot fetch the URL, it never sees the meta tag or header, so do not combine noindex with a Disallow entry that blocks the same URL.

Directive support

Google supports every directive in this tool. The full list is documented under the robots meta tag and X-Robots-Tag sections of Google Search Central.
Bing supports noindex, nofollow, none, noarchive, nosnippet, and noimageindex. Bing-specific values use the bingbot user-agent.
GPTBot and Google-Extended respect noindex and nofollow as opt-outs from being used as training data. Apply them per-bot using the tabs above.
Less common bots ignore directives they do not implement, so extra tokens never break support.

How to use

Pick a preset (noindex follow, noindex nofollow, allow rich snippets, hide snippets, or AI training opt-out) to set common configurations in one click.
Add specific crawlers as tabs (googlebot, bingbot, gptbot, google-extended, or any custom user-agent) to ship different directives per bot.
Toggle directives in the four groups: indexing and link following, snippets and cache, media previews, and time and other. Numeric, enum, and date directives expose their value field when enabled.
Watch the live HTML meta tag, X-Robots-Tag header, and per-bot warnings update as you change directives.
Copy the HTML tag for the <head>, the X-Robots-Tag header, or paste the nginx, Apache .htaccess, or Express middleware snippet into your server configuration.

About this tool

Robots Meta Tag Generator builds the per-page robots meta tag and the equivalent X-Robots-Tag HTTP header that Google, Bing, and other major search engines read to decide how to crawl, index, and render a URL. Toggle the directives you need (noindex, nofollow, none, noarchive, nosnippet, noimageindex, notranslate, nositelinkssearchbox, indexifembedded), set the value-bearing directives Google added in recent years (max-snippet, max-image-preview, max-video-preview, unavailable_after), and the tool emits the canonical comma-separated content string with each token in the correct order. Per-bot variants are supported: add tabs for googlebot, googlebot-news, bingbot, gptbot, google-extended, or any custom user-agent name to ship targeted directives for one crawler at a time, including AI-training opt-outs for GPTBot and Google-Extended. A quick AI training opt-out preset wires up the right per-bot tags in one click, and other presets cover the common cases (noindex follow, noindex nofollow, rich snippet allow, hide snippets and cache). The output includes the HTML meta tag for the <head>, the X-Robots-Tag HTTP header for PDFs, images, and JSON, plus ready-to-paste nginx add_header, Apache Header append (.htaccess), and Express middleware snippets. The page also flags common mistakes: combining nosnippet with max-snippet, using indexifembedded without noindex, and the redundant overlap between none and noindex,nofollow. Distinct from robots.txt: robots.txt controls whether a crawler can fetch a URL at all, while the robots meta tag and X-Robots-Tag tell crawlers how to treat the URL once they have fetched it; pair them carefully because a Disallow entry hides the meta tag from the crawler. Everything runs locally in your browser; no URL or directive ever leaves your device.

Free to use. Works in your browser. No signup, no login.

Related tools

Robots Meta Tag Generator

How to use

About this tool

You may also like

Robots.txt Generator

Robots.txt Tester

Meta Tag Preview

Canonical Tag Generator

Hreflang Tag Generator

Schema Markup Generator

Robots Meta Tag Generator