Paste your robots.txt to validate the syntax, see each crawler's rules, and catch mistakes that quietly block your pages — free, no signup.
Copied!
robots.txt is a plain-text file that lives at the root of your site —
https://yoursite.com/robots.txt — and tells search-engine crawlers which parts of
the site they may or may not request. It uses a simple set of directives: User-agent
to name a crawler, Disallow and Allow to control paths, and
Sitemap to point to your XML sitemap. A single typo here can accidentally hide your
whole site from Google, so it's worth checking.
User-agent so you can see what each crawler is told.Disallow: / — which blocks the entire site.User-agent line (they're ignored).Sitemap directives and warns if none is present.Copy the contents of your live robots.txt (or a draft) into the box.
The tool parses it instantly and groups the rules by crawler, with a list of issues to review.
Correct anything flagged, paste again to confirm, then verify in Google Search Console's robots.txt report.
Blocking what you want indexed. Disallow: / under User-agent: *
hides everything — a frequent accident after a site migration. Using Disallow to deindex.
Disallow stops crawling but a blocked URL can still appear in results without a snippet; use a
noindex tag for true removal. Forgetting the sitemap. Adding
Sitemap: helps crawlers discover your URLs faster.