Check explainer
robots.txt
A misconfigured robots.txt makes the site invisible to Google.
- to review
- ok
What we check
Checks robots.txt reachability and whether it blocks everything.
Why it matters
A misconfigured robots.txt makes the site invisible to Google.
Common causes
The most common triggers for this finding. Expand this section with real-world examples.
How to fix it
robots.txt at the web root: "User-agent: *" + "Allow: /" + "Sitemap: <URL>".
-
What does a failing robots.txt finding mean?
A misconfigured robots.txt makes the site invisible to Google. robots.txt at the web root: "User-agent: *" + "Allow: /" + "Sitemap: <URL>".
-
How do you fix robots.txt?
robots.txt at the web root: "User-agent: *" + "Allow: /" + "Sitemap: <URL>".