Robots.txt Checker
The robots.txt file tells crawlers which parts of a site they may fetch. The tool loads it, evaluates the rules and shows whether the address you entered is allowed.
Checked by our server, free and without an account
Loading the tool …
What the tool looks at
- Reachability of robots.txt and the rules it contains.
- Whether the entered address is allowed for crawlers, and the sitemaps listed.
How it works
- Enter the address. Only public web addresses on the standard ports are accepted.
- Our server fetches what the check needs once, with its own honest bot name, and evaluates it.
- You get the evaluated values with a short explanation. Nothing from the page is stored.
More On-page SEO tools
All tools- Title Tag CheckerCheck the title tag of any page – is it there, is it unique on the page and is its length right for search results?
- Meta Description CheckerCheck the meta description of a page – present, unique and of a length that fits search results.
- Meta Tags CheckerSee every meta tag of a web page in one list – name, property and content, evaluated and ready to compare.
- Heading Structure CheckerCheck the H1 and the heading structure of a page – one clear main heading and a logical order of H2 to H6.
- Canonical Tag CheckerCheck the canonical tag of a page – is it set, is it a valid absolute URL and does it point where you expect?
- Noindex CheckerFind out whether a page may appear in search results – meta robots, X-Robots-Tag header and robots.txt in one check.
- XML Sitemap CheckerCheck an XML sitemap – reachable, well-formed, how many URLs it lists and whether it is a sitemap index.
- Schema Markup CheckerFind the structured data of a page – JSON-LD, microdata and the schema.org types it declares.
- Open Graph CheckerCheck the Open Graph tags of a page – title, description, image and URL used when the page is shared.
- Image Alt Text CheckerFind images without alt text on a page – important for accessibility and for image search.
- SEO-Friendly URL CheckerCheck whether a URL is easy to read and share – length, characters, parameters and word separators.
- 404 Page CheckerCheck whether a site answers missing pages with a real 404 status instead of a soft 404 or a redirect.
- Language Attribute CheckerCheck the lang attribute of a page – set on the html element and a valid language code?
- Favicon CheckerCheck whether a website has a favicon – declared in the HTML or at the default location, and reachable.
- Social Links CheckerFind the links to social network profiles on a page – which networks are linked and where they point.
The whole page in one audit
One audit runs more than 50 tests, rates them by severity and turns them into a score. Websites are re-checked on a schedule.