Free SEO Tool
Robots.txt Checker
Analyze robots.txt and check crawling rules of any domain.
Robots.txt Checker
Analyze robots.txt of any domain
What is robots.txt?
robots.txt tells search engine crawlers which pages to crawl and which to skip. Misconfiguration can prevent your entire site from being indexed.
