top of page

Robots.txt

Robots.txt is a text file with instructions for search engine crawlers. It defines which areas of a website crawlers are allowed to...

bottom of page