Robots.txt Settings
User-Agent
All Bots (*)
Googlebot
Bingbot
Yandex
Custom Bot
Custom Bot Name
Default Access
Allow All
Disallow All
Disallow Paths
Allow Paths
Crawl Delay (Google/Bing)
Request Rate (Bing)
Visit Time (Bing)
Host (Yandex)
Clean-param (Yandex)
Sitemap URL
Generated robots.txt
Copy
Download