About SiteRewriteBot
SiteRewriteBot fetches public web pages only when someone submits a website to SiteRewrite and confirms they are authorized to modernize it.
- User agent:
Mozilla/5.0 (compatible; SiteRewriteBot/1.0; +https://siterewrite.com/bot) - It respects
robots.txtrules forSiteRewriteBotand*. - It reads a bounded number of pages (at most 30) per request, never logs in, and never bypasses paywalls, CAPTCHAs or access controls.
- It does not scan for vulnerabilities.
To block it, add to your robots.txt:
User-agent: SiteRewriteBot
Disallow: /
Concerned about a preview made from your website? Report it and we'll review it promptly.