About SiteRewriteBot

SiteRewriteBot fetches public web pages only when someone submits a website to SiteRewrite and confirms they are authorized to modernize it.

  • User agent: Mozilla/5.0 (compatible; SiteRewriteBot/1.0; +https://siterewrite.com/bot)
  • It respects robots.txt rules for SiteRewriteBot and *.
  • It reads a bounded number of pages (at most 30) per request, never logs in, and never bypasses paywalls, CAPTCHAs or access controls.
  • It does not scan for vulnerabilities.

To block it, add to your robots.txt:

User-agent: SiteRewriteBot
Disallow: /

Concerned about a preview made from your website? Report it and we'll review it promptly.