WPHunterBot
A small, polite crawler that looks at the public pages of local-business websites to see whether they run WordPress. Operated by the WP Hunter project.
What it does
WPHunterBot visits a business's website once, reads a handful of public pages, and records which technology the site is built with (for example WordPress, and which page builder). We use that to understand how small-business websites are built and to identify sites that might benefit from a modernised, faster replacement. It does not index your content for search, and it does not copy your site.
How it identifies itself
User-Agent: WPHunterBot/1.0 (+https://wphunter.link/; bot@wphunter.link)
How little it asks of your server
- At most 11 requests per site: the homepage,
robots.txt, the sitemap (and up to three child sitemaps), the public WordPress REST index and two count endpoints, and up to two pages sampled from the sitemap. - At least three seconds between requests to the same host.
- Never more than once a month per site.
- It follows the same rules for every page it is redirected to.
What it never does
- It never requests
/wp-login.php,/wp-admin,/xmlrpc.phpor/readme.html, and never lists users. Requests to those paths are blocked in its code, not just by policy. - It never logs in, submits forms, posts comments, or executes anything on your server.
- It never crawls password-protected or private pages.
robots.txt
WPHunterBot reads and obeys your robots.txt before fetching anything else. To keep it off your site entirely, add:
User-agent: WPHunterBot Disallow: /
If robots.txt cannot be fetched (a server error), the bot treats your whole site as off-limits.
Opting out or asking questions
Email bot@wphunter.link with your domain and we will exclude it. If something about the bot's behaviour looks wrong to you, please tell us: we would rather hear about it than have you block us.