REFFEDBOT · PUBLIC DISCLOSURE

ReffedBot reads one page you asked about.

ReffedBot is the user agent Reffed uses to check a web page for the AI-search readiness audit and for Watch monitoring. A visit starts when someone enters a URL or a site owner turns monitoring on. ReffedBot does not discover sites and does not crawl.

// what it requests

Few requests. Plain GET and HEAD.

  • The page that was entered, plus /robots.txt, /llms.txt and the sitemap.
  • Monitoring checks also request up to 20 links from that page on the same site, and the page's canonical URL, to check whether they exist.
  • Requests are made one at a time to a host and read only; ReffedBot never submits forms, logs in, or changes anything on your site.
  • Monitoring visits obey robots.txt: a site that disallows ReffedBot is not read, and Reffed reports that it was not read.
// how it identifies itself

The user agent, and a cryptographic signature.

Mozilla/5.0 (compatible; ReffedBot/1.0; +https://reffed.ai/bot)

When signing is enabled, requests also carry HTTP Message Signatures (Web Bot Auth) with Signature-Agent: "https://reffed.ai". The public key is published at https://reffed.ai/.well-known/http-message-signatures-directory. Each signature names the host it is for and expires within a minute.

// what it does not do

It does not get around your protection.

If your site's bot protection challenges a visit, Reffed reports that the page could not be read and scores nothing. Reffed does not try to defeat a challenge you turned on, and does not ask you to turn protection off.

// contact

Questions about a visit?

Write to support@reffed.ai with the domain and the time of the visit. Reffed will confirm what was requested and, if you ask, stop visiting that domain.