3 ms·https://sr.ht/robots.txt https://sr.ht/robots.txt is a robots.txtby slipknotfan 4mo agohttps://sr.ht/robots.txt https://sr.ht/robots.txt is a robots.txtzionsati 4mo agoDo LLM crawlers adide by this?gus_massa 4mo agoIgnoring that AI bot will ignore it, ... Doesn't "User-agent: AwarioRssBot" needs it's own "Disallow: /" line?