If they are well-coded, they don't constantly crawl. They use and pay attention to headers like ETag, If-Modified-Since and/or If-None-Match and support conditi

If they are well-coded, they don't constantly crawl. They use and pay attention to headers like ETag, If-Modified-Since and/or If-None-Match and support conditional requests. Badly behaving RSS readers on the other hand.... https://rachelbythebay.com/w/2024/05/27/feed/ https://rachelbythebay.com/w/2024/05/27/feed/