4 ms·

I see ChatGPT's bots pull down all of my Python wheels every couple of weeks. Wheels that haven't changed in years, with a "Last-Modified" and "ETag" that have

• fuzztester 2y ago
  Or, could it be, just possibly be (gasp), that some of the devs at these "hotshot" AI companies are just ignorant or lazy or pressured enough, so as to not do such normal checks? Wouldn't be surprised if so.
  • dalke 2y ago
    You think they do cache the data but don't use it? For what it's worth, mj12bot.com is even worse. They pull down every wheel every two or three days, even though something like chemfp-3.4-cp35-cp35m-manylinux1_x86_64.whl hasn't changed in years - it's for Python 3.5, after all.
    • fuzztester 2y ago
      >You think they do cache the data but don't use it? that's not what I meant. and it is not they, it is it. i.e. the web server, not bots or devs on the other end of the connection, is what tells you the needed info. all you have to do is check it and act accordingly, i.e. download the changed resource or don't download the unchanged one. google: http header last modified and look for the etag link too.
      • fuzztester 2y ago
        here you go: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Last-Modified https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/... https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/ETag https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/...