4 ms·
It's "robots.txt", not "robot.txt". I'm not just nitpicking -- it's a clear signal the journalist has no idea what they're talking about. That and the fact tha
by jonas21 2y ago
It's "robots.txt", not "robot.txt". I'm not just nitpicking -- it's a clear signal the journalist has no idea what they're talking about.
That and the fact that they're using a log file with the timestamps omitted as evidence of "how ruthelessly an OpenAI bot was accessing the site" makes the claims in the article a bit suspect.
OpenAI isn't necessarily in the clear here, but this is a low-quality article that doesn't provide much signal either way.
- peterldowns 2y agoWell said, I agree with you.
- ted_bunny 2y agoI wish they'd add a little arrow you could click.
- peterldowns 2y agoWell said, I agree with you.
- Thoreandan 2y agoHear hear. Poor article going out the door for publication with zero editorial checking.
- joelkoen 2y agoHaha yeah just noticed they call Bytespider "TokTok's crawler" too
- ted_bunny 2y agoThe best way to tell a journalist doesn't know their subject matter: check if they're a journalist.