2 ms·
Earlier this month I was downloading top 1 million websites' robots.txt and some guy just served 100mb of whitespace :/
by KomoD 3y ago
Earlier this month I was downloading top 1 million websites' robots.txt and some guy just served 100mb of whitespace :/
- pnpnp 3y agoI love it!
- hayley-patton 3y agoOnce I was trying to scrape from some internal API, and made a request that used parameters the real client wouldn't make. I didn't expect curl to scream "FUCK YOU <some name>" at me. Presumably the name belonged to someone who scraped it before.
- KomoD 3y agoLOL, that is great.