3 ms·
But they are in this case. They would not stop any scraped data from popping up for sale in shady places. That can be done by LinkedIn or whoever themselves usi
by FRex 9y ago
But they are in this case. They would not stop any scraped data from popping up for sale in shady places. That can be done by LinkedIn or whoever themselves using some smart way to detect bots and stop them from scraping their website.
The only people a robots.txt law would affect are private users who set up a Python script to scrape a single page for themselves to check for something, things like archive.org, researchers, automated website testers, etc. while anyone nefarious can just rent a shady VPN or use a server in Russia, China, Middle East, etc.
Requiring a human barely increases the cost if that data is so valuable in the first place and would be last resort anyway, far after just running the bots from a shady country, for captcha it's done because it's technically easier/cheaper (although supposedly automated solvers exist too).
But laws that punish outright gross negligence would help protect everyone who uses these American websites (and most of the world does) from data leaks of data that is arguably way more sensitive (emails, unhashed passwords, SS and CC numbers, real names even like in Ashley Madison case, etc.).
LinkedIn used sha1 with no salt as recently as 2012 (when they were hacked) for passwords and over 100 million such username + password combinations got stolen. Not only is sha1 not good enough for passwords but for many common and simple words (yes, yes, they are bad passwords, but people do use them) just googling can "crack" them due to lack of salt. The law should either go both ways or neither.
To suggest such heavy handed laws like considering robots.txt ignorance hacking while multi million corporations with millions of users get away with stuff like that (and I mean true negligence of most basic practices, not some obscure bug in the underlying software or something else that isn't absolutely obvious) over and over and over again that every random free my-first-login-page and my-first-SQL-injection-prevention tutorials advise against is absolutely ridiculous and anti-consumer.