2 ms·
Scraping the MTA site against the TOS, and intentionally obfuscating requests to limit the chance of the MTA identifying and shutting it down? Yeah, no surprise
by falcolas 9y ago
Scraping the MTA site against the TOS, and intentionally obfuscating requests to limit the chance of the MTA identifying and shutting it down? Yeah, no surprises here.
EDIT: Oops, missed that he didn't end up going live with the scraping.
It would still be great for the MTA to publish this data intentionally; but I'm sure the potential for trains to switch tracks and the associated backlash when they do is what prevents them from doing it.
Perhaps they could adopt the airline model, where there's a best effort to reach a particular pre-announced track, with notifications ringing out when they can't.
- teddythetwig 9y agoPretty sure that if you read a little further, you will find that the author decided to NOT go with the TOS violating process, and instead used an official MTA API.
- drinchev 9y agoThey could publish the log of passed trains, though. This will hurt nobody.
- boobsbr 9y ago> Scraping the MTA site against the TOS Where does it say it was against the TOS?
- Sir_Cmpwn 9y agoThe author didn't do that, though. As far as MTA is concerned the app was always using an API he had been given a legitimate key for. As an aside, fuck websites that prohibit scraping. If you send me some bytes I'm gonna do whatever I damn well please with them.
- EGreg 9y agoIn that case screw your API, if you allow DDOS I may as well do it for the lulz. Isn't it the same approach? If you can abuse a site you will, because they opened themselves up to the public?
- mustacheemperor 9y agoThat's an illogical extreme. I don't think anyone thinks that, rather, the question is whether this app was an abuse of MTA's system. They claim wide distribution of the app is a hazard. Really, it's not like MTA is stopping anyone with the inclination to do this themselves. If it's as reliable as the author says, you could do it without any automation if you were patient enough. Downvoting because that kind of exaggerated strawman reduces the quality of discussion with no benefit.
- EGreg 9y agoThanks for explaining the downvote. But this is not exaggerated nor a strawman. It mirrors the extreme "screw your website" reaction and investigates whether the logic for such an extreme disregard holds. So a website allows you to "download some bytes". Yes you can do anything you want with THOSE bytes. Does this mean you should make an app to let people scrape the website in an automated way and possibly overload it? If airlines, restaurants give something free or discounted, does that mean you can make an app to systematically let millions of people take advantage of the arbitrage? XKCD even has a comic for exactly this situation: https://xkcd.com/1499/ https://xkcd.com/1499/ I wasn't the one who said "FUCK WEBSITES". I used a nicer word and questioned the logic. Yet I get downvoted while the parent comment with expletives is upvoted.