6 ms·
I used to not bother with APIs and instead use selenium to drive the websites. It neatly avoided all of the politics around API access.
by chillfox 2y ago
I used to not bother with APIs and instead use selenium to drive the websites. It neatly avoided all of the politics around API access.
- bitwize 2y agoThat may be a CFAA violation, a felony in the United States. See Ryanair DAC v. Booking Holdings, Inc.
- chillfox 2y agoGood thing I am not an American, don't live in the US and have no intentions of visiting then. (I gave you an upvote, I don't think your comment deserve to be nuked for bringing up a possible legal risk)
- traviswingo 2y agoThis literally made me lol. Not sure if it’s true. I might be true. But come on!
- leros 2y agoThat's because Booking was also committing some type of misrepresentation and taking revenue away from Ryanair through their browser automation. Even then, the infraction was sooo bad that they got a $5k fine. I think the hiQ Labs vs LinkedIn case is a better representation that scraping is generally allowed: https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn
- CamperBob2 2y agoWhen it comes to CFAA violations, corporations get a $5K fine, while individuals get hounded to suicide.
- jascination 2y agoStart a company then do my nefarious work under that, got it!
- bdangubic 2y agoYou should do ABSOLUTELY EVERYTHING in your life under a company. And I mean EVERYTHING
- wombatpm 2y agoMy wife hates it when 1099 her for services rendered, especially since she refuses to bill against the PO I set up for her. The kids at least accepted NET15 payment terms. Although the oldest said if I short pay him for lawn care again, he’s going to take away my early pay discount.
- smallnamespace 2y agoHave you considered short paying him again through the same entity? After all, that's one use case of limited liability ;)
- lelandfe 2y agoKafka is smiling appreciatively at your approach to marriage
- exe34 2y agocorporations are more people than people.
- jamil7 2y agoUnironically, yes.
- oblio 2y agoHow does that work, though? Setting up a company has an initial cost and then recurrent costs (accountant, etc). Are the benefits that high for the average Joe?
- satvikpendem 2y ago> In November 2022 the U.S. District Court for the Northern District of California ruled that hiQ had breached LinkedIn's User Agreement and a settlement agreement was reached between the two parties. Reminder that the earlier ruling was overturned, it is no longer clear whether scraping is legal or not.
- djbusby 2y agoVery Probably No way they could find me.
- sunaookami 2y agoJust say you train an AI on it and it's all fair game.
- CalRobert 2y agoHTTP is an API of sorts…
- selcuka 2y ago> HTTP is an API of sorts… True, but it is an API that they can't easily deprecate on a whim.
- viraptor 2y agoYou're one CAPTCHA added to the flow from this access being effectively deprecated.
- xp84 2y agoIn the AI era, is this as true?
- viraptor 2y agoDepends on your development and per-action cost. And on the possible latency. It also changes your whole stack from "send a request" to "emulate each step in a browser while taking screenshots at (hopefully) the right event/delay" - that's a huge difference.
- reddalo 2y agoBut they can quickly change the "structure" of that API.
- chillfox 2y agoIf you build your scraper to find data on the page based on the shape of the data itself instead of the structure of the page then it will be resilient to most changes that don't materially change what data is displayed on the page. So, prefer regex over css selectors, and css selectors over xpath, where possible. And don't select based on nesting or position if possible.
- 2y ago