4 ms·
If API doesn’t have the data you want, this point is moot.
by hartator 10mo ago
If API doesn’t have the data you want, this point is moot.
- dotancohen 10mo agoNot GP, but I disagree. I've written successful, robust web scrapers without LLMs for decades. What do you think the E in perl stands for?
- deleted 10mo ago[deleted]
- llbbdd 10mo agoThis is probably just a parallel discussion. I written plenty of successful web scrapers without LLM's, but in the last couple years, I've written a lot more where I didn't need to look at the web markup for more than a few seconds first, if at all. Often you can just copy-paste an example page into the LLM and have it generate accurate, consistent selectors. It's not much different when integrating with a formal API, except that the API usually has more explicit usage rules, and APIs will also often restrict data that can very obviously be used competitively.
- llbbdd 10mo agoDouble-posting so I'm sorry but the more I read this the less it makes sense. The parent reply was talking about data that was straight-up not available via the API, how does perl help with that?