3 ms·
I've been in this boat before: Them: I need you pull data from a web site to integrate with our system. Me: Neat, how is the data exposed? Them: It's a we
by cognivore 13y ago
I've been in this boat before:
Them: I need you pull data from a web site to integrate with our system.
Me: Neat, how is the data exposed?
Them: It's a website. Web pages.
Me: I'm going to stab myself in the head now.
After spending days pulling messy HTML, attempting to navigate around with whatever method this site uses (JavaScript only maybe), and hammering everything into some sort of cohesive form you'll be seriously wishing they wasted money and time putting and API on their site.
I see he's a PhD researcher. Just sayin'.
- codygman 13y agoWell he isn't advocating that you parse html here. I thought that he was originally too, but later on you'll see he is talking about using the same URI's and content negotation.
- cognivore 13y agoIsn't that what he's advocating when he says: "...a developer can just write a script to convert the templated HTML into JSON. It’s not hard: the HTML version is open." The link makes me think so.
- girvo 13y agoThat is his point about keys: why use them, when you don't need them for viewing the data via HTML? Answer: because API's can do more, normally.
- ville 13y agoI didn't understand it as advocating scraping, just a reasoning why API keys are useless in case you are serving the same content as HTML without requiring a key.