3 ms·
Any recommendations for a js lib that does all the "easy" scraping (microdata, og tags, jsonld, etc)?
by thinkloop 6y ago
Any recommendations for a js lib that does all the "easy" scraping (microdata, og tags, jsonld, etc)?
- choward 6y agoI thought that's what this blog post was going to be about but it's just an ad for their app. I just need the scraping functionality.
- hundchenkatze 6y agoWhile they do end with pushing their product, I think they did a good job of outlining how they scrape the recipes. They inform the reader about json+ld, microdata, and how to scrape the sites that don't use those. They even link to a JS lib that handles the parsing for you. I think calling it "just an ad" is inaccurate. > There are libraries like https://github.com/digitalbazaar/jsonld.js/ https://github.com/digitalbazaar/jsonld.js/ to parse JSON-LD + Microdata for you.
- greglindahl 6y agoIn Python I use the "extruct" package from the scrapy people. It's not very good with syntax errors in the markup.
- hundchenkatze 6y agoThe article recommended one. > There are libraries like https://github.com/digitalbazaar/jsonld.js/ https://github.com/digitalbazaar/jsonld.js/ to parse JSON-LD + Microdata for you.