3 ms·
I like using Readability.js as a demo for my shot-scraper CLI utility: https://shot-scraper.datasette.io/en/stable/javascript.html#using-async-await https://sho
by simonw 3y ago
I like using Readability.js as a demo for my shot-scraper CLI utility: https://shot-scraper.datasette.io/en/stable/javascript.html#using-async-await https://shot-scraper.datasette.io/en/stable/javascript.html#...
Running this in a terminal (after installing shot-scraper):
shot-scraper javascript \
https://simonwillison.net/2022/Mar/14/scraping-web-pages-shot-scraper/ "
async () => {
const readability = await import('https://cdn.skypack.dev/@mozilla/readability');
return (new readability.Readability(document)).parse();
}"
Outputs this:
{
"title": "Scraping web pages from the command line with shot-scraper",
"byline": null,
"dir": null,
"lang": "en-gb",
"content": "... long string of HTML ...",
"length": 7104,
"excerpt": "I\u2019ve added a powerful new capability to my shot-scraper command line browser automation tool: you can now use it to load a web page in a headless browser, execute JavaScript \u2026",
"siteName": null,
"publishedTime": null
}
- nozzlegear 3y agoThis is really cool, thanks for sharing!
- stefankuehnel 3y agoWow, that's really cool! Thanks for sharing. I knew about "shot-scraper" before, but I didn't know you could do something so cool with it.