3 ms·
Extracting web page content using Readability.js and shot-scraper
- simonw 5y agoHere's the terminal one-liner: shot-scraper javascript https://simonwillison.net/2022/Mar/24/datasette-061/ " async () => { const readability = await import('https://cdn.skypack.dev/@mozilla/readability'); return (new readability.Readability(document)).parse(); }" To first install shot-scraper, run: pip install shot-scraper shot-scraper install
- rahimnathwani 5y agoSmall typo on the page: the tech that powers shot-scarper
- jaytaylor 5y agoThis looks cool. In the past, I've had good success with using GoOse. There are implementations in Scala as well as Golang. https://github.com/advancedlogic/GoOse https://github.com/advancedlogic/GoOse
- johnsbrayton 5y agoFWIW I offer a (paid) webpage content extraction API: https://www.goldenhillsoftware.com/webpage-text-api/ https://www.goldenhillsoftware.com/webpage-text-api/
- zhengiszen 5y agoThank you @simonw thanks to you I get to learn Python and pipelines :p