3 ms·
Mozilla has released Readability as a standalone package so you can avoid spinning up a browser entirely: https://github.com/mozilla/readability https://github.
by privatenumber 2y ago
Mozilla has released Readability as a standalone package so you can avoid spinning up a browser entirely:
https://github.com/mozilla/readability https://github.com/mozilla/readability
- asadalt 2y agoyou would still need to run. For js based websites.
- msp26 2y agoI still wanted the browser for UBlock Origin and handling sites with heavy JS. I was using the standalone Readability script already but today I ended up dropping it for Trafilatura. It works a lot better. The inefficiency of using a browser rather than just taking the html doesn't really matter because the limiting factor is the LLM here. And yes the LLM is essential for getting clean data. None of the existing methods are flexible enough for all cases even if people say "you don't need AI to do this".