3 ms·
This is really useful. I can remember that we had a research project once where we tried to crawl the internet for text stuff and cleaning that up was a hassle
by mqus 6y ago
This is really useful. I can remember that we had a research project once where we tried to crawl the internet for text stuff and cleaning that up was a hassle (I think we used jsoup for extracting the text). Firefox reader is not entirely flawless but more tools are definitely better.
- AshamedCaptain 6y agoI remember that at some point I was using something similar to readability.js as a pre-processor in plucker distiller (a program to spider and store websites so that you could read offline on e.g. PalmOS devices).
- loa_in_ 6y agoHave you tried Apache Tika?