I've had a lot of success by running HTML pages through mozilla's readability[0] tool (actually the go port of it[1]) before indexing it. [0]: https://github.c

I've had a lot of success by running HTML pages through mozilla's readability[0] tool (actually the go port of it[1]) before indexing it. [0]: https://github.com/mozilla/readability https://github.com/mozilla/readability [1]: https://github.com/go-shiori/go-readability https://github.com/go-shiori/go-readability