3 ms·
Neat. Do you have a recommendation for a headless browser library that would simplify this? The difference between checking the raw HTML and visible HTML can be
by arjie 10y ago
Neat. Do you have a recommendation for a headless browser library that would simplify this? The difference between checking the raw HTML and visible HTML can be pretty big.
- altavox 10y agoIn client-side Javascript, document.documentElement.innerText yields only the rendered text, without markup, CSS or JavaScript source.
- arjie 10y agoBut the theory was that the source would be different, and only the rendered content would be the same at the sample place, right? How would you find the 'same place' without fully rendering the page?
- tedmiston 10y agoOne way is for the source to use cache control headers so you can just do a quick HEAD instead of a GET on the whole contents.
- zlagen 10y agolxml provides a module to perform html diffs mainly looking at the content rather than markup. It's not perfect but most of the times works. http://lxml.de/api/lxml.html.diff-module.html http://lxml.de/api/lxml.html.diff-module.html For Go I don't know any. When I was working on a project like this another issue that I found regularly is that there are regions of the page(counters, advertising, etc.) that change all the time too but they are part of the content. For this I would make several requests to the page and find the regions that would change in every request and disregard them when doing the diff.