3 ms·
There are a lot more web sites than scrapers though. Why push the work to all the web sites in the world instead of fixing your scraper? But in any case, you a
by goto11 7y ago
There are a lot more web sites than scrapers though. Why push the work to all the web sites in the world instead of fixing your scraper?
But in any case, you are never getting all the web sites in the world to validate, so what does it matter if you can get some of them to validate? You still have to handle the rest.
- oefrha 7y ago"it will help with your own sanity" Writing unclosed/out-of-order tag soup is extra work, not less work. What a weird hill to die on.
- goto11 7y agoWhat people don't get about the web is that there is a significant (and growing) number of pages written by people who are now dead. So all the well-intentioned ideas about "cleaning up" html - it's not going to happen. It's not a question about being pro or con closing tags or which is "best". The question is about the billions of pages which already exist and is not going to change. If you write a web scraper, that is just the reality you work with.
- oefrha 6y agoWhere did I call for people to clean up existing HTML? Nowhere. Did I ask people to not follow bad advice (intentionally not closing non-self-closing tags just for the hell of it)? Absolutely.