4 ms·
"Might be less common now" It is less necessary now. One of the most important parts of the HTML5 standards, IMHO, is that it specifies how to parse HTML that
by jerf 4y ago
"Might be less common now"
It is less necessary now. One of the most important parts of the HTML5 standards, IMHO, is that it specifies how to parse HTML that doesn't conform to the standards in a standard way. In principle, every bag of bytes now has a standard-compliant way to parse it that every HTML5 parser should agree on. I don't use this to know how many edge cases the standard and/or implementations have, but it's a lot better than it used to be, and means that every HTML5 parser has many of the capabilities that Beautiful Soup used to (nearly-)uniquely have for parsing messy HTML.
I suspect Beautiful Soup was a non-trivial aspect of how the decision to implement such a spec was decided upon. It proved the idea to be a very valuable one at a time when most languages lacked such a library. Basically, BS won so hard that while it wasn't necessarily directly adopted as a standard, the essence of it certainly was.
- ThatPlayer 4y agoNot just HTML standards, but I've used their detwingle function because one of the sites I'm scraping has a mixture of Windows-1252 and Unicode. It was clearly stored correctly, but encoded differently depending on what page view you were looking at. For example titles in an outline were broken, but on the actual individual pages fine. Their rendering also treated multibyte characters incorrectly during truncation.