3 ms·
One of the main problems isn't just encoding XML correctly, but the additive mistakes that arise when information is copied and reused, where along the way some
by lobster_johnson 12y ago
One of the main problems isn't just encoding XML correctly, but the additive mistakes that arise when information is copied and reused, where along the way some part of the chain does a mistake:
- Information scraped from a web page that was in ISO-8859-1
- Stored in a database that is Windows-1252
- Then emitted through an API in UTF-8 by someone who writes strings by ("<tag>" + string concatenatation + "</tag>")
- Then stored in a new database as UTF-8 but not sanity-checked (ie., MySQL instead of Postgres)
- Then emitted as an XML feed
...etc. Along the way someone forgets to encode the "&" and the data contains random spatterings of ISO-8895-1 characters, and you're screwed.
Most parsers I have encountered aren't lenient by default, and will barf on non-conformant input. So now the last link in the chain needs to sanitize and normalize, which is a pain.