3 ms·
Sorry if this sounds a little clueless, but it sounds like the problem is because there's multiple Unicode standards (UTF-8, UTF-16, UTF-32). So it seems like
by magduf 8y ago
Sorry if this sounds a little clueless, but it sounds like the problem is because there's multiple Unicode standards (UTF-8, UTF-16, UTF-32). So it seems like if you just re-encode everything into one of these before committing to the storage layer, that you'd avoid this problem altogether, and you'd be able to do operations in the storage layer correctly too.
- jrochkind1 8y agoYeah, no. That is actually a different issue. Curious if you read the OP? But, no, you do not avoid issues with unicode normalization by ensuring everything is in (eg) UTF-8. Unicode normalization is a thing within UTF-8.
- happytoexplain 8y agoA Unicode string is an array of Unicode's building blocks: Code points. Normalization/composition refers to how we use code points to represent characters. Encoding refers to how we use bytes to represent code points. The two concepts exist at two independent levels of abstraction.
- dummyfunnytoo 8y agoAccurate but so arrogant. HN purity achieved.
- happytoexplain 8y agoSorry, I didn't intend that. I honestly considered it a reasonable question.
- FabHK 8y agoNo worries, your reply was a concise clarification of the issue and terminology, and a reasonable answer; I didn't perceive it as arrogant in the least.
- magduf 8y agoOP here; I didn't perceive it as arrogant either. I think the "arrogant" accuser might have a point about some stuff I see on HN, but this reply to my post just isn't it.