4 ms·
> as there is almost no language that can be written using just that. 99% of use cases I've seen have nothing to do with human language. 1% human language cas
by vardump 2y ago
> as there is almost no language that can be written using just that.
99% of use cases I've seen have nothing to do with human language.
1% human language case that is needs to be handled properly using a proper Unicode library.
Your mileage (percentages) may vary depending on your job.
- 9dev 2y agoIt's funny how software developers live in bubbles so much. Whether you deal with human language a lot or almost not at all depends entirely on your specific domain. Anyone working on user interfaces of any kind must accommodate for proper encoding, for example; that includes pretty much every line-of-business app out there, which is a lot of code.
- inexcf 2y agoWhy do you need upper- or lowercase conversion in cases that have nothing to do with human language?
- vardump 2y agoHere's an example. Hypothetically say you want to build an HTML parser. You might encounter tags like <html>, <HTML>, <Html>, etc., but you want to perform a hash table lookup. So first you're going to normalize to either lower- or uppercase.
- Muromec 2y agoBut but, I want to have a custom web component and register it under my own name, which can only be properly written in Ukrainian Cyrillic. How dare you not let me have it.
- inexcf 2y agoAh, i see, we disagree on what is "human language". An abbreviation like HTML and it's different capitalisations to me sound a lot like a feature of human language.
- recursive 2y agoIs this a serious argument? Humans don't directly use HTML to communicate with each other. It's a document markup language rendered by user agents, developed against a specification.
- tannhaeuser 2y agoMarkup languages and SGML in particular absolutely are designed for digital text communication by humans and to be written using plain text editors; it's kindof the entire point of avoiding binary data constructs. And to GP, SGML/HTML actually has a facility to define uppercasing rules beyond ASCII, namely the LCNMSTRT, UCNMSTRT, LCNMCHAR, UCNMCHAR options in the SYNTAX NAMING section in the SGML declaration introduced in the "Extended Naming Rules" revision of ISO 8879 (SGML std, cf. https://sgmljs.net/docs/sgmlrefman.html https://sgmljs.net/docs/sgmlrefman.html). Like basically everything else on this level, these rules are still used by HTML 5 to this date, and in particular, that while elements names can contain arbitrary characters, only those in the IRV (ASCII) get case-folded for canonization.
- recursive 2y agoHTML is a text-based medium. But that doesn't make it a human language. Some human languages are not text-based. And some text is not a human language. ANSI C was designed to be written by humans using a plain text editor. That doesn't make it a human language.
- ARandumGuy 2y agoConverting string case is almost never something you want to do for text that's displayed to the end user, but there are many situations where you need to do it internally. Generally when the spec is case insensitive, but you still need to verify or organize things using string comparison.
- kergonath 2y agoRight. That’s why I still get mail with my name mangled and my street name barely recognisable. Because I’m in the 1%. Too bad for me… In all seriousness, though, in the real world ASCII works only for a subset of a handful of languages. The vast majority of the population does not read or write any English in their day to day lives. As far as end users are concerned, you should probably swap your percentages. ASCII is mostly fine within your programs like the parser you mention in your other comment. But even then, it’s better if a Chinese user name does not break your reporting or logging systems or your parser, so it’s still a good idea to take Unicode seriously. Otherwise, anything that comes from a user or gets out of the program needs to behave.
- vardump 2y agoI said use a Unicode library if input data is actual human language. Which names and addresses are. 99% case being ASCII data generated by other software of unknown provenance. (Or sometimes by humans, but it's still data for machines, not for humans.)
- kergonath 2y agoI am really not sure about this 99%. A lot of programs deal with quite a lot of user-provided data, which you don’t control.
- account42 2y agoUser-provided data, yes, but also data where you can treat non-ASCII bytes as garbage in -> garbage out. E.g. the config file might be typed by a human but if you need to support case-insensitive keys you still don't need to worry about Unicode.
- kergonath 2y agoExactly. But in this case, don’t try to upper-case or otherwise transform anything.
- 2y ago
- elpocko 2y agoEvery search feature everywhere has to be case-insensitive or it's unusable. Search seems like a pretty ubiquitous feature in a lot of software, and has to work regardless of locale/encoding.