4 ms·
Well pretty much every other more recent language solved that problem.
by cm2187 2y ago
Well pretty much every other more recent language solved that problem.
- kccqzy 2y agoAlmost no programming language, perhaps other than Swift, solved that problem. Just use the article's examples as test cases. It's just as wrong as the C++ version in the article, except it's wrong with nicer syntax.
- zahlman 2y agoPython's strings have uppercase, lowercase and case-folding methods that don't choke on this. They don't use UTF-16 internally (they can use UCS-2 for strings whose code points will fit in that range; while a string might store code points from the surrogate-pair range, they're never interpreted as surrogate pairs, but instead as an error encoding so that e.g. invalid UTF-8 can be round-tripped) so they're never worried about surrogate pairs, and it knows a few things about localized text casing: >>> 'ß'.upper() 'SS' >>> 'ß'.lower() 'ß' >>> 'ß'.casefold() 'ss' There are a lot of really complicated tasks for Unicode strings. String casing isn't really one of them. (No, Python can't turn 'SS' back into 'ß'. But doing that requires metadata about language that a string simply doesn't represent.)
- kccqzy 2y agoStill breaks on, for example, Turkish i vs İ. It's impossible to do correctly without language information. > (No, Python can't turn 'SS' back into 'ß'. But doing that requires metadata about language that a string simply doesn't represent.) Yes that's my point. Because in typical languages strings don't store language metadata, this is impossible to do correctly in general.
- zahlman 2y agoI'm not seeing anything in the Swift documentation about strings carrying language metadata, either, though?
- kccqzy 2y agoThis lowercase function takes a locale argument https://developer.apple.com/documentation/foundation/nsstring/1417298-lowercased https://developer.apple.com/documentation/foundation/nsstrin... It looks like an old NSString method that's available in both Obj-C and Swift. The casefold function is even older than that. https://developer.apple.com/documentation/foundation/nsstring/1413779-folding https://developer.apple.com/documentation/foundation/nsstrin... Its documentation specifically includes a discussion of the Turkish İ/I issue.
- tedunangst 2y agoBut that's wrong. The upper case for ß is ẞ.
- IncreasePosts 2y agoThat was only adopted in Germany like 7 years ago!
- kccqzy 2y agoWell languages and conventions change. The € sign was added not that long ago and it was somewhat painful. The Chinese language uses a single character to refer to chemical elements so when IUPAC names new elements they will invent new characters. Etc.
- extraduder_ire 2y agoDoes unicode have space set aside for those new symbols to slot into? I know it's very rare, but it could get messy.
- account42 2y agoUnicode is already messy. Chinese characters especially so due to han unificiation.
- cm2187 2y agoC#'s "ToUpper" takes an optional CultureInfo argument if you want to play around with how to treat different languages. Again, solved problem decades ago.
- account42 2y agoThis is not a locale issue, it's a Unicode version issue. Which hightlights another problem with adding this to the base standard library.
- 2y ago
- crote 2y agoBut that's wrong. The uppercase for "in Maßen" ("in moderate amounts") is not "IN MASSEN" ("in Massen", meaning "in massive amounts").
- tialaramex 2y agoRust will cheerfully: assert_eq!("ὀδυσσεύς", "ὈΔΥΣΣΕΎΣ".to_lowercase()); [Notice that this is in fact entirely impossible with the naive strategy since Greek cares about position of symbols] Some of the latter examples aren't cases where a programming language or library should just "do the right thing" but cases of ambiguity where you need locale information to decide what's appropriate, which isn't "just as wrong as the C++ version" it's a whole other problem. It isn't wrong to capitalise A-acute as a capital A-acute, it's just not always appropriate depending on the locale.
- account42 2y agoIs this assert_eq!("\u1F41δυσσεύς", "ὈΔΥΣΣΕΎΣ".to_lowercase()); or assert_eq!("\u03BF\u0314δυσσεύς", "ὈΔΥΣΣΕΎΣ".to_lowercase()); For display it doesn't matter but most other applications really want some kind of normalizatin which does much much more so having a convenient to_lowercase() doesn't buy you as much as you think and can be actively misleading.