3 ms·
A char in C++ is one byte, right? Is it even possible for this "fixed" code to call ctype::tolower() on something like a UTF-8 or UTF-16 code point?
by ramshorns 6y ago
A char in C++ is one byte, right? Is it even possible for this "fixed" code to call ctype::tolower() on something like a UTF-8 or UTF-16 code point?
- kentonv 6y agoCorrect, it won't even work as intended with modern Unicode locales.
- ramshorns 6y agoSo maybe if the code is broken anyway for non-ASCII characters, it's fine to use tolower, since somewhere else in the code it ensures that c is a byte.
- kentonv 6y agoThe code is not broken for non-ASCII characters. UTF-8 works just fine with 8-bit chars, and the code I wrote correctly lower-cases ASCII letters even when UTF-8 is present (it just won't touch the UTF-8 chars, which is fine in this use case). It's only tolower() and toupper() specifically that are broken because they expect to be able to do their job on a single byte, which is no longer possible with UTF-8. Meanwhile, using tolower() to lower-case an HTTP header name won't give you the correct results if the locale is set to Turkish with the ISO 8859-9 character set, which is 8-bit, and where tolower('I') will produce the byte 0xFD which is 'ı' in this character set.
- ramshorns 6y agoI see, thanks for the explanation.