3 ms·
One of the classic FAQs about Lua is: "Why doesn't Lua have unicode support?" I mean, it does, but in the same fashion as ANSI C: you need to either pick a co
by jacques_chester 14y ago
One of the classic FAQs about Lua is:
"Why doesn't Lua have unicode support?"
I mean, it does, but in the same fashion as ANSI C: you need to either pick a convention for multibyte characters or use a third-party library.
And that's not an accident. One part of Lua's embeddability design is that the standard implementation is written in pure ANSI C, and pure ANSI C ... doesn't support unicode.
- zurn 14y agoStandard C has wchar_t, which is used for Unicode on platforms supporting Unicode.
- charliesome 14y agowchar_t should never be used in any new code
- ygra 14y agoJust out of curiosity: What is its replacement? If I remember correctly it began its life as the char type for multi-byte character sets and later found use for UTF-(16|32) code units. Should Unicode strings forcibly be UTF-8 in all programs and use char again (thus suffering from the same problem that Python 2 had where bytestrings and Unicode strings weren't sufficiently distinguished, often leading to programmers using the former for text)? Or should you use short/int arrays instead and have no support from the stdlib at all for handling them (not that the stdlib is particularly good at handling text)?
- charliesome 14y agoIf you're using UTF-8, use either uint8_t* (preferred) or char. If you're using UTF-32, use uint32_t. If you're using UTF-16, don't. As far as library support goes, there are a few good Unicode libraries for C, most notably ICU4C.
- deleted 14y ago[deleted]
- jacques_chester 14y agowchar_t is implementation-specific.
- zurn 14y agoSure. char is not specified to support ASCII either but it's not really a problem in practice :) If you start avoiding all the things that are not fully specified by the standard, your C programs will end up doing very little indeed. Now, it might be that some environments targeted by Lua have very spartan C runtimes: consoles. Anyone have a pointer to the C runtime docs for Xbox 360 and PS3?
- dalke 14y agoI don't understand how writing it in "pure ANSI C" prohibits Unicode support. Python, for example, is written in "pure ANSI C" and does support Unicode. Though doing so correctly and completely is hard, and requires code and data which bulks up what is otherwise a compact embeddable implementation.
- jacques_chester 14y agoBasically, it's as you guessed: Lua's string implementation is pretty much a transparent skin over C strings. The main difference is that Lua checks the boundaries and collects garbage. It's a real turnoff for lots of people. In terms of "things new to Lua find unattractive", it's second only to 1-based indexing.