3 ms·
How does Erlang fare with Unicode in practice?
by dtf 17y ago
How does Erlang fare with Unicode in practice?
- zacharypinter 17y agoIf I understand it correctly, Erlang fares well here because it stores each character as a 32-bit integer in a list. The implication of that approach is more memory overhead for each character, but that allows you to treat strings as a list of characters and gives you all the benefits of the standard list-manipulation libraries.
- naz 17y ago> Erlang fares well here because it stores each character as a 32-bit integer in a list Which is silly because if you make a list of 32 bit integers that have ASCII equivalents then Erlang assumes it is a string.
- tlack 17y agoYeah, but still: it lets you operate on unicode "characters" pretty easily. With a ton of associated downsides (unclear string/list of integers dichotomy, wasted memory for ascii, etc)
- silentbicycle 17y agoThe main difference is the way it's displayed by default in the shell. Strings are handled with list operations, and Erlang is great for list operations.
- anonjon 17y agoErlang doesn't make any assumptions about what it is. To Erlang, it is a list of 32 bit integers. There is no difference between a list of integers and a list of characters. The shell will transform this list of 32 bit integers with ASCII equivalents into text for your convenience, but it doesn't do any sort of conversion or typing.
- alxv 17y agoBut that tells us nothing about the quality of Erlang's Unicode support. Is it easy in Erlang to encode a string to its UTF-8 or Latin-1 representation and vice-versa? Are there utilities in Erlang to check whether a character is a punctuation symbol, a letter or a digit? Are there utilities to normalize Unicode strings? This one is important if you need to compare two strings. Sorry if I sound a bit pedantic, but there is a lot more to good Unicode support than using wide characters.
- noss 17y agoBeing pedantic: Erlang doesn't have a 32-bit intger type, it only has integers. The implementation uses the fixnum trick known from lisp, i.e. if the value is larger than a fixnum it is implemented as a bignum.