6 ms·

Python <3.3 uses UCS2 or UCS4, depending on the build Ruby >1.8 lets you choose the encoding .NET UCS2/UTF-16 (I know the difference, imho if the stdlib has a

Python <3.3 uses UCS2 or UCS4, depending on the build Ruby >1.8 lets you choose the encoding .NET UCS2/UTF-16 (I know the difference, imho if the stdlib has a .size, .length or .count that works on code units instead of code points it's broken... thus I'll mention only UCS2 from now on) Java UCS2 Clojure UCS2 Scala UCS2 QT UCS2 Haskell String UCS4 Haskell Data.Text UTF-16 (yes, not a naive UCS-2) Rust UCS4 (last time I checked) Javascript UCS2 Dart UCS2 PHP Unicode-oblivious Vala UCS4 Go UTF-8 (but it lets you call len() on strings, and it doesn't return the length of the string, but its size in bytes) I can't really think of another language that uses UTF-8 internally, are you sure?

sedachv 13y ago
Common Lisp comes with two character types, base-char and character, the former being allowed to be a subset of the latter. Clozure Common Lisp uses UTF-32 for all characters and strings internally. SBCL uses base-char and simple-base-string types for ASCII and character and (simple-array character) types for UTF-32 internally. IMO having this option for two types of characters that are compatible but may have different internal representations is a really good part of the Common Lisp standard.
pitterpatter 13y ago
> Rust UCS4 (last time I checked) Rust chars are 32bit Unicode codepoints. But strings themselves are utf-8. That is the string type, ~str, is basically just ~[u8], a vector of bytes and not ~[char]. `.len()` [O(1)] gives you byte length while `.char_len()` [O(n)] gives you the number of codepoints. So strings in rust are just vectors of bytes with the invariant that it's valid utf-8.
- berdario 13y ago
  Thanks, I didn't know that
jeltz 13y ago
Perl, Rust, Go and Vala. I take back the "most" part though. It seems like there are many popular solutions. Erlang uses Unicode code points and also binaries.

sedachv 13y ago

Common Lisp comes with two character types, base-char and character, the former being allowed to be a subset of the latter. Clozure Common Lisp uses UTF-32 for all characters and strings internally. SBCL uses base-char and simple-base-string types for ASCII and character and (simple-array character) types for UTF-32 internally. IMO having this option for two types of characters that are compatible but may have different internal representations is a really good part of the Common Lisp standard.

pitterpatter 13y ago

> Rust UCS4 (last time I checked) Rust chars are 32bit Unicode codepoints. But strings themselves are utf-8. That is the string type, ~str, is basically just ~[u8], a vector of bytes and not ~[char]. `.len()` [O(1)] gives you byte length while `.char_len()` [O(n)] gives you the number of codepoints. So strings in rust are just vectors of bytes with the invariant that it's valid utf-8.

berdario 13y ago
Thanks, I didn't know that

jeltz 13y ago

Perl, Rust, Go and Vala. I take back the "most" part though. It seems like there are many popular solutions. Erlang uses Unicode code points and also binaries.