3 ms·

The forced unicode String complaint is quite valid. Implementing any sort of large in-memory index in Java basically requires a parallel implementation of Stri

by drp 17y ago

The forced unicode String complaint is quite valid. Implementing any sort of large in-memory index in Java basically requires a parallel implementation of String (not actually 'implements String') to keep its memory usage down to a reasonable level. Unless of course you're actually indexing Unicode strings.

gtufano 17y ago
I've been countless hit by programs crashing because I'm using an accented character. It is really easy to think "I'm not using unicode strings there"... until an user uses unicode characters. Enforcing unicode is a good thing, whatever english speakers could think about.
axod 17y ago
IMHO this is something Java got totally right. With many other languages Unicode is some afterthought. Ever tried Unicode things in php? It's horrible. Having all strings as Unicode is fantastic. If you're dealing with something else and can't afford the memory, then sure - roll your own with byte[] arrays. Not a big deal.
- ars 17y ago
  I use unicode in php routinely. It works fine. If you need sub-string manipulations add mb_ to stuff (and if you are manipulating an entire string you need do nothing special). And if adding mb_ is too much trouble use mbstring.func_overload. UTF-8 only of course. All the other encodings should never be used anywhere. With UTF-8 the majority of the time you can use regular functions - for example trim() works just fine on a UTF-8 string.
- neilc 17y ago
  I'm curious why Java doesn't use UTF-8 encoding by default; that would offer much of the memory savings the parent poster is talking about, if the data is mostly ASCII anyway.
  - axod 17y ago
    because it'd have an extreme effect on performance? UTF-8 is great in terms of size, but terrible in terms of speed.
    - neilc 17y ago
      How "extreme" are we talking about? Some empirical data would be helpful. I'd also be curious to see data about what % of the characters in all the strings in a typical Java program aren't in 7-bit ASCII. I wouldn't be surprised if 7-bit ASCII characters are a large majority of the total, which makes the case for UTF-16 less compelling.

by drp 17y ago

gtufano 17y ago

I've been countless hit by programs crashing because I'm using an accented character. It is really easy to think "I'm not using unicode strings there"... until an user uses unicode characters. Enforcing unicode is a good thing, whatever english speakers could think about.

axod 17y ago

IMHO this is something Java got totally right. With many other languages Unicode is some afterthought. Ever tried Unicode things in php? It's horrible. Having all strings as Unicode is fantastic. If you're dealing with something else and can't afford the memory, then sure - roll your own with byte[] arrays. Not a big deal.

ars 17y ago
I use unicode in php routinely. It works fine. If you need sub-string manipulations add mb_ to stuff (and if you are manipulating an entire string you need do nothing special). And if adding mb_ is too much trouble use mbstring.func_overload. UTF-8 only of course. All the other encodings should never be used anywhere. With UTF-8 the majority of the time you can use regular functions - for example trim() works just fine on a UTF-8 string.
neilc 17y ago
I'm curious why Java doesn't use UTF-8 encoding by default; that would offer much of the memory savings the parent poster is talking about, if the data is mostly ASCII anyway.
- axod 17y ago
  because it'd have an extreme effect on performance? UTF-8 is great in terms of size, but terrible in terms of speed.
  - neilc 17y ago
    How "extreme" are we talking about? Some empirical data would be helpful. I'd also be curious to see data about what % of the characters in all the strings in a typical Java program aren't in 7-bit ASCII. I wouldn't be surprised if 7-bit ASCII characters are a large majority of the total, which makes the case for UTF-16 less compelling.