4 ms·
A brief search shows for "UTF-16 vulnerability" finds relevant results. UTF-8 overlong sequences haven't been a problem for a while. The choice of encoding is
by cryptonector 9y ago
A brief search shows for "UTF-16 vulnerability" finds relevant results.
UTF-8 overlong sequences haven't been a problem for a while.
The choice of encoding is definitely relevant for all external file formats, protocols, APIs. We can't just let the choice of encoding be purely internal, though I agree that programmers should not need to care about what their runtimes use internally for strings, as long as it is Unicode (UTF-8, 16, 32, or some other UTF say).
But no UTF actually gives you random access by character index because a) no runtime is going to be ensuring normalization of all strings to precomposed sequences (not even NFC because NFC is closed to new compositions!), b) not every valid character will have pre-composed codepoints therefore (a) isn't even possible. Given this, UTF-16 and UTF-32 have no real advantages over UTF-8, and the false sense of simplicity they convey is dangerous anyways.