4 ms·

A few copies aren't gonna kill us, but this may very well might, if charAt() is now O(n): for (int i = 0; i < str.length(); i++) doSomethingWith(st

A few copies aren't gonna kill us, but this may very well might, if charAt() is now O(n): for (int i = 0; i < str.length(); i++) doSomethingWith(str.charAt(i)); UTF-8 just a text encoding. If you're making something new, yeah, it's the obvious choice, but it's not better enough to justify breaking all sorts of shit just to switch over.

davexunit 3y ago
A copy for every string passed to the DOM API, to name just one thing, will be a significant limiting factor.
- alexvitkov 3y ago
  There are some Rust front-end web frameworks that presumably manipulate the DOM, and in C++/Rust to pass a string to JS you need to run a TextDecoder over your WASM memory, so it's probably not a deal breaker. But like... if you're writing a website, just use JavaScript.
  - davexunit 3y ago
    C++/Rust use WASM linear memory, but this article is about reference types via WASM GC. UTF-8 data in an (array i8) or UTF-16 data in an (array i16) are opaque to the host.
    - alexvitkov 3y ago
      Yeah, and you still have to marshal strings at the JS/WASM boundry, same as if you used (array i8/16) over JS strings in Java. In the case of non-managed strings, this overhead hasn't been big enough to stop people from writing fairly fast (by Web standards) frontend frameworks in Rust.
Ygg2 3y ago
That code makes zero sense in Unicode. First question how are representing your umlauts, followed by Zero Join Width Characters. You never work on characters, you work on grapheme clusters or whatnot but never characters.
- alexvitkov 3y ago
  I'm not advocating you write this, I'm saying people have written it, probably hundreds of thousands of times, and if charAt() becomes O(n) instead of O(1), this code suddenly hangs your CPU for 10 seconds on a long string, thus you can't really swap out UTF-16 for UTF-8 transparently.
  - Ygg2 3y ago
    Your point doesn't stand for UTF-16 either. It's not a fixed length encoding either. It's broken in UTF-16 as well. It was always O(n). Of course assuming you aren't using UTF-32, which has its own set of problems (BE or LE), and sees little usage outside of China.
    - taway1237 3y ago
      ...it's not O(n). Many languages, JS, Java and C# included, have O(1) access to a character at a given position. You correctly note that it won't work well with international strings, but GP is right that A LOT of code like this was written by western ASCII-brained developers.
    - alexvitkov 3y ago
      Haven't used Java in a while but I believe charAt() returns a UTF-16 codepoint and is constant time access. So something like the above works not only for ASCII, as well as for the majority of Western languages and special characters you may encounter on a day to day basis.
      - Ygg2 3y ago
        It's constant time iff you ignore surrogate pairs and Unicode. By that logic UTF8 is constant time if you ignore anything not ASCII because most text is in English. Saying it works fine if you ignore errors and avoid edge cases is just a clever rephrashing of it worked on my machine. Plus Emojis are Unicode U+1F600 and above, so even in Western language you are bound to find such "exceptions" .

davexunit 3y ago

A copy for every string passed to the DOM API, to name just one thing, will be a significant limiting factor.

alexvitkov 3y ago
There are some Rust front-end web frameworks that presumably manipulate the DOM, and in C++/Rust to pass a string to JS you need to run a TextDecoder over your WASM memory, so it's probably not a deal breaker. But like... if you're writing a website, just use JavaScript.
- davexunit 3y ago
  C++/Rust use WASM linear memory, but this article is about reference types via WASM GC. UTF-8 data in an (array i8) or UTF-16 data in an (array i16) are opaque to the host.
  - alexvitkov 3y ago
    Yeah, and you still have to marshal strings at the JS/WASM boundry, same as if you used (array i8/16) over JS strings in Java. In the case of non-managed strings, this overhead hasn't been big enough to stop people from writing fairly fast (by Web standards) frontend frameworks in Rust.

Ygg2 3y ago

That code makes zero sense in Unicode. First question how are representing your umlauts, followed by Zero Join Width Characters. You never work on characters, you work on grapheme clusters or whatnot but never characters.

alexvitkov 3y ago
I'm not advocating you write this, I'm saying people have written it, probably hundreds of thousands of times, and if charAt() becomes O(n) instead of O(1), this code suddenly hangs your CPU for 10 seconds on a long string, thus you can't really swap out UTF-16 for UTF-8 transparently.
- Ygg2 3y ago
  Your point doesn't stand for UTF-16 either. It's not a fixed length encoding either. It's broken in UTF-16 as well. It was always O(n). Of course assuming you aren't using UTF-32, which has its own set of problems (BE or LE), and sees little usage outside of China.
  - taway1237 3y ago
    ...it's not O(n). Many languages, JS, Java and C# included, have O(1) access to a character at a given position. You correctly note that it won't work well with international strings, but GP is right that A LOT of code like this was written by western ASCII-brained developers.
  - alexvitkov 3y ago
    Haven't used Java in a while but I believe charAt() returns a UTF-16 codepoint and is constant time access. So something like the above works not only for ASCII, as well as for the majority of Western languages and special characters you may encounter on a day to day basis.
    - Ygg2 3y ago
      It's constant time iff you ignore surrogate pairs and Unicode. By that logic UTF8 is constant time if you ignore anything not ASCII because most text is in English. Saying it works fine if you ignore errors and avoid edge cases is just a clever rephrashing of it worked on my machine. Plus Emojis are Unicode U+1F600 and above, so even in Western language you are bound to find such "exceptions" .