3 ms·

Standards objections

42 points by the_mitsuhiko 4y ago

the_mitsuhiko 4y ago
It’s worth mentioning since the title doesn’t say much that AssemblyScript removed WASI support. There are still ways to get it to work but they clearly are dissatisfied.
puffoflogic 4y ago
The author seems to be under the impression that utf8 and Unicode are distinct, incompatible string types, and the rest of the objections follow. Bizarre.
- dcode 4y ago
  That's not the issue / impression. The issue is that well-formed UTF-16 is rarely used in practice. All of JavaScript, Java, C#, Dart, Kotlin etc. effectively use WTF-16 for compatibility and performance reasons, and that's what's semantically distinct from UTF-8. These have asymmetric value spaces, so that strings in "legacy" languages would sometimes throw or implicitly mutate synchronously. Mixed systems typically use WTF-8 as the common denominator for this reason, i.e. not UTF-8, but Wasm decided against.
  - the_mitsuhiko 4y ago
    I'm sympathetic to the argument but I'm not convinced that the UTF-16 string format in those languages is much of a hurdle. Even Python which adopted the pretty ridiculous internal UCS4 encoding is now carrying around a utf-8 pre-encoded version of strings for crossing boundaries. UTF-8 is just too widespread that many languages can avoid having to support it natively in some form. Likewise WASM is not the first standard that has opinions about string encodings that are not native to a language. For instance Go and Rust which prefer to use UTF-8 internally have to re-encode on the way to Windows APIs (usually!). Likewise Cocoa/Objective-C traditionally use UCS-2 strings which are quite leaky, yet Swift nowadays uses UTF-8 internally and transcodes.
    - dcode 4y ago
      To me it's not so much a question of what's the best / recommended (for new languages) / most used encoding. It's rather the observation that there are so many popular languages operating on Unicode Code Points, not Unicode Scalar Values, and Wasm said it wants to support these (as) well, incl. integration with JS/Web APIs which already share their semantics. And for all of these, the restriction is unnecessary and trivially avoidable by allowing them to pass strings to each other, or from/to JavaScript, without errors or mutation, while not changing anything for those preferring UTF-8. Looking at this the other way around might be valuable as well: What if one would design a Component Model for any of the affected languages? There, introducing an unnecessary discontinuity would be a design mistake I'd argue.
      - the_mitsuhiko 4y ago
        I'm not sure I understand that point. JavaScript has one way to encode strings (WTF-16), many things that compile to WASM have another (UTF-8). The only thing that I see happening is that people generally are starting to recommend the use of UTF-8 as internal string encoding, and particularly across bridges. The question is IMO not really about if someone operates on unicode code points or not, but what encoding strings shared across a WASM bridge should have and coming to one makes sense.
        dcode 4y ago
        In more technical jargon, it is about value spaces. All UTF-8 strings map to WTF-16 strings semantically (lists of Unicode Scalar Values are a subset of lists of Unicode Code Points), but some WTF-16 strings do not map to UTF-8 (Unicode Scalar Values exclude some Code Points). That's something UTF-8-based languages have to deal with anyway (on the Web, which is WTF-16, or any other mixed system), but it's odd that the expectation has become that even languages that map to each other incl. to JS/Web APIs now have to share a problem that does not exist in their native VMs for reasons. To emphasize: These languages just do what's perfectly fine in WebIDL, JSON, ECMAScript and their own language specifications. let myString = "..."; let returnedMyString = someFunction(myString); // might or might not cross boundary if (myString == returnedMyString) { // sometimes silently false } I am having a hard time to understand how so many people prefer this as the outcome of a Web / polyglot / language neutral standard, especially since affected languages cannot change, and the problem is so trivially avoided, say with a boolean flag "this end is WTF-16, it's OK if the other end is as well" (otherwise use well-formed/UTF-8 semantics). For context, I once gave a presentation about the pitfalls: https://www.youtube.com/watch?v=Ri2NMnSQo4o https://www.youtube.com/watch?v=Ri2NMnSQo4o
- Jasper_ 4y ago
  Yep. See my comments on this the last time this came up: https://news.ycombinator.com/item?id=32767673 https://news.ycombinator.com/item?id=32767673
jasonwatkinspdx 4y ago
At the end of the day this person's argument is "WASI providing posix like APIs to WebAssembly is harmful for the web" and they are very angry the larger community disagrees with them. Despite all the diagrams, words, and tables repeating "In Violation" the author can't make an argument more substantiated than their own opinion of what's appropriate for the web. It's also clear this person would be extremely unpleasant to collaborate with. So none of this is surprising.
riskable 4y ago
I think the author is viewing WebAssembly strictly in the context of using it inside of a web browser on a web page. The problem with that view is that WebAssembly's goals go far beyond that, being more of a (much more) portable runtime/binary format akin to a Flatpak or AppImage. In the context of a web browser having low-level access to the client's filesystem (e.g. fd_write) seems like a bad idea/security problem. However, if you're executing WASM from the perspective of, "I'm running this on my desktop" or "this is a microservice I am running on my server" suddenly low-level filesystem access makes a lot more sense. It's up to the implementation to add controls around or completely restrict things like filesystem access. There's no reason why such features can't be included in the larger standard of WebAssembly. It's similar to .webm being a subset of the Matroska (.mkv) container format... For the web, specifically the folks that defined .webm (Google) decided that they didn't want or need all the features of Matroska.
- pie_flavor 4y ago
  It has been painstakingly designed not to be a security problem, and I would really recommend becoming familiar with the APIs in question. Every last I/O function is capability-based; for example, you must have a directory handle in order to open a file or subdirectory in it. Initial handles are provided by the runtime.
deleted 4y ago
[deleted]
maxgraey 4y ago
Ah shit, here we go again. I just link to my previous comment which quoted in WasmWeekly News https://news.ycombinator.com/item?id=32564105 https://news.ycombinator.com/item?id=32564105
syrusakbary 4y ago
I fully have to agree that it seems like new standards on the Wasm ecosystem are created to get dominion over them. WASI is one clear example of this (but not only, the Component Model is another example where only one runtime is pushing for this). To be honest, that's not the way standards should be done: standards are named this way because they require wide adoption. Until that happens, they are not standards... just technical specifications. In general I believe there's a great opportunity to create something based on true Web Standards and not proprietary APIs driven by the big corps, that also can run fully server side.

42 points by the_mitsuhiko 4y ago

It’s worth mentioning since the title doesn’t say much that AssemblyScript removed WASI support. There are still ways to get it to work but they clearly are dissatisfied.

puffoflogic 4y ago

The author seems to be under the impression that utf8 and Unicode are distinct, incompatible string types, and the rest of the objections follow. Bizarre.

dcode 4y ago
That's not the issue / impression. The issue is that well-formed UTF-16 is rarely used in practice. All of JavaScript, Java, C#, Dart, Kotlin etc. effectively use WTF-16 for compatibility and performance reasons, and that's what's semantically distinct from UTF-8. These have asymmetric value spaces, so that strings in "legacy" languages would sometimes throw or implicitly mutate synchronously. Mixed systems typically use WTF-8 as the common denominator for this reason, i.e. not UTF-8, but Wasm decided against.
- the_mitsuhiko 4y ago
  I'm sympathetic to the argument but I'm not convinced that the UTF-16 string format in those languages is much of a hurdle. Even Python which adopted the pretty ridiculous internal UCS4 encoding is now carrying around a utf-8 pre-encoded version of strings for crossing boundaries. UTF-8 is just too widespread that many languages can avoid having to support it natively in some form. Likewise WASM is not the first standard that has opinions about string encodings that are not native to a language. For instance Go and Rust which prefer to use UTF-8 internally have to re-encode on the way to Windows APIs (usually!). Likewise Cocoa/Objective-C traditionally use UCS-2 strings which are quite leaky, yet Swift nowadays uses UTF-8 internally and transcodes.
  - dcode 4y ago
    To me it's not so much a question of what's the best / recommended (for new languages) / most used encoding. It's rather the observation that there are so many popular languages operating on Unicode Code Points, not Unicode Scalar Values, and Wasm said it wants to support these (as) well, incl. integration with JS/Web APIs which already share their semantics. And for all of these, the restriction is unnecessary and trivially avoidable by allowing them to pass strings to each other, or from/to JavaScript, without errors or mutation, while not changing anything for those preferring UTF-8. Looking at this the other way around might be valuable as well: What if one would design a Component Model for any of the affected languages? There, introducing an unnecessary discontinuity would be a design mistake I'd argue.
    - the_mitsuhiko 4y ago
      I'm not sure I understand that point. JavaScript has one way to encode strings (WTF-16), many things that compile to WASM have another (UTF-8). The only thing that I see happening is that people generally are starting to recommend the use of UTF-8 as internal string encoding, and particularly across bridges. The question is IMO not really about if someone operates on unicode code points or not, but what encoding strings shared across a WASM bridge should have and coming to one makes sense.
      - dcode 4y ago
        In more technical jargon, it is about value spaces. All UTF-8 strings map to WTF-16 strings semantically (lists of Unicode Scalar Values are a subset of lists of Unicode Code Points), but some WTF-16 strings do not map to UTF-8 (Unicode Scalar Values exclude some Code Points). That's something UTF-8-based languages have to deal with anyway (on the Web, which is WTF-16, or any other mixed system), but it's odd that the expectation has become that even languages that map to each other incl. to JS/Web APIs now have to share a problem that does not exist in their native VMs for reasons. To emphasize: These languages just do what's perfectly fine in WebIDL, JSON, ECMAScript and their own language specifications. let myString = "..."; let returnedMyString = someFunction(myString); // might or might not cross boundary if (myString == returnedMyString) { // sometimes silently false } I am having a hard time to understand how so many people prefer this as the outcome of a Web / polyglot / language neutral standard, especially since affected languages cannot change, and the problem is so trivially avoided, say with a boolean flag "this end is WTF-16, it's OK if the other end is as well" (otherwise use well-formed/UTF-8 semantics). For context, I once gave a presentation about the pitfalls: https://www.youtube.com/watch?v=Ri2NMnSQo4o https://www.youtube.com/watch?v=Ri2NMnSQo4o
Jasper_ 4y ago
Yep. See my comments on this the last time this came up: https://news.ycombinator.com/item?id=32767673 https://news.ycombinator.com/item?id=32767673

jasonwatkinspdx 4y ago

At the end of the day this person's argument is "WASI providing posix like APIs to WebAssembly is harmful for the web" and they are very angry the larger community disagrees with them. Despite all the diagrams, words, and tables repeating "In Violation" the author can't make an argument more substantiated than their own opinion of what's appropriate for the web. It's also clear this person would be extremely unpleasant to collaborate with. So none of this is surprising.

riskable 4y ago

I think the author is viewing WebAssembly strictly in the context of using it inside of a web browser on a web page. The problem with that view is that WebAssembly's goals go far beyond that, being more of a (much more) portable runtime/binary format akin to a Flatpak or AppImage. In the context of a web browser having low-level access to the client's filesystem (e.g. fd_write) seems like a bad idea/security problem. However, if you're executing WASM from the perspective of, "I'm running this on my desktop" or "this is a microservice I am running on my server" suddenly low-level filesystem access makes a lot more sense. It's up to the implementation to add controls around or completely restrict things like filesystem access. There's no reason why such features can't be included in the larger standard of WebAssembly. It's similar to .webm being a subset of the Matroska (.mkv) container format... For the web, specifically the folks that defined .webm (Google) decided that they didn't want or need all the features of Matroska.

pie_flavor 4y ago
It has been painstakingly designed not to be a security problem, and I would really recommend becoming familiar with the APIs in question. Every last I/O function is capability-based; for example, you must have a directory handle in order to open a file or subdirectory in it. Initial handles are provided by the runtime.

deleted 4y ago

[deleted]

maxgraey 4y ago

Ah shit, here we go again. I just link to my previous comment which quoted in WasmWeekly News https://news.ycombinator.com/item?id=32564105 https://news.ycombinator.com/item?id=32564105

syrusakbary 4y ago

I fully have to agree that it seems like new standards on the Wasm ecosystem are created to get dominion over them. WASI is one clear example of this (but not only, the Component Model is another example where only one runtime is pushing for this). To be honest, that's not the way standards should be done: standards are named this way because they require wide adoption. Until that happens, they are not standards... just technical specifications. In general I believe there's a great opportunity to create something based on true Web Standards and not proprietary APIs driven by the big corps, that also can run fully server side.