4 ms·

One of the most exciting aspects of wasm is a safer and more expressive API surface for universal FFI. Currently, the lowest common denominator is the C ABI. In

by klabb3 4y ago

One of the most exciting aspects of wasm is a safer and more expressive API surface for universal FFI. Currently, the lowest common denominator is the C ABI. In the future, it could be wasm. Obviously wasm can be much more, but just that alone can be really useful. * No custom language-A to language-B bindings necessary, only 1 wasm binding per lang. That's O(N) bindings in total, instead of O(N^2), which isn't feasible and that's why we don't have Julia-intercal or PHP-Java today (or maybe we do?)

pjmlp 4y ago
IBM and Unisys language environments on IBM i, z/OS, and ClearPath MCP, for 40 to 60 years, depending on the model. Amsterdam Compiler Toolkit, 40 years ago. CLR since 2002.
- garganzol 4y ago
  > Amsterdam Compiler Toolkit, 40 years ago. It provided only a plain C-like API, without OOP classes and stuff, pretty much like anyone else. > CLR since 2002 It's only for .NET, it's opinionated, so not a universal thing.
  - pjmlp 4y ago
    Nope, better educate yourself before replying. "The Amsterdam Compiler Kit (ACK) is a retargetable compiler suite and toolchain written by Andrew Tanenbaum and Ceriel Jacobs, since 2005 maintained by David Given. It has frontends for the following programming languages: C, Pascal, Modula-2, Occam, and BASIC." "The ACK's notability stems from the fact that in the early 1980s it was one of the first portable compilation systems designed to support multiple source languages and target platforms." "Maximum portability is achieved by using an intermediate language using bytecode, called EM. Each language front-end produces EM object files, which are then processed through several generic optimisers before being translated by a back-end into native machine code. ACK comes with a generic linker and librarian capable of manipulating files in the ACK's own a.out-based format; it will work on files containing EM code as well as native machine code. However, EM code cannot be linked to native machine code without translating the EM binary first. " -- https://en.wikipedia.org/wiki/Amsterdam_Compiler_Kit https://en.wikipedia.org/wiki/Amsterdam_Compiler_Kit "One of the exciting things in Visual Studio .NET is its language agnosticism. If a vendor has written a .NET-compliant language, you can use it in Visual Studio .NET. It'll work just as well as C# or C++ or Visual Basic. This isn't just a future feature-in-planning. There are already nearly two dozen languages being developed for Visual Studio .NET: Visual Basic, C#, C++, JScript, APL, Cobol, Eiffel, Fortran, Pascal, Perl, Python, RPG, Smalltalk, Oberon, Component Pascal, Haskell/Mondrian, Scheme, Mercury, Alice, and even the Java language." -- February 2002 issue of MSDN Magazine https://learn.microsoft.com/en-us/archive/msdn-magazine/2002/february/editor-s-note-welcome-visual-studio-net https://learn.microsoft.com/en-us/archive/msdn-magazine/2002... A common theme among WASM advocates....
    - garganzol 4y ago
      > "The Amsterdam Compiler Kit (ACK) is a retargetable compiler suite. It has frontends for the following programming languages: C, Pascal, Modula-2, Occam, and BASIC." All those languages have only plain data and functions, no OOP. So it's just a C-like FFI all over again. It does not matter whether it's a bytecode or native code: instruction set is a totally separate thing from the component model. > There are already nearly two dozen languages being developed for Visual Studio .NET ... that can only be used from .NET. And that's the point about the absence of universality. Well, you can export functions (DllExport). However, it would be just a plain FFI.
      - deleted 4y ago
        [deleted]
      - pjmlp 4y ago
        And the difference from WASM is .....?
        garganzol 4y ago
        WASM is a runtime only. It does not try to define a component model. And that's a bonus for its wider appeal and adoption. You can treat WASM just like you treat a CPU architecture. One among many. It does its thing and does it reasonably well (official 64 bit support would be awesome though). But trying to tie a component model to a specific CPU is kind of a bad idea, don't you think so? For me it causes immediate Intel iAPX 432 vibes and I fear that WASM may diverge in that utopian direction some day.
        tempodox 4y ago
        > WASM is a runtime only. It does not try to define a component model. This GitHub page seems to contradict that: https://github.com/WebAssembly/component-model/blob/main/design/high-level/Goals.md https://github.com/WebAssembly/component-model/blob/main/des...
beached_whale 4y ago
I did a test of an ALU/BP(non-vecorized JSON deserialization) heavy library last year and the price of Wasmer/wasm was only about 20-25%. For many things, that's not going to even be noticed.
panic 4y ago
How would (e.g.) strings work in this FFI, considering that wasm doesn’t have a built-in string type or any way to pass variable-length data to a function without allocating space for it in the instance’s memory, and languages have different native string representations (UTF-8 vs UTF-16 vs “UTF-16”)?
- tempodox 4y ago
  Here's an example of handling strings with the WASI SDK in the browser (no wasmer involved): https://codeberg.org/bit-fu/Browser-WASI https://codeberg.org/bit-fu/Browser-WASI
- kevingadd 4y ago
  Very poorly considering that it seems like all the people doing wasm FFI work have decided that UTF-16/WTF-16 based language maintainers can pound sand and unless your native string type is UTF-8 your string performance is gonna suck. It's sad since many widely used runtimes (JS, Java, .NET) and one widely used operating system (Windows) all use WTF-16. UTF-8 is great and all but reality needs to be taken into account when drafting specs...
  - deleted 4y ago
    [deleted]
- garganzol 4y ago
  That's a cool question. There are several aspects to that. One is storage and another is encoding. Let's start with storage. C strings, for instance, are terminated by NUL (zero) characters. Are they good enough? You would never answer that question unless you are somehow versed in math. From the mathematical standpoint, it is a mistake to narrow down the space by giving its elements special meaning. Imagine an integer number from -100 to 100. Would you be happy if somebody said you that you cannot use zero because "it has a special meaning"? No, you would be outraged every time you need to use it! (even if it's a rare event). So, from the mathematical standpoint universal strings cannot give special meanings to its characters. Now to the encoding. Is UTF-8 good enough? Or maybe UTF-16? Or something else? How to decide? Let's use math. From the math standpoint, it's always preferable to use an existing concept if it fulfills the need. So, it means that we should use the existing encoding scheme to avoid reinventing the wheel. The next thing the math dictates is that: the most minimal and normalized thing is always preferable to more complex synthetic things provided that they all do the job. This puts UTF-8 encoding at a special place - it's minimal and universal as it can encode the endless number of Unicode symbols (if we, for whatever reason, will decide to colonize the space and consequently add some more billions of characters to Unicode to dial the new cultures and knowledge in). What to do we have in result? UTF-8 string is universal and should be used in a standard component model. But how to encode string length? We already decided that giving special meaning to the NUL characters is a no-go from the math standpoint because it is opinionated and thus informal. The formal approach says that we should store the length of a string alongside the string itself. The beginning of the string is the right place because we want to know string length beforehand. What should we use? Is 32-bit unsigned integer good enough? It surely will be enough for anyone for all times, right? But the math says that a 32-bit unsigned integer can only represent [0, 2^32) range of string lengths. That's not universal because, you know, there may be foreign cultures out there in the universe who have longer strings than that. So what should we use then? The answer is: an unsigned infinite integer number with variable length encoding. We already have a Variable-Length Quantity (VLQ) concept, so let's use just that because it's already invented and universal. What do we have in the final result? The answer may surprise you: UTF-8 encoded Pascal-like immutable strings, where the string lengths are located at the start of the string and encoded with VLQ encoding. They are formally universal and pass the high mark of a standard component model as we may call it some day. At the same time, this way of building strings is quite a departure from what we have now and will have performance implications. That's why we should cut corners to make such level of universality to be practical in the computer world. An obvious measure we can take is to use size_t instead of VLQ-encoded integer for string length representation. The computer memory cannot hold strings larger than that anyway, so this will do the job. The math behind this decision is formal and clear: max_string_size = min(∞, 2 ^ sizeof(size_t)) = 2 ^ sizeof(size_t). This also means that a standard component model should have a notion of size_t-like type which is tied to the memory size of a host architecture. In this way, we make our component model dependent on a host architecture. In return, we get considerable performance benefits while preserving the formal universality which is a win.
ilaksh 4y ago
What I want is to be able to call another web assembly library from inside of one.
kllrnohj 4y ago
WASM is entirely unrelated to that issue. If you want to "solve" cross-language interop & ABI surfaces, you want something like COM.
- duped 4y ago
  That's exactly what the component model defines, but it looks to be more useful than COM or CORBA. https://github.com/WebAssembly/component-model/blob/main/design/high-level/Goals.md https://github.com/WebAssembly/component-model/blob/main/des...

by klabb3 4y ago

pjmlp 4y ago

IBM and Unisys language environments on IBM i, z/OS, and ClearPath MCP, for 40 to 60 years, depending on the model. Amsterdam Compiler Toolkit, 40 years ago. CLR since 2002.

garganzol 4y ago
> Amsterdam Compiler Toolkit, 40 years ago. It provided only a plain C-like API, without OOP classes and stuff, pretty much like anyone else. > CLR since 2002 It's only for .NET, it's opinionated, so not a universal thing.
- pjmlp 4y ago
  Nope, better educate yourself before replying. "The Amsterdam Compiler Kit (ACK) is a retargetable compiler suite and toolchain written by Andrew Tanenbaum and Ceriel Jacobs, since 2005 maintained by David Given. It has frontends for the following programming languages: C, Pascal, Modula-2, Occam, and BASIC." "The ACK's notability stems from the fact that in the early 1980s it was one of the first portable compilation systems designed to support multiple source languages and target platforms." "Maximum portability is achieved by using an intermediate language using bytecode, called EM. Each language front-end produces EM object files, which are then processed through several generic optimisers before being translated by a back-end into native machine code. ACK comes with a generic linker and librarian capable of manipulating files in the ACK's own a.out-based format; it will work on files containing EM code as well as native machine code. However, EM code cannot be linked to native machine code without translating the EM binary first. " -- https://en.wikipedia.org/wiki/Amsterdam_Compiler_Kit https://en.wikipedia.org/wiki/Amsterdam_Compiler_Kit "One of the exciting things in Visual Studio .NET is its language agnosticism. If a vendor has written a .NET-compliant language, you can use it in Visual Studio .NET. It'll work just as well as C# or C++ or Visual Basic. This isn't just a future feature-in-planning. There are already nearly two dozen languages being developed for Visual Studio .NET: Visual Basic, C#, C++, JScript, APL, Cobol, Eiffel, Fortran, Pascal, Perl, Python, RPG, Smalltalk, Oberon, Component Pascal, Haskell/Mondrian, Scheme, Mercury, Alice, and even the Java language." -- February 2002 issue of MSDN Magazine https://learn.microsoft.com/en-us/archive/msdn-magazine/2002/february/editor-s-note-welcome-visual-studio-net https://learn.microsoft.com/en-us/archive/msdn-magazine/2002... A common theme among WASM advocates....
  - garganzol 4y ago
    > "The Amsterdam Compiler Kit (ACK) is a retargetable compiler suite. It has frontends for the following programming languages: C, Pascal, Modula-2, Occam, and BASIC." All those languages have only plain data and functions, no OOP. So it's just a C-like FFI all over again. It does not matter whether it's a bytecode or native code: instruction set is a totally separate thing from the component model. > There are already nearly two dozen languages being developed for Visual Studio .NET ... that can only be used from .NET. And that's the point about the absence of universality. Well, you can export functions (DllExport). However, it would be just a plain FFI.
    - deleted 4y ago
      [deleted]
    - pjmlp 4y ago
      And the difference from WASM is .....?
      - garganzol 4y ago
        WASM is a runtime only. It does not try to define a component model. And that's a bonus for its wider appeal and adoption. You can treat WASM just like you treat a CPU architecture. One among many. It does its thing and does it reasonably well (official 64 bit support would be awesome though). But trying to tie a component model to a specific CPU is kind of a bad idea, don't you think so? For me it causes immediate Intel iAPX 432 vibes and I fear that WASM may diverge in that utopian direction some day.
        tempodox 4y ago
        > WASM is a runtime only. It does not try to define a component model. This GitHub page seems to contradict that: https://github.com/WebAssembly/component-model/blob/main/design/high-level/Goals.md https://github.com/WebAssembly/component-model/blob/main/des...

beached_whale 4y ago

I did a test of an ALU/BP(non-vecorized JSON deserialization) heavy library last year and the price of Wasmer/wasm was only about 20-25%. For many things, that's not going to even be noticed.

panic 4y ago

How would (e.g.) strings work in this FFI, considering that wasm doesn’t have a built-in string type or any way to pass variable-length data to a function without allocating space for it in the instance’s memory, and languages have different native string representations (UTF-8 vs UTF-16 vs “UTF-16”)?

tempodox 4y ago
Here's an example of handling strings with the WASI SDK in the browser (no wasmer involved): https://codeberg.org/bit-fu/Browser-WASI https://codeberg.org/bit-fu/Browser-WASI
kevingadd 4y ago
Very poorly considering that it seems like all the people doing wasm FFI work have decided that UTF-16/WTF-16 based language maintainers can pound sand and unless your native string type is UTF-8 your string performance is gonna suck. It's sad since many widely used runtimes (JS, Java, .NET) and one widely used operating system (Windows) all use WTF-16. UTF-8 is great and all but reality needs to be taken into account when drafting specs...
- deleted 4y ago
  [deleted]
garganzol 4y ago
That's a cool question. There are several aspects to that. One is storage and another is encoding. Let's start with storage. C strings, for instance, are terminated by NUL (zero) characters. Are they good enough? You would never answer that question unless you are somehow versed in math. From the mathematical standpoint, it is a mistake to narrow down the space by giving its elements special meaning. Imagine an integer number from -100 to 100. Would you be happy if somebody said you that you cannot use zero because "it has a special meaning"? No, you would be outraged every time you need to use it! (even if it's a rare event). So, from the mathematical standpoint universal strings cannot give special meanings to its characters. Now to the encoding. Is UTF-8 good enough? Or maybe UTF-16? Or something else? How to decide? Let's use math. From the math standpoint, it's always preferable to use an existing concept if it fulfills the need. So, it means that we should use the existing encoding scheme to avoid reinventing the wheel. The next thing the math dictates is that: the most minimal and normalized thing is always preferable to more complex synthetic things provided that they all do the job. This puts UTF-8 encoding at a special place - it's minimal and universal as it can encode the endless number of Unicode symbols (if we, for whatever reason, will decide to colonize the space and consequently add some more billions of characters to Unicode to dial the new cultures and knowledge in). What to do we have in result? UTF-8 string is universal and should be used in a standard component model. But how to encode string length? We already decided that giving special meaning to the NUL characters is a no-go from the math standpoint because it is opinionated and thus informal. The formal approach says that we should store the length of a string alongside the string itself. The beginning of the string is the right place because we want to know string length beforehand. What should we use? Is 32-bit unsigned integer good enough? It surely will be enough for anyone for all times, right? But the math says that a 32-bit unsigned integer can only represent [0, 2^32) range of string lengths. That's not universal because, you know, there may be foreign cultures out there in the universe who have longer strings than that. So what should we use then? The answer is: an unsigned infinite integer number with variable length encoding. We already have a Variable-Length Quantity (VLQ) concept, so let's use just that because it's already invented and universal. What do we have in the final result? The answer may surprise you: UTF-8 encoded Pascal-like immutable strings, where the string lengths are located at the start of the string and encoded with VLQ encoding. They are formally universal and pass the high mark of a standard component model as we may call it some day. At the same time, this way of building strings is quite a departure from what we have now and will have performance implications. That's why we should cut corners to make such level of universality to be practical in the computer world. An obvious measure we can take is to use size_t instead of VLQ-encoded integer for string length representation. The computer memory cannot hold strings larger than that anyway, so this will do the job. The math behind this decision is formal and clear: max_string_size = min(∞, 2 ^ sizeof(size_t)) = 2 ^ sizeof(size_t). This also means that a standard component model should have a notion of size_t-like type which is tied to the memory size of a host architecture. In this way, we make our component model dependent on a host architecture. In return, we get considerable performance benefits while preserving the formal universality which is a win.

ilaksh 4y ago

What I want is to be able to call another web assembly library from inside of one.

kllrnohj 4y ago

WASM is entirely unrelated to that issue. If you want to "solve" cross-language interop & ABI surfaces, you want something like COM.

duped 4y ago
That's exactly what the component model defines, but it looks to be more useful than COM or CORBA. https://github.com/WebAssembly/component-model/blob/main/design/high-level/Goals.md https://github.com/WebAssembly/component-model/blob/main/des...