5 ms·

> That idea of a universal/generic VM supporting many languages has been tried many times, with limited success, for example with the JVM, CLR, or Parrot. What

by titzer 3y ago

> That idea of a universal/generic VM supporting many languages has been tried many times, with limited success, for example with the JVM, CLR, or Parrot. What makes this different? I would say that the JVM and CLR were not designed to support many languages. The JVM has one master, which is the Java language, and other languages that want to run on the JVM do so with a burden that increases as the distance from Java's type system increases. Wasm is lower-level than most of the above bytecode formats. Even with Wasm GC, which adds statically-typed structs and arrays, and the function-references proposal which brings typed functions. With Wasm GC, there is also explicit support for tagged pointers (i31ref). All of the above makes Wasm GC more general (by virtue of being lower-level) than the efforts you listed.

mmcdermott 3y ago
JVM wasn't originally intended to support multiple languages. IIRC, Sun wasn't welcoming to new languages on the JVM (Java Virtual Machine). But CLR stands for Common Language Runtime and had three official languages shortly after release (VB.NET, C# and J#, offhand). In fairness, both Sun and Microsoft advertised what they were shooting for in the name.
- orthoxerox 3y ago
  Microsoft planned to support all kinds of languages and the list of CLR languages is respectably long, but most of them can't really keep up with the co-evolution of .NET and C#. The generics in .NET 2.0 were the first big compatibility break, and I think async/await was the second and even bigger one that killed off the bulk of alt-langs.
  - WorldMaker 3y ago
    async/await was never a compatibility break at the CLR level. Most of async/await is "just" the TPL (Task Parallel Library), a subset of the .NET BCL (Base Class Library) and entirely handled as libraries, which any language can use/interact with even without async/await language support. The async/await language support in C# is basically a Monad transformer, building a generator-like asynchronous state machine. (The output of an async/await function in C# is rather similar in concept/implementation to the output of a generator function using yield syntax.) This is also why F# has never had a tough time with async/await because it has even more generic Monad transformer support ("computation expressions"). Any "alt-lang" could make use of the TPL whether or not they could transform their own similar syntax, and there's nothing stopping them from building a similar syntax. As far as I saw that wasn't a killer of languages.
    - orthoxerox 3y ago
      It wasn't a break at the CLR level, but it was a break at the BCL level. To consume the new async methods your language had to come up with a mechanism that was broadly similar to how C# worked or miss out on the vast majority of libraries and the BCL itself.
      - WorldMaker 3y ago
        If a language didn't do anything at all like C# it dealt with a lot of task.ContinueWith(value => {}) callbacks (Action<T>), no different to JS promise.then(value => {}) chaining (truly, no different at all because Task/Promise are basically twins). At worst case you had callback "waterfalls" to deal with, but that was hardly "breaking" and shouldn't have "killed" languages. async/await is much nicer than callbacks/callback waterfalls, but that doesn't mean you can't write callbacks/callback waterfalls (a lot of JS projects certainly did; many still do even despite most browsers supporting async/await today).
        orthoxerox 3y ago
        The language would also need to have some support for anonymous delegates and closures, or its callback waterfalls would have to be broken up into multiple methods that would have to manage the call state themselves. CLS really expected most languages to look a lot of like early C#, that is, Java with some Delphi/VB features grafted on.
- chubot 3y ago
  > But CLR stands for Common Language Runtime and had three official languages shortly after release (VB.NET, C# and J#, offhand). CLR is not really common -- the "tells" are VB.NET and PowerShell. VB6 was a very popular and useful language, and VB.NET broke everything in that ecosystem, turning it into a variant of C# with different syntax. Does that sound like a multi-language VM? If so, why not compile VB6 to it? PowerShell is likewise best described as C# with a horrible shell-like syntax. It's not really a shell in the original sense, because it doesn't talk to the kernel directly -- it runs on a VM, which mediates all access to the kernel. fork() is a syscall that shells rely on critically, which is problematic for almost all VMs. Some notes on "interior" (VM-based) vs "exterior" (OS kernel-based) shells here: https://www.oilshell.org/blog/2023/06/ysh-design.html https://www.oilshell.org/blog/2023/06/ysh-design.html
  - mmcdermott 3y ago
    It was intended to be common whether or not it succeeded. Those examples point to a differing design philosophy at Microsoft rather than some inability to write a language on the CLR. The design philosophy was clearly to do Java-ish typing across the board, supplanting VB6's loose typing and bash's text based mindset. I don't really like the approach. I think shell scripting succeeds because it allows ad-hoc programming and I don't think Java-ish weak but static typing is some perfect ideal to be used everywhere, but you could certainly implement a more loosely typed language on the CLR.
Nelkins 3y ago
> I would say that the JVM and CLR were not designed to support many languages. Hm...CLR stands for Common Language Runtime. I think it was explicitly created to support many languages.
- titzer 3y ago
  It still has C#'s object type system baked into it. It also has reified generics (IMHO a good thing from a performance perspective), which bakes in a particular way of doing parametric polymorphism. If your language's generics work differently than C#'s, you are probably going to have a bad time, and you might have to bring all your own libraries. The CLR's support for functional programming is better than the JVM partly because of the reified generics and partly because Microsoft developed F# and thus had skin in that game and worked hard to make it better. The JVM's support for functional programming has always been weak; invokedynamic and the additional MethodHandle framework reduces boilerplate but is not really the right feature for FP, but a very clunky and heavyweight OO view of FP.
  - jeswin 3y ago
    > It also has reified generics (IMHO a good thing from a performance perspective), which bakes in a particular way of doing parametric polymorphism. Reified generics need additional support, and if you're using type erasure you didn't need any support. What's missing then when it comes to supporting mainstream languages - HKT?
    - titzer 3y ago
      You can't use the .NET standard libraries that have type parameters if you simply erase (which is what I meant by bring your own libraries).
- kaba0 3y ago
  Funnily enough, the CLR barely runs F#, while the JVM’s (which is supposedly was designed as java-only) guest language ecosystem is blooming.
kaba0 3y ago
Maybe sticking to the one true universal bytecode format is already a fallacy — why not just stick to AST interpreters, when they can be made efficient? This is the premise of Graal’s Truffle, and I think they succeed at it quite well.
- luismedel 3y ago
  You'll find this[0] interesting then: The abstract syntax tree is then traversed and encoded into a stream of symbols from the evolving vocabulary. The encoder processes whole subtrees of the abstract syntax tree at a time; these roughly correspond to statements on the level of the source language If I'm not wrong, this technique originated in Oberon and was polished by the paper authors (disciples of Wirth) [0] https://dl.acm.org/doi/pdf/10.1145/265563.265576 https://dl.acm.org/doi/pdf/10.1145/265563.265576
equalsione 3y ago
GraalVM has multiple language implementations that can now run on both GraalVM and OpenJDK. They are typically as fast and sometimes faster than their standard counterparts (e.g. can't find the announcement now but mastodon runs much faster on GraalRuby than on JRuby, and JRuby is pretty fast). I have personal experience with GraalJS being very fast. So I _think_ that you could now say that the JVM can now properly support multiple languages. There are still limitations, mostly related to language libraries that have native bindings so it's not all flowers and chocolates. But still worth considering. But as a general purpose multi-language platform I think you could say it's viable now. But I share the scepticism - is this really the point of WASM? Whether it is or not, it does seem to be the general direction https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/Languages/ https://www.graalvm.org/latest/graalvm-as-a-platform/languag...
ncruces 3y ago
But then WASM GC immediately assumes it can be a moving, compacting GC, with no support for interior pointers. There's a reason Go isn't mentioned in the blog post: the GC is a leaky abstraction.
- saagarjha 3y ago
  Does Go leak observable pointers?
  - tedunangst 3y ago
    People have certainly written unsafe code which uses uintptr in naughty ways which only work without moving gc, but every piece of documentation for such features also tells you not to do it.
  - deleted 3y ago
    [deleted]
  - yencabulator 3y ago
    Intentionally so: that's the only way to cooperate with C and memory-mapped kernel and hardware ABIs. There is an explicit mechanism to pin memory. https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers https://pkg.go.dev/runtime#Pinner https://pkg.go.dev/runtime#Pinner I think it's also worth pointing out that the main Go implementation has a non-moving GC (at this time), and all kinds of fecal matter may impact a rotary ventilation device if and when that changes. Like, are we supposed to pin byte arrays we pass to syscalls? Nobody does that, right now, for short-living calls.

by titzer 3y ago

mmcdermott 3y ago

JVM wasn't originally intended to support multiple languages. IIRC, Sun wasn't welcoming to new languages on the JVM (Java Virtual Machine). But CLR stands for Common Language Runtime and had three official languages shortly after release (VB.NET, C# and J#, offhand). In fairness, both Sun and Microsoft advertised what they were shooting for in the name.

orthoxerox 3y ago
Microsoft planned to support all kinds of languages and the list of CLR languages is respectably long, but most of them can't really keep up with the co-evolution of .NET and C#. The generics in .NET 2.0 were the first big compatibility break, and I think async/await was the second and even bigger one that killed off the bulk of alt-langs.
- WorldMaker 3y ago
  async/await was never a compatibility break at the CLR level. Most of async/await is "just" the TPL (Task Parallel Library), a subset of the .NET BCL (Base Class Library) and entirely handled as libraries, which any language can use/interact with even without async/await language support. The async/await language support in C# is basically a Monad transformer, building a generator-like asynchronous state machine. (The output of an async/await function in C# is rather similar in concept/implementation to the output of a generator function using yield syntax.) This is also why F# has never had a tough time with async/await because it has even more generic Monad transformer support ("computation expressions"). Any "alt-lang" could make use of the TPL whether or not they could transform their own similar syntax, and there's nothing stopping them from building a similar syntax. As far as I saw that wasn't a killer of languages.
  - orthoxerox 3y ago
    It wasn't a break at the CLR level, but it was a break at the BCL level. To consume the new async methods your language had to come up with a mechanism that was broadly similar to how C# worked or miss out on the vast majority of libraries and the BCL itself.
    - WorldMaker 3y ago
      If a language didn't do anything at all like C# it dealt with a lot of task.ContinueWith(value => {}) callbacks (Action<T>), no different to JS promise.then(value => {}) chaining (truly, no different at all because Task/Promise are basically twins). At worst case you had callback "waterfalls" to deal with, but that was hardly "breaking" and shouldn't have "killed" languages. async/await is much nicer than callbacks/callback waterfalls, but that doesn't mean you can't write callbacks/callback waterfalls (a lot of JS projects certainly did; many still do even despite most browsers supporting async/await today).
      - orthoxerox 3y ago
        The language would also need to have some support for anonymous delegates and closures, or its callback waterfalls would have to be broken up into multiple methods that would have to manage the call state themselves. CLS really expected most languages to look a lot of like early C#, that is, Java with some Delphi/VB features grafted on.
chubot 3y ago
> But CLR stands for Common Language Runtime and had three official languages shortly after release (VB.NET, C# and J#, offhand). CLR is not really common -- the "tells" are VB.NET and PowerShell. VB6 was a very popular and useful language, and VB.NET broke everything in that ecosystem, turning it into a variant of C# with different syntax. Does that sound like a multi-language VM? If so, why not compile VB6 to it? PowerShell is likewise best described as C# with a horrible shell-like syntax. It's not really a shell in the original sense, because it doesn't talk to the kernel directly -- it runs on a VM, which mediates all access to the kernel. fork() is a syscall that shells rely on critically, which is problematic for almost all VMs. Some notes on "interior" (VM-based) vs "exterior" (OS kernel-based) shells here: https://www.oilshell.org/blog/2023/06/ysh-design.html https://www.oilshell.org/blog/2023/06/ysh-design.html
- mmcdermott 3y ago
  It was intended to be common whether or not it succeeded. Those examples point to a differing design philosophy at Microsoft rather than some inability to write a language on the CLR. The design philosophy was clearly to do Java-ish typing across the board, supplanting VB6's loose typing and bash's text based mindset. I don't really like the approach. I think shell scripting succeeds because it allows ad-hoc programming and I don't think Java-ish weak but static typing is some perfect ideal to be used everywhere, but you could certainly implement a more loosely typed language on the CLR.

Nelkins 3y ago

> I would say that the JVM and CLR were not designed to support many languages. Hm...CLR stands for Common Language Runtime. I think it was explicitly created to support many languages.

titzer 3y ago
It still has C#'s object type system baked into it. It also has reified generics (IMHO a good thing from a performance perspective), which bakes in a particular way of doing parametric polymorphism. If your language's generics work differently than C#'s, you are probably going to have a bad time, and you might have to bring all your own libraries. The CLR's support for functional programming is better than the JVM partly because of the reified generics and partly because Microsoft developed F# and thus had skin in that game and worked hard to make it better. The JVM's support for functional programming has always been weak; invokedynamic and the additional MethodHandle framework reduces boilerplate but is not really the right feature for FP, but a very clunky and heavyweight OO view of FP.
- jeswin 3y ago
  > It also has reified generics (IMHO a good thing from a performance perspective), which bakes in a particular way of doing parametric polymorphism. Reified generics need additional support, and if you're using type erasure you didn't need any support. What's missing then when it comes to supporting mainstream languages - HKT?
  - titzer 3y ago
    You can't use the .NET standard libraries that have type parameters if you simply erase (which is what I meant by bring your own libraries).
kaba0 3y ago
Funnily enough, the CLR barely runs F#, while the JVM’s (which is supposedly was designed as java-only) guest language ecosystem is blooming.

kaba0 3y ago

Maybe sticking to the one true universal bytecode format is already a fallacy — why not just stick to AST interpreters, when they can be made efficient? This is the premise of Graal’s Truffle, and I think they succeed at it quite well.

luismedel 3y ago
You'll find this[0] interesting then: The abstract syntax tree is then traversed and encoded into a stream of symbols from the evolving vocabulary. The encoder processes whole subtrees of the abstract syntax tree at a time; these roughly correspond to statements on the level of the source language If I'm not wrong, this technique originated in Oberon and was polished by the paper authors (disciples of Wirth) [0] https://dl.acm.org/doi/pdf/10.1145/265563.265576 https://dl.acm.org/doi/pdf/10.1145/265563.265576

equalsione 3y ago

GraalVM has multiple language implementations that can now run on both GraalVM and OpenJDK. They are typically as fast and sometimes faster than their standard counterparts (e.g. can't find the announcement now but mastodon runs much faster on GraalRuby than on JRuby, and JRuby is pretty fast). I have personal experience with GraalJS being very fast. So I _think_ that you could now say that the JVM can now properly support multiple languages. There are still limitations, mostly related to language libraries that have native bindings so it's not all flowers and chocolates. But still worth considering. But as a general purpose multi-language platform I think you could say it's viable now. But I share the scepticism - is this really the point of WASM? Whether it is or not, it does seem to be the general direction https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/Languages/ https://www.graalvm.org/latest/graalvm-as-a-platform/languag...

ncruces 3y ago

But then WASM GC immediately assumes it can be a moving, compacting GC, with no support for interior pointers. There's a reason Go isn't mentioned in the blog post: the GC is a leaky abstraction.

saagarjha 3y ago
Does Go leak observable pointers?
- tedunangst 3y ago
  People have certainly written unsafe code which uses uintptr in naughty ways which only work without moving gc, but every piece of documentation for such features also tells you not to do it.
- deleted 3y ago
  [deleted]
- yencabulator 3y ago
  Intentionally so: that's the only way to cooperate with C and memory-mapped kernel and hardware ABIs. There is an explicit mechanism to pin memory. https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers https://pkg.go.dev/runtime#Pinner https://pkg.go.dev/runtime#Pinner I think it's also worth pointing out that the main Go implementation has a non-moving GC (at this time), and all kinds of fecal matter may impact a rotary ventilation device if and when that changes. Like, are we supposed to pin byte arrays we pass to syscalls? Nobody does that, right now, for short-living calls.