13 ms·

> Juice's distribution format ... is based on a tree-shaped program representation as is typically used transiently within optimizing compilers. Rather than con

> Juice's distribution format ... is based on a tree-shaped program representation as is typically used transiently within optimizing compilers. Rather than containing a linear code-sequence that can be interpreted byte-by-byte, a Juice-encoded applet contains a compressed tree that describes the actions of the original program. The tree preserves the control-flow structure of the original program, which makes it much easier to perform code optimization while the tree is translated into the native instruction set Wow, that came out of the left field. Shipping a lexical compiler IR and letting the target do the code generation sound like a very powerful technique compared to compiling for virtual targets like WASM, JVM etc. Wonder why it didn't caught on. Another intriguing tidbit: > By the very definition of our tree-encoding scheme, it is impossible to encode a tree that violates scoping rules of our source language. Such an "illegal" program cannot even be constructed by hand.

ikari_pl 3y ago
> Shipping a lexical compiler IR and letting the target do the code generation sound like a very powerful technique compared to compiling for virtual targets like WASM, JVM. Wonder why it didn't catch on. This is somewhat similar to what V8 does with JIT compilation of javascript by now. I like the idea of doing it on AST rather than direct (heavier) source code. But maybe in the long run, gzipping the JS stream turned out to be just enough?
- vidarh 3y ago
  Gzipping the JS stream is conceptually very different, though. With Juice's approach, the intent is in part that the compiler gets to do any amount of additional work before serializing. What is shipped is already "half compiled", not just serialized as a tree. It's closer to an intermediate representation in tree form than just a representation of the source syntax. Today, with a lot of JS toolchains involving source-to-source transformation first, that could in theory achieve much of the same potential of modifying the original input, and lowering some things to a simpler subset etc., but just compression isn't the same. The size reduction is pretty much a nice bonus with Juice rather than the point - the dictionary encoding matters because it means the decode gets similarity measures "for free" and can generated patchable code segments for partial subtrees that will be subsequently referenced.
vidarh 3y ago
It's a direct continuation of Franz PhD thesis in '94 [1]. (I note I know need to keep my URLs stable, as it turns out one of the references to SDE in that Wikipedia article is my own summary...). I think there are two reasons it didn't catch on: The first obviously being Java, and the sheer amount of mindshare Sun was able to get for Java very quickly. The other is that Oberon had gaps. It's a very austere language. And while all of those gaps can be plugged - Franz himself proposed extensions, as did other students of Wirth - without someone putting in the effort to create an Oberon version with sufficient expansions to appeal to business users, the way Sun did with Java, that the underlying technology was interesting didn't really matter. I'm a big fan of the concept, but it's also more complex in some ways, even though a crude implementation is also potentially very simple. E.g. you "chop the compiler in two", and unlike a bytecode compiler you get a tree to work on. At the same time you can apply other JIT techniques - part of the work on SDE meant reconstituting the dictionary used to compress the tree on loading, and while doing so, if you generate code during the process, you can also directly reference the generated code to use as templates for a patching JIT and insert those templates directly into the dictionary. "One day" I hope to get time to explore SDE more, because it seems JIT's have gotten so far down the bytecode path that we're missing out on seeing what else might be viable. Franz own later research shifted a lot towards Java, but his students have interesting work in the JIT space too. Especially Andreas Gal, and his work on trace trees. [1] https://en.wikipedia.org/wiki/Semantic_dictionary_encoding https://en.wikipedia.org/wiki/Semantic_dictionary_encoding
kgeist 3y ago
Isn't it limiting? Say, the language evolves and its scoping rules change, new control flow features are added etc. Now you can't ship anything in the new version because of the old clients. With bytecode, language implementers have more freedom to change the language without having to force everyone to use a new runtime version. I guess it's not a problem if the language implementers also control the client (say, V8 in Chrome) but in case of Juice, they were just another third-party extension to Netscape, so overall it sounds like a bad idea to me (if I understood it correctly that they serialized and shipped AST to end users).
- vidarh 3y ago
  You can read Franz' PhD dissertation on it here[1]. While it's reasonable to describe it as serialising an AST, most of the Oberon compilers of the time did not have an AST per se, and so what you serialize is entirely arbitrary to your choice of node types, and the operations used by the Oberon version are low-level enough that while some work would have needed to be done to be generic enough if one wanted to more easily support other languages, it's largely moot. There's no large semantic gap between what is serialized and the generated code. At that point it's no different from with Java - if you add new bytecodes, the JVM needs to support it. Same here - if you add new node types, the code generator will need to support it. Just as you wouldn't change the JVM bytecodes for every language change, you likewise wouldn't need to add new node types to support most language changes for Juice either. You could just as well choose to generate what in effect would be an intermediate representation instead. You'd only add new node types if they provide a sufficiently big win over generating more complex output, just as with the JVM That said, nothing would've prevented dynamically updating the code generator. [1] https://oberoncore.ru/_media/library/franz_m.code-generation_on-the-fly_a_key_to_portable_software.en.pdf https://oberoncore.ru/_media/library/franz_m.code-generation...
nickpsecurity 3y ago
It would also make client-side verification easier. You could statically analyze code for safety before running it. You could optimize it much further for your platform. Safe mashups might be easier. Quite a few benefits to this. One more is that, if you used an Oberon-based OS (eg A2 Bluebottle), the system and browser languages were the same. No need to learn separate languages. Integration will be smoother, too.

ikari_pl 3y ago

> Shipping a lexical compiler IR and letting the target do the code generation sound like a very powerful technique compared to compiling for virtual targets like WASM, JVM. Wonder why it didn't catch on. This is somewhat similar to what V8 does with JIT compilation of javascript by now. I like the idea of doing it on AST rather than direct (heavier) source code. But maybe in the long run, gzipping the JS stream turned out to be just enough?

vidarh 3y ago
Gzipping the JS stream is conceptually very different, though. With Juice's approach, the intent is in part that the compiler gets to do any amount of additional work before serializing. What is shipped is already "half compiled", not just serialized as a tree. It's closer to an intermediate representation in tree form than just a representation of the source syntax. Today, with a lot of JS toolchains involving source-to-source transformation first, that could in theory achieve much of the same potential of modifying the original input, and lowering some things to a simpler subset etc., but just compression isn't the same. The size reduction is pretty much a nice bonus with Juice rather than the point - the dictionary encoding matters because it means the decode gets similarity measures "for free" and can generated patchable code segments for partial subtrees that will be subsequently referenced.

vidarh 3y ago

It's a direct continuation of Franz PhD thesis in '94 [1]. (I note I know need to keep my URLs stable, as it turns out one of the references to SDE in that Wikipedia article is my own summary...). I think there are two reasons it didn't catch on: The first obviously being Java, and the sheer amount of mindshare Sun was able to get for Java very quickly. The other is that Oberon had gaps. It's a very austere language. And while all of those gaps can be plugged - Franz himself proposed extensions, as did other students of Wirth - without someone putting in the effort to create an Oberon version with sufficient expansions to appeal to business users, the way Sun did with Java, that the underlying technology was interesting didn't really matter. I'm a big fan of the concept, but it's also more complex in some ways, even though a crude implementation is also potentially very simple. E.g. you "chop the compiler in two", and unlike a bytecode compiler you get a tree to work on. At the same time you can apply other JIT techniques - part of the work on SDE meant reconstituting the dictionary used to compress the tree on loading, and while doing so, if you generate code during the process, you can also directly reference the generated code to use as templates for a patching JIT and insert those templates directly into the dictionary. "One day" I hope to get time to explore SDE more, because it seems JIT's have gotten so far down the bytecode path that we're missing out on seeing what else might be viable. Franz own later research shifted a lot towards Java, but his students have interesting work in the JIT space too. Especially Andreas Gal, and his work on trace trees. [1] https://en.wikipedia.org/wiki/Semantic_dictionary_encoding https://en.wikipedia.org/wiki/Semantic_dictionary_encoding

kgeist 3y ago

Isn't it limiting? Say, the language evolves and its scoping rules change, new control flow features are added etc. Now you can't ship anything in the new version because of the old clients. With bytecode, language implementers have more freedom to change the language without having to force everyone to use a new runtime version. I guess it's not a problem if the language implementers also control the client (say, V8 in Chrome) but in case of Juice, they were just another third-party extension to Netscape, so overall it sounds like a bad idea to me (if I understood it correctly that they serialized and shipped AST to end users).

vidarh 3y ago
You can read Franz' PhD dissertation on it here[1]. While it's reasonable to describe it as serialising an AST, most of the Oberon compilers of the time did not have an AST per se, and so what you serialize is entirely arbitrary to your choice of node types, and the operations used by the Oberon version are low-level enough that while some work would have needed to be done to be generic enough if one wanted to more easily support other languages, it's largely moot. There's no large semantic gap between what is serialized and the generated code. At that point it's no different from with Java - if you add new bytecodes, the JVM needs to support it. Same here - if you add new node types, the code generator will need to support it. Just as you wouldn't change the JVM bytecodes for every language change, you likewise wouldn't need to add new node types to support most language changes for Juice either. You could just as well choose to generate what in effect would be an intermediate representation instead. You'd only add new node types if they provide a sufficiently big win over generating more complex output, just as with the JVM That said, nothing would've prevented dynamically updating the code generator. [1] https://oberoncore.ru/_media/library/franz_m.code-generation_on-the-fly_a_key_to_portable_software.en.pdf https://oberoncore.ru/_media/library/franz_m.code-generation...

nickpsecurity 3y ago

It would also make client-side verification easier. You could statically analyze code for safety before running it. You could optimize it much further for your platform. Safe mashups might be easier. Quite a few benefits to this. One more is that, if you used an Oberon-based OS (eg A2 Bluebottle), the system and browser languages were the same. No need to learn separate languages. Integration will be smoother, too.