6 ms·

Why Is No One Writing Language Runtimes?

50 points by nkurz 11y ago

dukoid 11y ago
Well... what about WebAssembly?
Russtopia 11y ago
A related point which has always bugged me... the html script tag has a type attribute. The only valid value, to this day, is "text/javascript". Why oh why has no one written a runtime for the major browsers for others, say "text/python", "txt/golang", ... Anything other than javascript? Once an alternative engine were in the major browsers they would hopefully take off as the world's hackers enhanced them.. and we could finally be freed from the shackles of a language designed in a weekend and bandaid-ed ever since.
- petercooper 11y ago
  People have. type="text/vbscript" probably got the most use, but it was only ever in IE. The W3C was gungho on TCL at one point and text/tcl was used there - notably, the HTML 4.01 spec gives an example of setting text/tcl as the default with <META http-equiv="Content-Script-Type" content="text/tcl"> The problem is getting multiple browsers to agree, and no real standard has emerged there. Even if one did, you're either relying on browser creators to include a runtime, or you have to patch one in, Flash-style. I think Google was trying to do this with Dart but failed. A modern workaround I've seen a few times is to write/compile an interpreter in JavaScript that looks for script tags with different types and runs them, but that leads to yet more JS bloat when loading pages.
  - derefr 11y ago
    > Even if one did, you're either relying on browser creators to include a runtime, or you have to patch one in, Flash-style. This would actually be the perfect use-case for Google's Native Client, if other browsers would bother to support it. The arguments against webapp authors writing their app in NaCl are probably valid. But those arguments don't apply to the case where the "apps" are run times, there are only a common ones floating around, and people pull them in by URL in order to script against them. Interestingly, this would solve one additional pain-point: every script would have the particular ABI version of the runtime it was coded against "locked" into it, in the same sense as a Ruby/Node dependency-lock file. So the runtimes themselves could be in very heavy flux version-to-version, as long as they kept old ABI-lines around and patched with security-updates.
- epc 11y ago
  For a very brief time, WebExplorer for OS/2 supported Rexx as a scripting language, but it was limited to OS/2 (which had a Rexx runtime builtin). I don't think it made it to WebExplorer for Windows (and became entirely moot when IBM killed WebExplorer in a deal with Netscape).
- pdkl95 11y ago
  > The only valid value, to this day, is "text/javascript". Other script types have been supported for a while. Some compile-to-javascript languages can be setup to use script tags in their language. For example, CoffeeScript can be setup[1] to compile <script type="text/coffeescript"> tags at runtime. Also, WebGL can load shaders[2] from script tags of type "x-shader/x-vertex" and "x-shader/x-fragment". [1] http://coffeescript.org/#scripts http://coffeescript.org/#scripts [2] see the source of http://mdn.github.io/webgl-examples/tutorial/sample3/ http://mdn.github.io/webgl-examples/tutorial/sample3/ for a basic example.
- deleted 11y ago
  [deleted]
- al2o3cr 11y ago
  There was a some interest in something like that ("text/ruby" support via an applet + JRuby) a while back: http://almaer.com/blog/running-ruby-in-the-browser-via-script-typetextruby http://almaer.com/blog/running-ruby-in-the-browser-via-scrip... I remember hearing about it initially, but it doesn't appear to have gotten much traction.
- ptman 11y ago
  Grail seems to have supported python scripts http://grail.sourceforge.net/ http://grail.sourceforge.net/ https://en.wikipedia.org/wiki/Grail_(web_browser) https://en.wikipedia.org/wiki/Grail_(web_browser)
carsongross 11y ago
It's a great question. I suppose it's a fairly solved problem at this point and very hard to do right, but when I look at what even "simple" bytecode for the JVM looks like... yikes. It would be so nice to have a universal VM that stripped out all the unnecessary non-64 bit data types, cleaned out all the crazy method-overloading related overhead, dropped the security model and added a nice runtime reflection model and, dare I dream, a sane isolation mechanism between modules...
tiles 11y ago
The JVM is making huge strides on becoming a generic, static and dynamic language runtime with the GraalVM prototype: http://www.oracle.com/technetwork/oracle-labs/program-languages/overview/index.html http://www.oracle.com/technetwork/oracle-labs/program-langua...
BWStearns 11y ago
Can someone wiser than myself explain why there seems to be not a lot of support for languages targeting multiple runtimes? I see Clojure has a CLR version that seems to have died on the vine (maybe it didn't and I just haven't investigated enough) but I can't recall seeing more examples. Given the explanation offered int the article, it would seem especially worthwhile for a language's portability to target the LLVM since it's fairly generic. The most compelling reason I can see to not target multiple runtimes is to avoid ecosystem fracture, but it's really not that much overhead to understand a runtime as just another facet of the environment that has to be accounted for.
- xaduha 11y ago
  I asked why there wasn't an implementation of Rakudo for CLR* some time ago, one of the reasons given was that JVM has InvokeDynamic aka indy, while CLR does not. It also has to be said that even their JVM implementation fell behind MoarVM now, so the second and most important reason would be lack of time and manpower. *Yes, I know about Niecza, not the same.
- jerf 11y ago
  Conceptually, it's easy. In practice, it'll drown you in a thousand details, and a lot of those details turn out to be foundational. For example, a slight difference in atomicity guarantees that would be no problem at all to build around from scratch can end up becoming a nightmare to try to port an existing set of runtime expectations on to. Such things tend to subtly but firmly work their way all the way up the stack. JVM's InvokeDynamic is a real-world example of how differences in performance guarantees can also cause surprisingly large differences. There's obviously nothing impossible about it, since it has been done, it's just that it takes a lot more resources than you might have initially thought to do it well. And I think from an economic perspective, the vast bulk of open source projects and similarly-staffed projects are way better off putting that "lot more resources" into becoming better on their native platform than being not-quite-as-good on a lot more platforms. (I had a lengthy defensive discussion of the differences between commercial and open source resource allocation, which grew to be more than half this post, so let's just pretend I proved that I understand the difference, and I still think this paragraph is true even given those differences.)
deleted 11y ago
[deleted]
panic 11y ago
It's worth noting that Python uses bytecode just like Java does. Even the original Ruby implementation created an abstract syntax tree instead of interpreting the text directly. To answer the question in the title, maybe the reason nobody's writing runtimes is that the JVM is just too good? What impact does all the "baggage" have in practice?
- jesserayadkins2 11y ago
  While that first part is true, what's important when trying to target someone's vm is how easily that I can use it. Suppose I want to write a new language that uses Python's vm as the backend. * What files do I need to copy to get just the vm parts? * How do I prepare functions and objects for the vm to use? * What advantages do I get to targeting Python/Ruby/etc's vm? * Will I be able to embed the vm, or is the vm defined in a way that it depends on global data. Lua, for example, wraps everything into a state object. So if I extend Lua (which, if I recall correctly, Terra does), I can provide the same functionality upward. But if the vm relies on global data, then I can't provide an embeddable language that extends off of that.
- _RPM 11y ago
  I was about to say this. Thank you for clearing it up.
- gwright 11y ago
  Just a quick clarification regarding Ruby. All the major Ruby implementations (MRI, JRuby, Rubinius) use virtual machines in their current incarnations.
  - panic 11y ago
    Thanks, I clearly haven't used Ruby for a while. I edited the post to be clearer.
    - steveklabnik 11y ago
      This was one of the largest differences between 1.8 and 1.9, to give context for when this change happened in MRI. During the split, the original interpreter was "MRI" and the bytecode VM was "YARV", but once 1.9 shipped, YARV became MRI, even though it was basically rewritten. Yay naming!
Null-Set 11y ago
You can call methods by name in c on Posix systems. dlopen can give you a handle to your own main program from which you can look up symbols.
- thristian 11y ago
  That depends on executables (such as your own main program) and shared libraries (what dlopen() reads) having the same file format. That's a feature of the ELF executable format commonly used on Linux systems, but isn't necessarily true of other executable formats, like MachO (used on OS X).
qaq 11y ago
Because it requires very substantial investment and provides little benefit?
thristian 11y ago
> Rust programs, however, won’t leak memory because the compiler makes sure you didn’t write code that could leak memory, and then writes down the parts about freeing memory into the final executable; A nitpick: Rust doesn't guarantee you won't leak memory, although it makes it much less likely than in C. Leaking memory is actually officially Safe (in the sense of Rust's safe/unsafe division) since it can't cause a program to access not-yet-allocated memory or already-freed memory.
- vvanders 11y ago
  Yup, there's nothing keeping you from having a few RCs around that create a cycle. That said the language does guide you away from things like that much more than C/C++.
icebraining 11y ago
PyPy is an interesting alternative, since it's not a common language runtime, but a toolkit for building runtimes. Besides Python, other languages for which runtimes have been built using PyPy components (RPython, GC and JIT) are Ruby, PHP, Pixie (a Clojure-like Lisp), Squeak, Prolog and BEAM bytecode.
pcwalton 11y ago
A few points: 1. There is no such thing as a universal runtime. All runtimes are coupled to some language's semantics. The idea of a universal runtime has been tried several times and has always failed, or in the best case been limited to languages that are basically skins on top of the same underlying semantics (e.g. C# and VB.NET). 2. The IRs in the Go compiler are especially strongly coupled to Go semantics. Your language would have to replicate many idiosyncrasies of Go to run on its runtime. For example, you would be unable to benefit from performant lexically scoped cleanup the way e.g. Java (or basically any other language) can, because the Go language "defer" construct has extremely dynamic semantics and there is no support in the compiler for C++-style unwind tables (at least, I don't see why there would be, given that nothing in Go needs them). 3. Go is not a good candidate for this universal runtime in 2016 due its lack of a mature optimization pipeline. Maybe in a few years, but in the meantime you'd be better off using HotSpot or .NET for this purpose.
- panic 11y ago
  Isn't machine code in some sense a bytecode for a universal runtime?
  - vshan 11y ago
    But then again, machine code is platform specific too.
    - Pyxl101 11y ago
      Not necessarily. Have you ever run Linux in a browser on OS X or Windows? http://bellard.org/jslinux/ http://bellard.org/jslinux/
- voidlogic 11y ago
  >3. Go is not a good candidate for this universal runtime in 2016 due its lack of a mature optimization pipeline. Maybe in a few years, but in the meantime you'd be better off using HotSpot or .NET for this purpose. Go 1.7 on amd64 is looking to be pretty competitive (ranging to better than) with HotSpot or .NET in micro-benchmarks. Most of this is due to the new SSA compiler back-end. "Years" might be a pessimistic appraisal.
stonemetal 11y ago
Is this supposed to be like the joke "Nobody drives in New York the streets are too crowded."? If no one is writing them how can you rattle a half dozen runtimes that have achieved various levels of success. If anything you might ask why there are so many rather than getting behind one opensource runtime and pushing it farther rather than starting over all the time.
- jsli 11y ago
  I think the author is saying a generic runtime instead of having each language to create its own.
rectang 11y ago
The Apache Clownfish symbiotic object system is an alternative to the "universal runtime" approach. It is a library rather than a platform. Instead of trying to provide a superset of semantics which encompasses all languages, it provides a single runtime which supports a subset of features, which then lives inside the "host" language environment. https://github.com/apache/lucy-clownfish https://github.com/apache/lucy-clownfish
- iamcreasy 11y ago
  Can you explain it in simpler terms - what is 'symbiotic object system' and how it's an alternate to an universal intermediate code?
DonaldFisk 11y ago
I have written and use my own Lisp, Emblem. It's syntactically similar to Common Lisp, but like Scheme is a Lisp 1. Functions are incrementally compiled into byte code instructions for a virtual stack machine I designed specifically to run Lisp. Compiled functions are added to the Lisp image, instead of being run directly in a Unix shell. The virtual machine is written in C++. Using the read function as the parser and Lisp-specific byte code as the target simplifies the compiler, which is only 1000 lines long. It also reduces the size of the object code. This does come at the expense of speed, but it's fast enough for my requirements.
- Nzen 11y ago
  It looks like you're talking about this. http://web.onetel.com/~hibou/Emblem.html http://web.onetel.com/~hibou/Emblem.html And you started a visual dataflow language on the same runtime http://web.onetel.com/~hibou/Full%20Metal%20Jacket.html http://web.onetel.com/~hibou/Full%20Metal%20Jacket.html
  - DonaldFisk 11y ago
    Yes. That's an old page on Emblem, but the virtual machine is much the same as it was back then, except for improved X11 event handling, and improved 3d graphics. Full Metal Jacket is implemented in Emblem.
mangeletti 11y ago
I think what we're really talking about is, "why aren't there more lower level abstraction layers that are simple and 'open minded'?", and I like the idea of such a thing. The problem is that there isn't really any immediate gain for the organization doing this. The benefit is for the community as a whole. So, we end up in a Tragedy of the Commons-like scenario with disparate pieces that disregard each other. Going down a few turtles, my understanding of RISC is such that it does something similar (providing a much simpler set of possibilities, leaving more to the next level above), but at the machine level. What would 1 or 2 steps up from RISC look like, if it were built with some of the same principles?
dschiptsov 11y ago
Because runtime is hard to understand, leave alone sell. Notably LuaJit, Golang (and before Plan9 and Inferno), Erlang, Haskell, Swift, libc++, etc are writing runtimes. The problem is that it becomes a mess very quickly, unless you are Mike Pall. Also, the idea of a VM isolated from an OS is a marketing meme (how could a mere OS process be isolated from an OS?) So, smallest, thin-layer-over-an-OS runtime - approach pioneered by Inferno, is a much saner and efficient one. Compilation to machine code + thin layer runtime + native ABI based FFI is still the hardest and still the best way to make a runtime since times of Common Lisp. I am not writing runtimes because I am not Mike Pall, not a Google employee and no one pays me to do so.)) BTW, marrying, say, Arc (or femtolisp, or a sane subset of scheme) to LuaJIT runtime could be a nice project, to have a really powerful scripting language.
- bogomipz 11y ago
  Could you elaborate on this sentence, specifically - the native ABI based FFI part? When I think of the ABI I think of the runtime linker of the OS, wouldn't this always be native? What would the non-native ABI be? "Compilation to machine code + thin layer runtime + native ABI based FFI is still the hardest and still the best way to make a runtime since times of Common Lisp." Also what is the thin layer, in this context? I don't know much about Lisp runtimes, I'm sure its fascinating as its both compiled and interpreted. Thanks.
  - dschiptsov 11y ago
    To make FFI lightweight, one follows underlying OS ABI, so there is no need to do any conversions. Compare Golang or LuaJIT FFI with what Java does. Java uses libffi, but it does unnecessary copying and type and encoding conversions. Golang and LuaJIT has zero-copy FFI due to reusing of OS types (for UNIX-like systems it is C) - just dlload-and-call. Thin layer (of abstraction) is how Golang's runtime is organized - delegating to an OS instead of reimplementing inside a VM.
    - bogomipz 11y ago
      Thanks. Makes sense.
bitmapbrother 11y ago
>The JVM is very powerful, but it has its own potential issues (type erasure and garbage collection come to mind). Isn't type erasure one of the very reasons dynamic languages are so plentiful and abundant on the JVM?
- steveklabnik 11y ago
  The addition of invokedynamic helped a lot too.
bogomipz 11y ago
The author states: " The JVM is very powerful, but it has its own potential issues (type erasure and garbage collection come to mind)." Can someone explain why type erasures would be an issue? GC is a loaded subject I know but don't type erasures allow Java to have generics? What is the negative here?
- iamcreasy 11y ago
  Java doesn't have true generics. It uses erasures to emulate this feature. During compilation, template variables are removed and replaced with specific type. I think that's why it's an issue.
  - bogomipz 11y ago
    How are true generics implemented then in other compiled languages?
    - zastrowm 11y ago
      In the CLR/C# for example, List<int> is a distinct type from List<bool>. At runtime, when you create List<int>, the CLR will create a new type based on List<T> that is specialized for int. For List<bool>, a new type based on List<T> is specialized for bool. Then at runtime, you can test variable (if (untypedObj is List<int>)) because there are distinct types for each closed generic type. In java, the compiler would instead (I'm simplifying here), replace List<int> and List<bool> with List<object>, and at runtime, you wouldn't be able to tell a List<int> from List<bool> because their types were "erased" and became List<object>.
      - bogomipz 11y ago
        I see, thanks!
  - bogomipz 11y ago
    How are true generics implemented then in other compiled languages?
rurban 11y ago
> There is a project called Parrot VM that was an attempt to do just this, provide a generic IR format for executing dynamic languages, but it never took off, probably because it was originally tied to the tragic development of Perl 6, and thus didn’t gain traction outside of that community. The parrot story was a bit different. The tragic element was new parrot maintainership who destroyed it, and not perl6. perl6 was always fine and never tragic. And perl6 always kept the lead on board, in contrast to perl5 and parrot, where they left. To the point: A lot of people are writing language runtimes. And some of them even target multiple languages. But no one really cares about pluggable bytecode, as parrot did. rpython the only comparable one, standing against the jvm, clr and llvm. Most people care about an efficient compiler and run-time for their language. And that's what is being done. Even more so in the last decade. Esp. with llvm getting more and more usable for those. Nowadays lua influences a lot of the smaller runtimes, btw. the spot which used to be Scheme/your tiny lisp in the previous decades.
justinlardinois 11y ago
To answer the question posed in the title: because it's hard and the people with those skills are already getting paid to do it. It's also the answer to why there's no free software that's as good as Photoshop. As others have said, the JVM has been a popular runtime target for the last few years. But it's been in constant development for a quarter century, and it powers countless enterprise systems, so the amount of time and resources that have gone into it are enormous. It's hard to get that level of investment in any software, period.

50 points by nkurz 11y ago

Well... what about WebAssembly?

A related point which has always bugged me... the html script tag has a type attribute. The only valid value, to this day, is "text/javascript". Why oh why has no one written a runtime for the major browsers for others, say "text/python", "txt/golang", ... Anything other than javascript? Once an alternative engine were in the major browsers they would hopefully take off as the world's hackers enhanced them.. and we could finally be freed from the shackles of a language designed in a weekend and bandaid-ed ever since.

petercooper 11y ago
People have. type="text/vbscript" probably got the most use, but it was only ever in IE. The W3C was gungho on TCL at one point and text/tcl was used there - notably, the HTML 4.01 spec gives an example of setting text/tcl as the default with <META http-equiv="Content-Script-Type" content="text/tcl"> The problem is getting multiple browsers to agree, and no real standard has emerged there. Even if one did, you're either relying on browser creators to include a runtime, or you have to patch one in, Flash-style. I think Google was trying to do this with Dart but failed. A modern workaround I've seen a few times is to write/compile an interpreter in JavaScript that looks for script tags with different types and runs them, but that leads to yet more JS bloat when loading pages.
- derefr 11y ago
  > Even if one did, you're either relying on browser creators to include a runtime, or you have to patch one in, Flash-style. This would actually be the perfect use-case for Google's Native Client, if other browsers would bother to support it. The arguments against webapp authors writing their app in NaCl are probably valid. But those arguments don't apply to the case where the "apps" are run times, there are only a common ones floating around, and people pull them in by URL in order to script against them. Interestingly, this would solve one additional pain-point: every script would have the particular ABI version of the runtime it was coded against "locked" into it, in the same sense as a Ruby/Node dependency-lock file. So the runtimes themselves could be in very heavy flux version-to-version, as long as they kept old ABI-lines around and patched with security-updates.
epc 11y ago
For a very brief time, WebExplorer for OS/2 supported Rexx as a scripting language, but it was limited to OS/2 (which had a Rexx runtime builtin). I don't think it made it to WebExplorer for Windows (and became entirely moot when IBM killed WebExplorer in a deal with Netscape).
pdkl95 11y ago
> The only valid value, to this day, is "text/javascript". Other script types have been supported for a while. Some compile-to-javascript languages can be setup to use script tags in their language. For example, CoffeeScript can be setup[1] to compile <script type="text/coffeescript"> tags at runtime. Also, WebGL can load shaders[2] from script tags of type "x-shader/x-vertex" and "x-shader/x-fragment". [1] http://coffeescript.org/#scripts http://coffeescript.org/#scripts [2] see the source of http://mdn.github.io/webgl-examples/tutorial/sample3/ http://mdn.github.io/webgl-examples/tutorial/sample3/ for a basic example.
deleted 11y ago
[deleted]
al2o3cr 11y ago
There was a some interest in something like that ("text/ruby" support via an applet + JRuby) a while back: http://almaer.com/blog/running-ruby-in-the-browser-via-script-typetextruby http://almaer.com/blog/running-ruby-in-the-browser-via-scrip... I remember hearing about it initially, but it doesn't appear to have gotten much traction.
ptman 11y ago
Grail seems to have supported python scripts http://grail.sourceforge.net/ http://grail.sourceforge.net/ https://en.wikipedia.org/wiki/Grail_(web_browser) https://en.wikipedia.org/wiki/Grail_(web_browser)

carsongross 11y ago

It's a great question. I suppose it's a fairly solved problem at this point and very hard to do right, but when I look at what even "simple" bytecode for the JVM looks like... yikes. It would be so nice to have a universal VM that stripped out all the unnecessary non-64 bit data types, cleaned out all the crazy method-overloading related overhead, dropped the security model and added a nice runtime reflection model and, dare I dream, a sane isolation mechanism between modules...

tiles 11y ago

The JVM is making huge strides on becoming a generic, static and dynamic language runtime with the GraalVM prototype: http://www.oracle.com/technetwork/oracle-labs/program-languages/overview/index.html http://www.oracle.com/technetwork/oracle-labs/program-langua...

BWStearns 11y ago

Can someone wiser than myself explain why there seems to be not a lot of support for languages targeting multiple runtimes? I see Clojure has a CLR version that seems to have died on the vine (maybe it didn't and I just haven't investigated enough) but I can't recall seeing more examples. Given the explanation offered int the article, it would seem especially worthwhile for a language's portability to target the LLVM since it's fairly generic. The most compelling reason I can see to not target multiple runtimes is to avoid ecosystem fracture, but it's really not that much overhead to understand a runtime as just another facet of the environment that has to be accounted for.

xaduha 11y ago
I asked why there wasn't an implementation of Rakudo for CLR* some time ago, one of the reasons given was that JVM has InvokeDynamic aka indy, while CLR does not. It also has to be said that even their JVM implementation fell behind MoarVM now, so the second and most important reason would be lack of time and manpower. *Yes, I know about Niecza, not the same.
jerf 11y ago
Conceptually, it's easy. In practice, it'll drown you in a thousand details, and a lot of those details turn out to be foundational. For example, a slight difference in atomicity guarantees that would be no problem at all to build around from scratch can end up becoming a nightmare to try to port an existing set of runtime expectations on to. Such things tend to subtly but firmly work their way all the way up the stack. JVM's InvokeDynamic is a real-world example of how differences in performance guarantees can also cause surprisingly large differences. There's obviously nothing impossible about it, since it has been done, it's just that it takes a lot more resources than you might have initially thought to do it well. And I think from an economic perspective, the vast bulk of open source projects and similarly-staffed projects are way better off putting that "lot more resources" into becoming better on their native platform than being not-quite-as-good on a lot more platforms. (I had a lengthy defensive discussion of the differences between commercial and open source resource allocation, which grew to be more than half this post, so let's just pretend I proved that I understand the difference, and I still think this paragraph is true even given those differences.)

deleted 11y ago

[deleted]

panic 11y ago

It's worth noting that Python uses bytecode just like Java does. Even the original Ruby implementation created an abstract syntax tree instead of interpreting the text directly. To answer the question in the title, maybe the reason nobody's writing runtimes is that the JVM is just too good? What impact does all the "baggage" have in practice?

jesserayadkins2 11y ago
While that first part is true, what's important when trying to target someone's vm is how easily that I can use it. Suppose I want to write a new language that uses Python's vm as the backend. * What files do I need to copy to get just the vm parts? * How do I prepare functions and objects for the vm to use? * What advantages do I get to targeting Python/Ruby/etc's vm? * Will I be able to embed the vm, or is the vm defined in a way that it depends on global data. Lua, for example, wraps everything into a state object. So if I extend Lua (which, if I recall correctly, Terra does), I can provide the same functionality upward. But if the vm relies on global data, then I can't provide an embeddable language that extends off of that.
_RPM 11y ago
I was about to say this. Thank you for clearing it up.
gwright 11y ago
Just a quick clarification regarding Ruby. All the major Ruby implementations (MRI, JRuby, Rubinius) use virtual machines in their current incarnations.
- panic 11y ago
  Thanks, I clearly haven't used Ruby for a while. I edited the post to be clearer.
  - steveklabnik 11y ago
    This was one of the largest differences between 1.8 and 1.9, to give context for when this change happened in MRI. During the split, the original interpreter was "MRI" and the bytecode VM was "YARV", but once 1.9 shipped, YARV became MRI, even though it was basically rewritten. Yay naming!

Null-Set 11y ago

You can call methods by name in c on Posix systems. dlopen can give you a handle to your own main program from which you can look up symbols.

thristian 11y ago
That depends on executables (such as your own main program) and shared libraries (what dlopen() reads) having the same file format. That's a feature of the ELF executable format commonly used on Linux systems, but isn't necessarily true of other executable formats, like MachO (used on OS X).

qaq 11y ago

Because it requires very substantial investment and provides little benefit?

thristian 11y ago

> Rust programs, however, won’t leak memory because the compiler makes sure you didn’t write code that could leak memory, and then writes down the parts about freeing memory into the final executable; A nitpick: Rust doesn't guarantee you won't leak memory, although it makes it much less likely than in C. Leaking memory is actually officially Safe (in the sense of Rust's safe/unsafe division) since it can't cause a program to access not-yet-allocated memory or already-freed memory.

vvanders 11y ago
Yup, there's nothing keeping you from having a few RCs around that create a cycle. That said the language does guide you away from things like that much more than C/C++.

icebraining 11y ago

PyPy is an interesting alternative, since it's not a common language runtime, but a toolkit for building runtimes. Besides Python, other languages for which runtimes have been built using PyPy components (RPython, GC and JIT) are Ruby, PHP, Pixie (a Clojure-like Lisp), Squeak, Prolog and BEAM bytecode.

pcwalton 11y ago

A few points: 1. There is no such thing as a universal runtime. All runtimes are coupled to some language's semantics. The idea of a universal runtime has been tried several times and has always failed, or in the best case been limited to languages that are basically skins on top of the same underlying semantics (e.g. C# and VB.NET). 2. The IRs in the Go compiler are especially strongly coupled to Go semantics. Your language would have to replicate many idiosyncrasies of Go to run on its runtime. For example, you would be unable to benefit from performant lexically scoped cleanup the way e.g. Java (or basically any other language) can, because the Go language "defer" construct has extremely dynamic semantics and there is no support in the compiler for C++-style unwind tables (at least, I don't see why there would be, given that nothing in Go needs them). 3. Go is not a good candidate for this universal runtime in 2016 due its lack of a mature optimization pipeline. Maybe in a few years, but in the meantime you'd be better off using HotSpot or .NET for this purpose.

panic 11y ago
Isn't machine code in some sense a bytecode for a universal runtime?
- vshan 11y ago
  But then again, machine code is platform specific too.
  - Pyxl101 11y ago
    Not necessarily. Have you ever run Linux in a browser on OS X or Windows? http://bellard.org/jslinux/ http://bellard.org/jslinux/
voidlogic 11y ago
>3. Go is not a good candidate for this universal runtime in 2016 due its lack of a mature optimization pipeline. Maybe in a few years, but in the meantime you'd be better off using HotSpot or .NET for this purpose. Go 1.7 on amd64 is looking to be pretty competitive (ranging to better than) with HotSpot or .NET in micro-benchmarks. Most of this is due to the new SSA compiler back-end. "Years" might be a pessimistic appraisal.

stonemetal 11y ago

Is this supposed to be like the joke "Nobody drives in New York the streets are too crowded."? If no one is writing them how can you rattle a half dozen runtimes that have achieved various levels of success. If anything you might ask why there are so many rather than getting behind one opensource runtime and pushing it farther rather than starting over all the time.

jsli 11y ago
I think the author is saying a generic runtime instead of having each language to create its own.

rectang 11y ago

The Apache Clownfish symbiotic object system is an alternative to the "universal runtime" approach. It is a library rather than a platform. Instead of trying to provide a superset of semantics which encompasses all languages, it provides a single runtime which supports a subset of features, which then lives inside the "host" language environment. https://github.com/apache/lucy-clownfish https://github.com/apache/lucy-clownfish

iamcreasy 11y ago
Can you explain it in simpler terms - what is 'symbiotic object system' and how it's an alternate to an universal intermediate code?

DonaldFisk 11y ago

I have written and use my own Lisp, Emblem. It's syntactically similar to Common Lisp, but like Scheme is a Lisp 1. Functions are incrementally compiled into byte code instructions for a virtual stack machine I designed specifically to run Lisp. Compiled functions are added to the Lisp image, instead of being run directly in a Unix shell. The virtual machine is written in C++. Using the read function as the parser and Lisp-specific byte code as the target simplifies the compiler, which is only 1000 lines long. It also reduces the size of the object code. This does come at the expense of speed, but it's fast enough for my requirements.

Nzen 11y ago
It looks like you're talking about this. http://web.onetel.com/~hibou/Emblem.html http://web.onetel.com/~hibou/Emblem.html And you started a visual dataflow language on the same runtime http://web.onetel.com/~hibou/Full%20Metal%20Jacket.html http://web.onetel.com/~hibou/Full%20Metal%20Jacket.html
- DonaldFisk 11y ago
  Yes. That's an old page on Emblem, but the virtual machine is much the same as it was back then, except for improved X11 event handling, and improved 3d graphics. Full Metal Jacket is implemented in Emblem.

mangeletti 11y ago

I think what we're really talking about is, "why aren't there more lower level abstraction layers that are simple and 'open minded'?", and I like the idea of such a thing. The problem is that there isn't really any immediate gain for the organization doing this. The benefit is for the community as a whole. So, we end up in a Tragedy of the Commons-like scenario with disparate pieces that disregard each other. Going down a few turtles, my understanding of RISC is such that it does something similar (providing a much simpler set of possibilities, leaving more to the next level above), but at the machine level. What would 1 or 2 steps up from RISC look like, if it were built with some of the same principles?

dschiptsov 11y ago

Because runtime is hard to understand, leave alone sell. Notably LuaJit, Golang (and before Plan9 and Inferno), Erlang, Haskell, Swift, libc++, etc are writing runtimes. The problem is that it becomes a mess very quickly, unless you are Mike Pall. Also, the idea of a VM isolated from an OS is a marketing meme (how could a mere OS process be isolated from an OS?) So, smallest, thin-layer-over-an-OS runtime - approach pioneered by Inferno, is a much saner and efficient one. Compilation to machine code + thin layer runtime + native ABI based FFI is still the hardest and still the best way to make a runtime since times of Common Lisp. I am not writing runtimes because I am not Mike Pall, not a Google employee and no one pays me to do so.)) BTW, marrying, say, Arc (or femtolisp, or a sane subset of scheme) to LuaJIT runtime could be a nice project, to have a really powerful scripting language.

bogomipz 11y ago
Could you elaborate on this sentence, specifically - the native ABI based FFI part? When I think of the ABI I think of the runtime linker of the OS, wouldn't this always be native? What would the non-native ABI be? "Compilation to machine code + thin layer runtime + native ABI based FFI is still the hardest and still the best way to make a runtime since times of Common Lisp." Also what is the thin layer, in this context? I don't know much about Lisp runtimes, I'm sure its fascinating as its both compiled and interpreted. Thanks.
- dschiptsov 11y ago
  To make FFI lightweight, one follows underlying OS ABI, so there is no need to do any conversions. Compare Golang or LuaJIT FFI with what Java does. Java uses libffi, but it does unnecessary copying and type and encoding conversions. Golang and LuaJIT has zero-copy FFI due to reusing of OS types (for UNIX-like systems it is C) - just dlload-and-call. Thin layer (of abstraction) is how Golang's runtime is organized - delegating to an OS instead of reimplementing inside a VM.
  - bogomipz 11y ago
    Thanks. Makes sense.

bitmapbrother 11y ago

>The JVM is very powerful, but it has its own potential issues (type erasure and garbage collection come to mind). Isn't type erasure one of the very reasons dynamic languages are so plentiful and abundant on the JVM?

steveklabnik 11y ago
The addition of invokedynamic helped a lot too.

bogomipz 11y ago

The author states: " The JVM is very powerful, but it has its own potential issues (type erasure and garbage collection come to mind)." Can someone explain why type erasures would be an issue? GC is a loaded subject I know but don't type erasures allow Java to have generics? What is the negative here?

iamcreasy 11y ago
Java doesn't have true generics. It uses erasures to emulate this feature. During compilation, template variables are removed and replaced with specific type. I think that's why it's an issue.
- bogomipz 11y ago
  How are true generics implemented then in other compiled languages?
  - zastrowm 11y ago
    In the CLR/C# for example, List<int> is a distinct type from List<bool>. At runtime, when you create List<int>, the CLR will create a new type based on List<T> that is specialized for int. For List<bool>, a new type based on List<T> is specialized for bool. Then at runtime, you can test variable (if (untypedObj is List<int>)) because there are distinct types for each closed generic type. In java, the compiler would instead (I'm simplifying here), replace List<int> and List<bool> with List<object>, and at runtime, you wouldn't be able to tell a List<int> from List<bool> because their types were "erased" and became List<object>.
    - bogomipz 11y ago
      I see, thanks!
- bogomipz 11y ago
  How are true generics implemented then in other compiled languages?

rurban 11y ago

> There is a project called Parrot VM that was an attempt to do just this, provide a generic IR format for executing dynamic languages, but it never took off, probably because it was originally tied to the tragic development of Perl 6, and thus didn’t gain traction outside of that community. The parrot story was a bit different. The tragic element was new parrot maintainership who destroyed it, and not perl6. perl6 was always fine and never tragic. And perl6 always kept the lead on board, in contrast to perl5 and parrot, where they left. To the point: A lot of people are writing language runtimes. And some of them even target multiple languages. But no one really cares about pluggable bytecode, as parrot did. rpython the only comparable one, standing against the jvm, clr and llvm. Most people care about an efficient compiler and run-time for their language. And that's what is being done. Even more so in the last decade. Esp. with llvm getting more and more usable for those. Nowadays lua influences a lot of the smaller runtimes, btw. the spot which used to be Scheme/your tiny lisp in the previous decades.

justinlardinois 11y ago

To answer the question posed in the title: because it's hard and the people with those skills are already getting paid to do it. It's also the answer to why there's no free software that's as good as Photoshop. As others have said, the JVM has been a popular runtime target for the last few years. But it's been in constant development for a quarter century, and it powers countless enterprise systems, so the amount of time and resources that have gone into it are enormous. It's hard to get that level of investment in any software, period.