4 ms·

I wish there was something like llvm for scripting languages. Imagine if python, php, javascript, dart or ruby would not interpret the code themself, but compil

by sod 4y ago

I wish there was something like llvm for scripting languages. Imagine if python, php, javascript, dart or ruby would not interpret the code themself, but compile to an interpretable common language, where you could just plug in the fastest interpreter there is for the job.

jacobn 4y ago
JVM and GraalVM? https://www.graalvm.org/ https://www.graalvm.org/
MarkSweep 4y ago
The RPython language PyPy is written in is designed to enable writing JIT-compiled interceptors for any language: https://rpython.readthedocs.io/en/latest/ https://rpython.readthedocs.io/en/latest/ There is also the Dynamic Language Runtime, which is used to implement IronPython and IronRuby on top of .NET: https://en.wikipedia.org/wiki/Dynamic_Language_Runtime https://en.wikipedia.org/wiki/Dynamic_Language_Runtime
- adsharma 4y ago
  rpython is designed to write new interpreters. Not aware of many apps written in the rpython dialect. There are bunch of transpilers which might provide a path from statically typed python3 to native binaries. py2many is one of them. The downside is that all of the C extensions that python3 uses become unusable and need to be rewritten in the subset of python3 that the transpiler accepts.
Aperocky 4y ago
That's why they are called scripting languages.
chickenpotpie 4y ago
Webassembly can fulfill that requirement for certain use cases
- cmrdporcupine 4y ago
  I don't think that was the original intent of WASM nor what its VM is designed for, even if people are now attempting to make it work in this fashion. It was designed for "native-like" execution in the browser (or now other runtimes.) It's more like a VM for something like Forth than something like Python. It's a VM that runs at a much lower level than the ones you'd find inside the interpreter in e.g. Python: No runtime data model beyond primitive types - nothing beyond simple scalar types; no strings, no collections, no maps. No garbage collection, and in fact just a simple "machine-like" linear memory model And in fact this is the point of WASM. You can run compiled C programs in the browser or, increasingly, in other places. Now, are people building stuff like this overtop of WASM? Yes. Are there moves afoot to add GC, etc.? Yes. Personally I'm still getting my feet wet with WebAssembly, so I'm not clear on where these things are at, but I worry that trying to make something like that generic enough to be useful for more than just 1 or 2 target languages could get tricky. Anyways it feels like we've been here before. It's called the JVM or the .NET CLR. I like WASM because it's fundamentally more flexible than the above, and there's good momentum behind it. But I'm wary about positioning it as a "generic high level language VM." You can build a high-level language VM on top of it but right now it's more of a target for portable C/C++/Rust programs.
samuelstros 4y ago
Is that not where WebAssembly is headed? Taken from their website "Wasm is designed as a portable compilation target for programming languages [...]"
- dragonwriter 4y ago
  Not really. Scripting languages implemented on WASM are doing it by compiling their main interpreter to WASM, not the source that the interpreter would execute.
int_19h 4y ago
https://en.wikipedia.org/wiki/Parrot_virtual_machine https://en.wikipedia.org/wiki/Parrot_virtual_machine
dekhn 4y ago
In a sense the xla compiler for tensorflow and other python based machine learning systems is exactly this. And mlir is based on llvm. I predict there will be a second generation of such systems that are more general than mlir and compile large scale computations that work on out of core data into efficient data flow applications that run at native speed
roelschroeven 4y ago
Is Perl's Parrot not meant to be something like that?
- dragonwriter 4y ago
  Parrot [0] was meant to be that and the main VM for Perl6 (now Raku) and failed at both. [0] http://www.parrot.org/ http://www.parrot.org/ states: The Parrot VM is no longer being actively developed. Last commit: 2017-10-02 The role of Parrot as VM for Perl 6 (now "Raku") has been filled by MoarVM, supporting the Rakudo compiler. [...] Parrot, as potential VM for other dynamic languages, never supplanted the existing VMs of those languages.
  - latenightcoding 4y ago
    Raku uses MoarVM
    - dragonwriter 4y ago
      Yes, the quote from the Parrot website I provided says exactly that: the role Parrot intended to fill for Perl6 has been filled by MoarVM supporting the Rakudo compiler.
dragonwriter 4y ago
There have been many attempts to do this, but the problem seems to be that a good target for one scripting language isn't necessarily a good target for another one with a different model, and the interest in a common backend isn't strong enough to overcome that. (Though JS had some potential to do that by the power of it's browser role, before compiling interpreters to WASM became a better way of getting browser support than compiling another scripting language source to JS.)
- cmrdporcupine 4y ago
  Arguably the JVM has been the most successful variant of this type of thing. It's not a bad generic higher level language target overall. It's just it also has sooo much historical baggage and normative lifestyle assumptions along with it, and now it's passed out of favour. And historically support in the browser was crap and done at the wrong level ("applets") and then that became a dead end and was dropped.
staticassertion 4y ago
https://github.com/mypyc/mypyc https://github.com/mypyc/mypyc > Mypyc compiles Python modules to C extensions. It uses standard Python type hints to generate fast code. Mypyc uses mypy to perform type checking and type inference.
- maxloh 4y ago
  You can't compile to an executable with mypyc, just native C extensions.
  - staticassertion 4y ago
    My bad, thanks for clarifying.
IceHegel 4y ago
Is there a reason python hasn't already done this?
NelsonMinar 4y ago
The Java VM has been that in the past with projects like Jython being the bridge. But it's never worked out very well in practice.
deleted 4y ago
[deleted]
Starlevel001 4y ago
This is called the JVM and everyone has spent the last twenty years getting really upset about it.
- mathisonturing 4y ago
  > everyone has spent the last twenty years getting really upset about it Could you expand on this?
nestorD 4y ago
This is already a thing, LLVM has a JIT interface which is used by the Julia interpreter.
rich_sasha 4y ago
The issue is that Python is sort of necessarily slow. Most code translates 1:1 to straightforward C but of course it is possible to create some monstrosity where, after the 3rd time an exception is thrown, all integer arithmetics change and string addition turns into 'eval'. All approaches to "make Python fast again" tend to lock down some of the flexibility so aren't realy general purpose.

by sod 4y ago

jacobn 4y ago

JVM and GraalVM? https://www.graalvm.org/ https://www.graalvm.org/

MarkSweep 4y ago

The RPython language PyPy is written in is designed to enable writing JIT-compiled interceptors for any language: https://rpython.readthedocs.io/en/latest/ https://rpython.readthedocs.io/en/latest/ There is also the Dynamic Language Runtime, which is used to implement IronPython and IronRuby on top of .NET: https://en.wikipedia.org/wiki/Dynamic_Language_Runtime https://en.wikipedia.org/wiki/Dynamic_Language_Runtime

adsharma 4y ago
rpython is designed to write new interpreters. Not aware of many apps written in the rpython dialect. There are bunch of transpilers which might provide a path from statically typed python3 to native binaries. py2many is one of them. The downside is that all of the C extensions that python3 uses become unusable and need to be rewritten in the subset of python3 that the transpiler accepts.

Aperocky 4y ago

That's why they are called scripting languages.

chickenpotpie 4y ago

Webassembly can fulfill that requirement for certain use cases

cmrdporcupine 4y ago
I don't think that was the original intent of WASM nor what its VM is designed for, even if people are now attempting to make it work in this fashion. It was designed for "native-like" execution in the browser (or now other runtimes.) It's more like a VM for something like Forth than something like Python. It's a VM that runs at a much lower level than the ones you'd find inside the interpreter in e.g. Python: No runtime data model beyond primitive types - nothing beyond simple scalar types; no strings, no collections, no maps. No garbage collection, and in fact just a simple "machine-like" linear memory model And in fact this is the point of WASM. You can run compiled C programs in the browser or, increasingly, in other places. Now, are people building stuff like this overtop of WASM? Yes. Are there moves afoot to add GC, etc.? Yes. Personally I'm still getting my feet wet with WebAssembly, so I'm not clear on where these things are at, but I worry that trying to make something like that generic enough to be useful for more than just 1 or 2 target languages could get tricky. Anyways it feels like we've been here before. It's called the JVM or the .NET CLR. I like WASM because it's fundamentally more flexible than the above, and there's good momentum behind it. But I'm wary about positioning it as a "generic high level language VM." You can build a high-level language VM on top of it but right now it's more of a target for portable C/C++/Rust programs.

samuelstros 4y ago

Is that not where WebAssembly is headed? Taken from their website "Wasm is designed as a portable compilation target for programming languages [...]"

dragonwriter 4y ago
Not really. Scripting languages implemented on WASM are doing it by compiling their main interpreter to WASM, not the source that the interpreter would execute.

int_19h 4y ago

https://en.wikipedia.org/wiki/Parrot_virtual_machine https://en.wikipedia.org/wiki/Parrot_virtual_machine

dekhn 4y ago

In a sense the xla compiler for tensorflow and other python based machine learning systems is exactly this. And mlir is based on llvm. I predict there will be a second generation of such systems that are more general than mlir and compile large scale computations that work on out of core data into efficient data flow applications that run at native speed

roelschroeven 4y ago

Is Perl's Parrot not meant to be something like that?

dragonwriter 4y ago
Parrot [0] was meant to be that and the main VM for Perl6 (now Raku) and failed at both. [0] http://www.parrot.org/ http://www.parrot.org/ states: The Parrot VM is no longer being actively developed. Last commit: 2017-10-02 The role of Parrot as VM for Perl 6 (now "Raku") has been filled by MoarVM, supporting the Rakudo compiler. [...] Parrot, as potential VM for other dynamic languages, never supplanted the existing VMs of those languages.
- latenightcoding 4y ago
  Raku uses MoarVM
  - dragonwriter 4y ago
    Yes, the quote from the Parrot website I provided says exactly that: the role Parrot intended to fill for Perl6 has been filled by MoarVM supporting the Rakudo compiler.

dragonwriter 4y ago

There have been many attempts to do this, but the problem seems to be that a good target for one scripting language isn't necessarily a good target for another one with a different model, and the interest in a common backend isn't strong enough to overcome that. (Though JS had some potential to do that by the power of it's browser role, before compiling interpreters to WASM became a better way of getting browser support than compiling another scripting language source to JS.)

cmrdporcupine 4y ago
Arguably the JVM has been the most successful variant of this type of thing. It's not a bad generic higher level language target overall. It's just it also has sooo much historical baggage and normative lifestyle assumptions along with it, and now it's passed out of favour. And historically support in the browser was crap and done at the wrong level ("applets") and then that became a dead end and was dropped.

staticassertion 4y ago

https://github.com/mypyc/mypyc https://github.com/mypyc/mypyc > Mypyc compiles Python modules to C extensions. It uses standard Python type hints to generate fast code. Mypyc uses mypy to perform type checking and type inference.

maxloh 4y ago
You can't compile to an executable with mypyc, just native C extensions.
- staticassertion 4y ago
  My bad, thanks for clarifying.

IceHegel 4y ago

Is there a reason python hasn't already done this?

NelsonMinar 4y ago

The Java VM has been that in the past with projects like Jython being the bridge. But it's never worked out very well in practice.

deleted 4y ago

[deleted]

Starlevel001 4y ago

This is called the JVM and everyone has spent the last twenty years getting really upset about it.

mathisonturing 4y ago
> everyone has spent the last twenty years getting really upset about it Could you expand on this?

nestorD 4y ago

This is already a thing, LLVM has a JIT interface which is used by the Julia interpreter.

rich_sasha 4y ago

The issue is that Python is sort of necessarily slow. Most code translates 1:1 to straightforward C but of course it is possible to create some monstrosity where, after the 3rd time an exception is thrown, all integer arithmetics change and string addition turns into 'eval'. All approaches to "make Python fast again" tend to lock down some of the flexibility so aren't realy general purpose.