5 ms·

> I think there's a space for a new project, which takes LuaJIT as a starting point Unfortunately, LuaJIT is tightly-coupled to the Lua language and its code i

by pansa2 6y ago

> I think there's a space for a new project, which takes LuaJIT as a starting point Unfortunately, LuaJIT is tightly-coupled to the Lua language and its code is very complex. I'd be surprised if anyone other than Mike Pall was able to retarget it to support a different language. Also, is JIT compilation really necessary for a scripting language? Projects like the PyPy JIT for Python have seen little adoption, and many platforms (e.g. iOS and game consoles) don't support JIT at all.

jashmatthews 6y ago
Compared to other JIT compilers LuaJIT is very simple. Thomas Schilling forked LuaJIT to run Haskell with what looked like some good results http://files.catwell.info/misc/mirror/tracing-jit-haskell-schilling.pdf http://files.catwell.info/misc/mirror/tracing-jit-haskell-sc...
- Rochus 6y ago
  > Compared to other JIT compilers LuaJIT is very simple. Wow, the understatement of the year.
Rochus 6y ago
> I'd be surprised if anyone other than Mike Pall was able to retarget it to support a different language. Not that difficult. See e.g. https://github.com/rochus-keller/Oberon#lua-source-and-bytecode-generator-features https://github.com/rochus-keller/Oberon#lua-source-and-bytec..., https://github.com/rochus-keller/Smalltalk#a-smalltalk-80-interpreted-virtual-machine-on-luajit https://github.com/rochus-keller/Smalltalk#a-smalltalk-80-in..., or here a whole list of other language frontends: https://github.com/hengestone/lua-languages https://github.com/hengestone/lua-languages
- pansa2 6y ago
  Sorry, that's not quite what I had in mind. I was considering the feasibility of modifying LuaJIT to directly target a different language. The projects you link to all either compile to, or are written in, Lua source code or LuaJIT bytecode. That's less efficient than retargeting the LuaJIT VM. For example, LuaJIT includes bytecode instructions specifically tailored to Lua tables. It looks like these projects build, say, their array data structures on top of those existing instructions, rather than adding array-specific support to LuaJIT.
  - mingodad 6y ago
    Maybe this https://github.com/mingodad/ljs https://github.com/mingodad/ljs and this https://github.com/mingodad/ljsjit https://github.com/mingodad/ljsjit does to some degree what you have in mind ?
  - Rochus 6y ago
    > That's less efficient than retargeting the LuaJIT VM That's like saying LLVM is inefficient because all the frontends compile to LLVM IR. E.g. the Oberon compiler directly generates efficient LuaJIT bytecode also using FFI and even reuses the VM features for it's source-level debugger. Then benchmark written in Oberon has nearly the same performance like compiled to native when it runs on LuaJIT
saagarjha 6y ago
PyPy JIT is unsuccessful because CPython makes it hard for PyPy, not because people don't want it. When there's a need and support for it, we have extremely good JITs spring into existence–JavaScript has at least three, for example.
- sitkack 6y ago
  CPython needs to grow up and not hoard the ecosystem. A successful PyPy is a successful Python.
  - setr 6y ago
    I don't think they're doing anything in particular to hinder PyPy -- the damage was done long back, when things like the C-API exposed too much detail on CPython internals that severely limit PyPy's potential to replace it with more optimal systems/datastructures
    - sitkack 6y ago
      The issue with crowding out alternative Pythons is a socio-political one. The C-API issue is as well, as you said Python extensions are over coupled to the runtime. ctypes and subsequently cffi fixed this 10+ years ago. 1. Decouple the extension mechanism. Use cffi 2. Modularize the batteries Each alternative Python has to re-implement large swaths of the stdlib. The code is the spec in Python. Stackless got the PyPy treatment before PyPy even existed. Besides PyPy, we now have graalpython, rustpython and ... https://github.com/beeware/ouroboros/issues/21 https://github.com/beeware/ouroboros/issues/21 points to a python stdlib I had never heard of, https://github.com/pfalcon/pycopy-lib https://github.com/pfalcon/pycopy-lib The point is, the core CPython devs need stop being so selfish with their toys. The greater Python ecosystem is diverse in spite of their actions not because of it.
    - ksec 6y ago
      It is similar situation in CRuby as well.
nitrogen 6y ago
many platforms (e.g. iOS and game consoles) don't support JIT at all. I was under the impression that LuaJIT has support for at least one game console because it's used in games. Is that not the case?
- pansa2 6y ago
  The LuaJIT project includes both a fast interpreter and a JIT compiler. "Due to restrictions on consoles, the JIT compiler is disabled and only the fast interpreter is built." [0] [0] https://luajit.org/install.html https://luajit.org/install.html
scythe 6y ago
Aside from Javascript: Racket, Guile, Raku, and Ruby all have JIT in the main implementation these days. Even PHP has moved towards JIT compilation.

by pansa2 6y ago

jashmatthews 6y ago

Compared to other JIT compilers LuaJIT is very simple. Thomas Schilling forked LuaJIT to run Haskell with what looked like some good results http://files.catwell.info/misc/mirror/tracing-jit-haskell-schilling.pdf http://files.catwell.info/misc/mirror/tracing-jit-haskell-sc...

Rochus 6y ago
> Compared to other JIT compilers LuaJIT is very simple. Wow, the understatement of the year.

Rochus 6y ago

> I'd be surprised if anyone other than Mike Pall was able to retarget it to support a different language. Not that difficult. See e.g. https://github.com/rochus-keller/Oberon#lua-source-and-bytecode-generator-features https://github.com/rochus-keller/Oberon#lua-source-and-bytec..., https://github.com/rochus-keller/Smalltalk#a-smalltalk-80-interpreted-virtual-machine-on-luajit https://github.com/rochus-keller/Smalltalk#a-smalltalk-80-in..., or here a whole list of other language frontends: https://github.com/hengestone/lua-languages https://github.com/hengestone/lua-languages

pansa2 6y ago
Sorry, that's not quite what I had in mind. I was considering the feasibility of modifying LuaJIT to directly target a different language. The projects you link to all either compile to, or are written in, Lua source code or LuaJIT bytecode. That's less efficient than retargeting the LuaJIT VM. For example, LuaJIT includes bytecode instructions specifically tailored to Lua tables. It looks like these projects build, say, their array data structures on top of those existing instructions, rather than adding array-specific support to LuaJIT.
- mingodad 6y ago
  Maybe this https://github.com/mingodad/ljs https://github.com/mingodad/ljs and this https://github.com/mingodad/ljsjit https://github.com/mingodad/ljsjit does to some degree what you have in mind ?
- Rochus 6y ago
  > That's less efficient than retargeting the LuaJIT VM That's like saying LLVM is inefficient because all the frontends compile to LLVM IR. E.g. the Oberon compiler directly generates efficient LuaJIT bytecode also using FFI and even reuses the VM features for it's source-level debugger. Then benchmark written in Oberon has nearly the same performance like compiled to native when it runs on LuaJIT

saagarjha 6y ago

PyPy JIT is unsuccessful because CPython makes it hard for PyPy, not because people don't want it. When there's a need and support for it, we have extremely good JITs spring into existence–JavaScript has at least three, for example.

sitkack 6y ago
CPython needs to grow up and not hoard the ecosystem. A successful PyPy is a successful Python.
- setr 6y ago
  I don't think they're doing anything in particular to hinder PyPy -- the damage was done long back, when things like the C-API exposed too much detail on CPython internals that severely limit PyPy's potential to replace it with more optimal systems/datastructures
  - sitkack 6y ago
    The issue with crowding out alternative Pythons is a socio-political one. The C-API issue is as well, as you said Python extensions are over coupled to the runtime. ctypes and subsequently cffi fixed this 10+ years ago. 1. Decouple the extension mechanism. Use cffi 2. Modularize the batteries Each alternative Python has to re-implement large swaths of the stdlib. The code is the spec in Python. Stackless got the PyPy treatment before PyPy even existed. Besides PyPy, we now have graalpython, rustpython and ... https://github.com/beeware/ouroboros/issues/21 https://github.com/beeware/ouroboros/issues/21 points to a python stdlib I had never heard of, https://github.com/pfalcon/pycopy-lib https://github.com/pfalcon/pycopy-lib The point is, the core CPython devs need stop being so selfish with their toys. The greater Python ecosystem is diverse in spite of their actions not because of it.
  - ksec 6y ago
    It is similar situation in CRuby as well.

nitrogen 6y ago

many platforms (e.g. iOS and game consoles) don't support JIT at all. I was under the impression that LuaJIT has support for at least one game console because it's used in games. Is that not the case?

pansa2 6y ago
The LuaJIT project includes both a fast interpreter and a JIT compiler. "Due to restrictions on consoles, the JIT compiler is disabled and only the fast interpreter is built." [0] [0] https://luajit.org/install.html https://luajit.org/install.html

scythe 6y ago

Aside from Javascript: Racket, Guile, Raku, and Ruby all have JIT in the main implementation these days. Even PHP has moved towards JIT compilation.