5 ms·

Show HN: LuaJIT assembler/assembly language for direct bytecode programming

141 points by Rochus 7y ago

luadigest 7y ago
Would you be interested in writing a short editorial on this for the Lua Digest newsletter? https://luadigest.netlify.app https://luadigest.netlify.app
Rochus 7y ago
Thanks for the suggestion. I think mostly people who want to use LuaJIT as a backend for another language, not primarily Lua programmers, might be interested in my work. Here is an article describing the project: https://medium.com/@rochus.keller/implementing-call-by-reference-and-call-by-name-in-lua-47b9d1003cc2 https://medium.com/@rochus.keller/implementing-call-by-refer...
- deleted 7y ago
  [deleted]
- luadigest 7y ago
  Perfect, I will include it in the next issue.
- SlowRobotAhead 7y ago
  > LuaJIT as a backend for another language, not primarily Lua programmers Not exactly backend but that kinda fits my use case. I have a C embedded application that I need scripting on. Nothing unsecure from the user, from my own backend, but it requires a foreign function interface, and its a super constrained device. Lua, JavaScript, forth, etc. None I’ve been able to find anything I like under 50 kB flash or are even slightly “friendly”. Found one in JavaScript was close, but it’s an interpreted language which means every letter of code is a byte I need to transfer then have parsed. I’ll look into this. Esp if it’s still possible to send just bytecode in.
  - bryanrasmussen 7y ago
    Normally JavaScript is a JIT compiled language because of the need to do stuff like hoisting etc.
    - Stratoscope 7y ago
      Just a nit on the JIT... JavaScript is almost always compiled, regardless of whether there is a JIT or not. A typical JavaScript system first compiles the JS source code into bytecode. Then it starts executing (interpreting) the bytecode. If there is a JIT, that is a second compiler. It monitors the execution of the bytecode to find hot spots where it compiles the bytecode into machine language.
      - bryanrasmussen 7y ago
        yeah but I was just nitpicking on the "interpreted" statement above. What comes around goes around I guess.
  - rafa1981 7y ago
    I don't know if luajit runs in your device, but vanilla luajit supports generating and loading bytecode. https://luajit.org/running.html https://luajit.org/running.html
    - SlowRobotAhead 7y ago
      I think primarily it’s the ffi I would be missing. This would get me a turing complete language but set up as an executable it’s wrong for me.
      - rafa1981 7y ago
        Notice that if I remember correctly, you can generate bytecode on the PC (by using the luajit command) and have the resulting bytecode loaded through the C API on your device as if it were a regular script.
  - 7y ago
- rurban 7y ago
  You are missing https://libraries.io/github/fperrad/tvmjit https://libraries.io/github/fperrad/tvmjit which adds a nice macro and OO layer (in lisp syntax) on top of LuaJIT. A much nicer way to use this superior VM.
  - minxomat 7y ago
    404 on all GitHub links
    - rurban 7y ago
      Yeah, he left github. Maybe you'll get it through shine, which uses tvm. https://github.com/richardhundt/shine https://github.com/richardhundt/shine Or I'll upload a fork.
      - rurban 7y ago
        Here you are: https://github.com/perl11/tvmjit https://github.com/perl11/tvmjit From an older richardhundt/shine source, in its deps. Found it also at https://github.com/richardhundt/tvmjit https://github.com/richardhundt/tvmjit
        Rochus 7y ago
        Thanks, interesting. But so far I don't see such a big advantage over LuaJIT or the Lua language that would justify creating a new, incompatible fork. It's just another variant, not necessarily a better one. Or have I missed something which is so much better than the original?
- microcolonel 7y ago
  Is RaptorJIT abandoned? It seems like they got a couple optimizations, deleted most of the dynasm backends, then gave up.
  - lukego 7y ago
    RaptorJIT is working well for Snabb but it's not clear what other projects can benefit from it. We're using Lua to compete with C while most other projects seem to be using it to compete with Go/Python/node. So for the moment I'm not actively courting new users because I don't find many kindred spirits in the wider LuaJIT community. EDIT: To clarify, the big advantage of RaptorJIT is that it takes dramatically less time and effort to profile and optimize programs to become competitive with C. If you're spending a lot of time scratching your head over long 'jit.dump' reports then RaptorJIT will save you lots of time. That's a big win for Snabb hackers but we seem to spend much more time on JIT-optimization than other LuaJIT users. (Anecdotally it seems like most other LuaJIT users eventually give up on understanding the JIT entirely because they don't see it as being worth the effort. This doesn't really bode well for LuaJIT world domination.)
    - Rochus 7y ago
      I had also a look at RaptorJIT. But as far as I understood it only runs on x86-64, not on x86. Or do I have a misunderstanding?
      - lukego 7y ago
        That's correct.
    - Rochus 7y ago
      Does your fork base on LJ 2.0 or 2.1 beta?
    - microcolonel 7y ago
      I really like the streams and resources you put up, Luke. Thanks for clarifying. Studio is so cool, I've been looking at maybe hacking it to work with my spreadsheet compiler, using Lua (or maybe LJ bytecode) as IR, and source maps. Your live streams are a massive resource. They are so dazzling to others that they'll sit through an explanation of phi nodes, while I assure you they would not otherwise. If you ever want a change of pace, we'd probably be interested in bringing you on board in 2020. Cheers, and I hope the fellas are doing well. :- )
rsrsrs86 7y ago
This is very interesting work! I am poking around CPython and Python bytecode to learn about compilation/VM/interpretation and I was thinking about writing cose directly in Python bytecode to play with. The UI on top looks very handy! Kudos
kzrdude 7y ago
Tangential, but what's LuaJIT's maintenance status?
- faizshah 7y ago
  I was also interested, here's the latest update: https://www.freelists.org/post/luajit/Regular-releases-for-luajit-from-the-new-repo-and-future-roadmap https://www.freelists.org/post/luajit/Regular-releases-for-l... It doesn't look like there's been a release since May 2017 though.

141 points by Rochus 7y ago

Would you be interested in writing a short editorial on this for the Lua Digest newsletter? https://luadigest.netlify.app https://luadigest.netlify.app

Rochus 7y ago

Thanks for the suggestion. I think mostly people who want to use LuaJIT as a backend for another language, not primarily Lua programmers, might be interested in my work. Here is an article describing the project: https://medium.com/@rochus.keller/implementing-call-by-reference-and-call-by-name-in-lua-47b9d1003cc2 https://medium.com/@rochus.keller/implementing-call-by-refer...

deleted 7y ago
[deleted]
luadigest 7y ago
Perfect, I will include it in the next issue.
SlowRobotAhead 7y ago
> LuaJIT as a backend for another language, not primarily Lua programmers Not exactly backend but that kinda fits my use case. I have a C embedded application that I need scripting on. Nothing unsecure from the user, from my own backend, but it requires a foreign function interface, and its a super constrained device. Lua, JavaScript, forth, etc. None I’ve been able to find anything I like under 50 kB flash or are even slightly “friendly”. Found one in JavaScript was close, but it’s an interpreted language which means every letter of code is a byte I need to transfer then have parsed. I’ll look into this. Esp if it’s still possible to send just bytecode in.
- bryanrasmussen 7y ago
  Normally JavaScript is a JIT compiled language because of the need to do stuff like hoisting etc.
  - Stratoscope 7y ago
    Just a nit on the JIT... JavaScript is almost always compiled, regardless of whether there is a JIT or not. A typical JavaScript system first compiles the JS source code into bytecode. Then it starts executing (interpreting) the bytecode. If there is a JIT, that is a second compiler. It monitors the execution of the bytecode to find hot spots where it compiles the bytecode into machine language.
    - bryanrasmussen 7y ago
      yeah but I was just nitpicking on the "interpreted" statement above. What comes around goes around I guess.
- rafa1981 7y ago
  I don't know if luajit runs in your device, but vanilla luajit supports generating and loading bytecode. https://luajit.org/running.html https://luajit.org/running.html
  - SlowRobotAhead 7y ago
    I think primarily it’s the ffi I would be missing. This would get me a turing complete language but set up as an executable it’s wrong for me.
    - rafa1981 7y ago
      Notice that if I remember correctly, you can generate bytecode on the PC (by using the luajit command) and have the resulting bytecode loaded through the C API on your device as if it were a regular script.
- 7y ago
rurban 7y ago
You are missing https://libraries.io/github/fperrad/tvmjit https://libraries.io/github/fperrad/tvmjit which adds a nice macro and OO layer (in lisp syntax) on top of LuaJIT. A much nicer way to use this superior VM.
- minxomat 7y ago
  404 on all GitHub links
  - rurban 7y ago
    Yeah, he left github. Maybe you'll get it through shine, which uses tvm. https://github.com/richardhundt/shine https://github.com/richardhundt/shine Or I'll upload a fork.
    - rurban 7y ago
      Here you are: https://github.com/perl11/tvmjit https://github.com/perl11/tvmjit From an older richardhundt/shine source, in its deps. Found it also at https://github.com/richardhundt/tvmjit https://github.com/richardhundt/tvmjit
      - Rochus 7y ago
        Thanks, interesting. But so far I don't see such a big advantage over LuaJIT or the Lua language that would justify creating a new, incompatible fork. It's just another variant, not necessarily a better one. Or have I missed something which is so much better than the original?
microcolonel 7y ago
Is RaptorJIT abandoned? It seems like they got a couple optimizations, deleted most of the dynasm backends, then gave up.
- lukego 7y ago
  RaptorJIT is working well for Snabb but it's not clear what other projects can benefit from it. We're using Lua to compete with C while most other projects seem to be using it to compete with Go/Python/node. So for the moment I'm not actively courting new users because I don't find many kindred spirits in the wider LuaJIT community. EDIT: To clarify, the big advantage of RaptorJIT is that it takes dramatically less time and effort to profile and optimize programs to become competitive with C. If you're spending a lot of time scratching your head over long 'jit.dump' reports then RaptorJIT will save you lots of time. That's a big win for Snabb hackers but we seem to spend much more time on JIT-optimization than other LuaJIT users. (Anecdotally it seems like most other LuaJIT users eventually give up on understanding the JIT entirely because they don't see it as being worth the effort. This doesn't really bode well for LuaJIT world domination.)
  - Rochus 7y ago
    I had also a look at RaptorJIT. But as far as I understood it only runs on x86-64, not on x86. Or do I have a misunderstanding?
    - lukego 7y ago
      That's correct.
  - Rochus 7y ago
    Does your fork base on LJ 2.0 or 2.1 beta?
  - microcolonel 7y ago
    I really like the streams and resources you put up, Luke. Thanks for clarifying. Studio is so cool, I've been looking at maybe hacking it to work with my spreadsheet compiler, using Lua (or maybe LJ bytecode) as IR, and source maps. Your live streams are a massive resource. They are so dazzling to others that they'll sit through an explanation of phi nodes, while I assure you they would not otherwise. If you ever want a change of pace, we'd probably be interested in bringing you on board in 2020. Cheers, and I hope the fellas are doing well. :- )

rsrsrs86 7y ago

This is very interesting work! I am poking around CPython and Python bytecode to learn about compilation/VM/interpretation and I was thinking about writing cose directly in Python bytecode to play with. The UI on top looks very handy! Kudos

kzrdude 7y ago

Tangential, but what's LuaJIT's maintenance status?

faizshah 7y ago
I was also interested, here's the latest update: https://www.freelists.org/post/luajit/Regular-releases-for-luajit-from-the-new-repo-and-future-roadmap https://www.freelists.org/post/luajit/Regular-releases-for-l... It doesn't look like there's been a release since May 2017 though.