8 ms·

My perception of LuaJIT is that it's Mike Pall's creation (and what a creation at that!). He stepped down a while ago[1] and since then, at least to me, not muc

by sebcat 10y ago

My perception of LuaJIT is that it's Mike Pall's creation (and what a creation at that!). He stepped down a while ago[1] and since then, at least to me, not much seem to have happened with LuaJIT. CloudFlare was in talks of project governance [2], and Cisco has pushed some MIPS support, but I havn't heard much else. There was a beta release in march this year for 2.1, but that's it. The fork between PUC Lua and LuaJIT mentioned in the article is a major point of concern. For good and bad, PUC Lua is not known for backward compatibility. PUC Lua is at 5.3 and LuaJIT targets 5.1. There have been some breaking changes since then. Without a strong community built around LuaJIT, or a corporate backer investing resources in it, I fear that it will die out eventually. I would be hesitant to implement something that uses LuaJIT with the project looking the way it does to me. I would love to be proven wrong though. [1]: https://news.ycombinator.com/item?id=9981802 https://news.ycombinator.com/item?id=9981802 [2]: https://news.ycombinator.com/item?id=10078848 https://news.ycombinator.com/item?id=10078848

jgrahamc 10y ago
We still would like to hire someone to work on LuaJIT. We've found it very, very hard to find anyone suitable. There are few people who understand LuaJIT internals and few who want to get into it. If there is someone reading this who wants to work on LuaJIT full time and thinks they have relevant background then please contact me.
- atemerev 10y ago
  I know a guy, who might be up to a job: https://github.com/wizzard0/js2lua https://github.com/wizzard0/js2lua I'll pass him your offer.
  - etiene 10y ago
    OMG THANK YOU FOR THIS LINK I also worked on a JS2Lua transpiler, which is called exactly the same btw, for a university assignment (https://github.com/Etiene/js2lua https://github.com/Etiene/js2lua) , but it's trash and I never really finished it until it incorporated all native JS functions and was actually usable. I have so many ideas that could use this. This is awesome. Thank you so much!
- lukego 10y ago
  I am confused about why you guys haven't already hired Peter Cawley, Vyacheslav Egorov, and/or Thomas Fransham.
  - corsix 10y ago
    They've explored that route - and at least in my case, I so almost very nearly accepted their offer.
    - lukego 10y ago
      This comment does not clear up my confusion at all... :-)
      - ajdlinux 10y ago
        corsix == Peter Cawley (https://github.com/corsix https://github.com/corsix)
        lukego 10y ago
        Here is what I mean: From the outside the message I am receiving is "hurry up and wait for a new maintainer to emerge from closed-door discussions between Mike, CloudFlare, and unspecified other parties." However, this has been the situation for over a year now, and presumably there have already been fruitless discussions with all of the obvious candidates. (Reading between the lines I suspect that the job description was "Mike Pall 2.0" and the qualified candidates were simply too expensive, but that is speculation.) So what next for LuaJIT users? Keep waiting? Create a fork? Whine on Hacker News? Personally I don't think we need a "Mike Pall 2.0." What we need is mere-mortal active maintainers who can review, merge, and maintain extensions developed by the community. This work could conceivably be spread between many hands -- if some of those hands would have a commit bit. I think it unfortunate that excellent work like Thomas Fransham's intrinsics implementation are stuck in limbo because there is nobody available to review and commit them. End rant. I say this as a lover of LuaJIT and somebody who is using it for the long haul and thus very interested in its ongoing development.
- patrec 10y ago
  Out of curiosity – would you have gotten much better value for money if you had donated more to Luajit whilst Mike was still at the helm? My uninformed outsider's impression is that Mike Pall wasn't really widely successful at getting funding for the work he wanted to do on Luajit (e.g. new GC) – despite being probably the world's foremost JIT for dynamic languages expert[1] and the mileage that many extremely wealthy companies seem to have gotten out of luajit (e.g. I understand it plays a pretty important role in your firewall logic, and probably not one that would be easily replaced with some other technology). I don't mean to single Cloudflare out for criticism, I'm aware that you have in fact funded Mike's work in the past. And maybe there is absolutely no relationship between Mike moving on and funding for luajit. But since you replied here, I'd be genuinely interested to hear your perspective on this: To me it appears that apparently even single-developer open source projects which directly affect the bottomline of large companies can struggle to make enough money to continue their work. Why is this? [1] Seeing how he essentially single handedly wrote a JIT that seems superior in many respects to what the likes of Google have managed to achieve with Javascript after many lifetimes' worth of top developer effort.
  - mikemike 10y ago
    This is a wrong perception. There is/was no shortage of sponsorships. I had to turn down most of these offers, due to time constraints.
- mikepavone 10y ago
  Apart from replying to this thread, what would be the best way for an interested individual to contact you?
  - jgrahamc 10y ago
    Email jgc.
- etiene 10y ago
  I would love to go through the internals of LuaJIT, learn all the necessary things and take over it. This is like my favorite project on earth. Unfortunately I just graduated and don't have the necessary background yet. Since I live in Germany I was almost thinking in just going over to Mike Pall on my next vacations and be like "teach me master" xD
petre 10y ago
LuaJIT is used for NeoVim tests and will be used in the future for writing extensions (in addition to VimScript). The NeoVim developers are planning to implement VimScript itself on top of LuaJIT. This is a bit sad considering the article and your comment. I wish they had picked PUC Lua, but can understand the reason why they went with LuaJIT instead. https://github.com/neovim/neovim/wiki/FAQ#design https://github.com/neovim/neovim/wiki/FAQ#design
- daurnimator 10y ago
  > but can understand the reason why they went with LuaJIT instead. why is that? For a text editor I don't see how ludicrous speed would help... Especially when considering that luajit doesn't compile string operations well; which is what I imagine text editors do a lot of.
  - deleted 10y ago
    [deleted]
  - deleted 10y ago
    [deleted]
  - petre 10y ago
    Probably just because its faster than PUC Lua, enables FFI calls w/ less overhead, easy to interface with the existing C code? They are aiming at reimplementing the current C VimScript runtime (suffering from code rot and filled with preprocessor declarations for dead platforms) in Lua w/o degrading performance too much.
    - justinmk 10y ago
      > reimplementing the current C VimScript runtime See the updated FAQ[1]. We are punting on the VimL-to-Lua translator. We think it will be more valuable to instead ship Lua as a first-class alternative to VimL. > (suffering from code rot and filled with preprocessor declarations for dead platforms) in Lua w/o degrading performance too much eval.c is being refactored in PR 5119[2]. 1: https://github.com/neovim/neovim/wiki/FAQ https://github.com/neovim/neovim/wiki/FAQ 2: https://github.com/neovim/neovim/pull/5119 https://github.com/neovim/neovim/pull/5119
deleted 10y ago
[deleted]
lukego 10y ago
Just one (more) encouraging note: The maintenance challenge that LuaJIT faces is actually kind of trivial compared with other projects like the extremely successful evolution of Spice Lisp to CMU Common Lisp to SBCL over a period of 30 years. I am sure that project seemed like a corpse after Rob McLauchlan moved on in the 90s leaving behind a compiler much more complex than LuaJIT. However, dozens of open source heroes have picked up that project and moved it forward. Kudos to them :). Could be that we in the LuaJIT user community need to take a leaf from their book.
- armitron 10y ago
  SBCL is enormous compared to LuaJIT, true. But most of SBCL is written in Common Lisp which is extremely easy to understand (statically and at runtime). It's also not a JIT compiler (in the deterministic sense). LuaJIT source is terribly, terribly, terribly-written C code with minimal (and opaque) documentation that doesn't really facilitate human understanding. In that regard, LuaJIT __completely fails__ one of the maxims from SICP (code should be written primarily for humans to understand). I therefore propose that this be one of the first issues addressed. The LuaJIT codebase needs to be refactored and extensively documented in such a manner as to facilitate rapid understanding by 3rd parties.
  - camperman 10y ago
    I got the impression during my deep dive that it was elegantly designed and written code that solved problems that were a good two levels above my current experience. I don't think it's the place to go if you want to understand how Lua works or how JITs work. If you have a good working knowledge of those (I'm getting there) then you will be able to appreciate the codebase more.
    - armitron 10y ago
      One should not need good working experience of how Lua works or how JITs work in order to rapidly achieve a basic understanding of the codebase. I am being downvoted because people can't look past personal likes/dislikes and are conditioned to look uncritically at software that they like/depend on. Yet the obvious point I'm making here has nothing to do with the quality of LuaJIT as a tool that can solve problems. There are lots of examples of powerful, high-quality software that sits on an atrocious, impossible-to-understand codebase (e.g. pretty much anything by fabrice bellard, which ironically enough, some even consider as _good_ C) and while LuaJIT may not be at that level, it's not currently _good enough_. I think we need to move past personality-worship and start striving for quality where it matters. Read this thread and you'll find numerous posts referring to Mike Pall as an alien/god-of-coding and so on. This is not at all helpful. A software project that is positioned as something with staying power and a base for others to build on should be judged more on its ability to attract and maintain a community around it rather than its creator/profit that it generates/other superficial attributes. It is for this reason that a well-documented codebase that facilitates understanding is of _paramount importance_.

by sebcat 10y ago

jgrahamc 10y ago

We still would like to hire someone to work on LuaJIT. We've found it very, very hard to find anyone suitable. There are few people who understand LuaJIT internals and few who want to get into it. If there is someone reading this who wants to work on LuaJIT full time and thinks they have relevant background then please contact me.

atemerev 10y ago
I know a guy, who might be up to a job: https://github.com/wizzard0/js2lua https://github.com/wizzard0/js2lua I'll pass him your offer.
- etiene 10y ago
  OMG THANK YOU FOR THIS LINK I also worked on a JS2Lua transpiler, which is called exactly the same btw, for a university assignment (https://github.com/Etiene/js2lua https://github.com/Etiene/js2lua) , but it's trash and I never really finished it until it incorporated all native JS functions and was actually usable. I have so many ideas that could use this. This is awesome. Thank you so much!
lukego 10y ago
I am confused about why you guys haven't already hired Peter Cawley, Vyacheslav Egorov, and/or Thomas Fransham.
- corsix 10y ago
  They've explored that route - and at least in my case, I so almost very nearly accepted their offer.
  - lukego 10y ago
    This comment does not clear up my confusion at all... :-)
    - ajdlinux 10y ago
      corsix == Peter Cawley (https://github.com/corsix https://github.com/corsix)
      - lukego 10y ago
        Here is what I mean: From the outside the message I am receiving is "hurry up and wait for a new maintainer to emerge from closed-door discussions between Mike, CloudFlare, and unspecified other parties." However, this has been the situation for over a year now, and presumably there have already been fruitless discussions with all of the obvious candidates. (Reading between the lines I suspect that the job description was "Mike Pall 2.0" and the qualified candidates were simply too expensive, but that is speculation.) So what next for LuaJIT users? Keep waiting? Create a fork? Whine on Hacker News? Personally I don't think we need a "Mike Pall 2.0." What we need is mere-mortal active maintainers who can review, merge, and maintain extensions developed by the community. This work could conceivably be spread between many hands -- if some of those hands would have a commit bit. I think it unfortunate that excellent work like Thomas Fransham's intrinsics implementation are stuck in limbo because there is nobody available to review and commit them. End rant. I say this as a lover of LuaJIT and somebody who is using it for the long haul and thus very interested in its ongoing development.
patrec 10y ago
Out of curiosity – would you have gotten much better value for money if you had donated more to Luajit whilst Mike was still at the helm? My uninformed outsider's impression is that Mike Pall wasn't really widely successful at getting funding for the work he wanted to do on Luajit (e.g. new GC) – despite being probably the world's foremost JIT for dynamic languages expert[1] and the mileage that many extremely wealthy companies seem to have gotten out of luajit (e.g. I understand it plays a pretty important role in your firewall logic, and probably not one that would be easily replaced with some other technology). I don't mean to single Cloudflare out for criticism, I'm aware that you have in fact funded Mike's work in the past. And maybe there is absolutely no relationship between Mike moving on and funding for luajit. But since you replied here, I'd be genuinely interested to hear your perspective on this: To me it appears that apparently even single-developer open source projects which directly affect the bottomline of large companies can struggle to make enough money to continue their work. Why is this? [1] Seeing how he essentially single handedly wrote a JIT that seems superior in many respects to what the likes of Google have managed to achieve with Javascript after many lifetimes' worth of top developer effort.
- mikemike 10y ago
  This is a wrong perception. There is/was no shortage of sponsorships. I had to turn down most of these offers, due to time constraints.
mikepavone 10y ago
Apart from replying to this thread, what would be the best way for an interested individual to contact you?
- jgrahamc 10y ago
  Email jgc.
etiene 10y ago
I would love to go through the internals of LuaJIT, learn all the necessary things and take over it. This is like my favorite project on earth. Unfortunately I just graduated and don't have the necessary background yet. Since I live in Germany I was almost thinking in just going over to Mike Pall on my next vacations and be like "teach me master" xD

petre 10y ago

LuaJIT is used for NeoVim tests and will be used in the future for writing extensions (in addition to VimScript). The NeoVim developers are planning to implement VimScript itself on top of LuaJIT. This is a bit sad considering the article and your comment. I wish they had picked PUC Lua, but can understand the reason why they went with LuaJIT instead. https://github.com/neovim/neovim/wiki/FAQ#design https://github.com/neovim/neovim/wiki/FAQ#design

daurnimator 10y ago
> but can understand the reason why they went with LuaJIT instead. why is that? For a text editor I don't see how ludicrous speed would help... Especially when considering that luajit doesn't compile string operations well; which is what I imagine text editors do a lot of.
- deleted 10y ago
  [deleted]
- deleted 10y ago
  [deleted]
- petre 10y ago
  Probably just because its faster than PUC Lua, enables FFI calls w/ less overhead, easy to interface with the existing C code? They are aiming at reimplementing the current C VimScript runtime (suffering from code rot and filled with preprocessor declarations for dead platforms) in Lua w/o degrading performance too much.
  - justinmk 10y ago
    > reimplementing the current C VimScript runtime See the updated FAQ[1]. We are punting on the VimL-to-Lua translator. We think it will be more valuable to instead ship Lua as a first-class alternative to VimL. > (suffering from code rot and filled with preprocessor declarations for dead platforms) in Lua w/o degrading performance too much eval.c is being refactored in PR 5119[2]. 1: https://github.com/neovim/neovim/wiki/FAQ https://github.com/neovim/neovim/wiki/FAQ 2: https://github.com/neovim/neovim/pull/5119 https://github.com/neovim/neovim/pull/5119

deleted 10y ago

[deleted]

lukego 10y ago

Just one (more) encouraging note: The maintenance challenge that LuaJIT faces is actually kind of trivial compared with other projects like the extremely successful evolution of Spice Lisp to CMU Common Lisp to SBCL over a period of 30 years. I am sure that project seemed like a corpse after Rob McLauchlan moved on in the 90s leaving behind a compiler much more complex than LuaJIT. However, dozens of open source heroes have picked up that project and moved it forward. Kudos to them :). Could be that we in the LuaJIT user community need to take a leaf from their book.

armitron 10y ago
SBCL is enormous compared to LuaJIT, true. But most of SBCL is written in Common Lisp which is extremely easy to understand (statically and at runtime). It's also not a JIT compiler (in the deterministic sense). LuaJIT source is terribly, terribly, terribly-written C code with minimal (and opaque) documentation that doesn't really facilitate human understanding. In that regard, LuaJIT __completely fails__ one of the maxims from SICP (code should be written primarily for humans to understand). I therefore propose that this be one of the first issues addressed. The LuaJIT codebase needs to be refactored and extensively documented in such a manner as to facilitate rapid understanding by 3rd parties.
- camperman 10y ago
  I got the impression during my deep dive that it was elegantly designed and written code that solved problems that were a good two levels above my current experience. I don't think it's the place to go if you want to understand how Lua works or how JITs work. If you have a good working knowledge of those (I'm getting there) then you will be able to appreciate the codebase more.
  - armitron 10y ago
    One should not need good working experience of how Lua works or how JITs work in order to rapidly achieve a basic understanding of the codebase. I am being downvoted because people can't look past personal likes/dislikes and are conditioned to look uncritically at software that they like/depend on. Yet the obvious point I'm making here has nothing to do with the quality of LuaJIT as a tool that can solve problems. There are lots of examples of powerful, high-quality software that sits on an atrocious, impossible-to-understand codebase (e.g. pretty much anything by fabrice bellard, which ironically enough, some even consider as _good_ C) and while LuaJIT may not be at that level, it's not currently _good enough_. I think we need to move past personality-worship and start striving for quality where it matters. Read this thread and you'll find numerous posts referring to Mike Pall as an alien/god-of-coding and so on. This is not at all helpful. A software project that is positioned as something with staying power and a base for others to build on should be judged more on its ability to attract and maintain a community around it rather than its creator/profit that it generates/other superficial attributes. It is for this reason that a well-documented codebase that facilitates understanding is of _paramount importance_.