5 ms·

Just one (more) encouraging note: The maintenance challenge that LuaJIT faces is actually kind of trivial compared with other projects like the extremely succes

by lukego 10y ago

Just one (more) encouraging note: The maintenance challenge that LuaJIT faces is actually kind of trivial compared with other projects like the extremely successful evolution of Spice Lisp to CMU Common Lisp to SBCL over a period of 30 years. I am sure that project seemed like a corpse after Rob McLauchlan moved on in the 90s leaving behind a compiler much more complex than LuaJIT. However, dozens of open source heroes have picked up that project and moved it forward. Kudos to them :). Could be that we in the LuaJIT user community need to take a leaf from their book.

armitron 10y ago
SBCL is enormous compared to LuaJIT, true. But most of SBCL is written in Common Lisp which is extremely easy to understand (statically and at runtime). It's also not a JIT compiler (in the deterministic sense). LuaJIT source is terribly, terribly, terribly-written C code with minimal (and opaque) documentation that doesn't really facilitate human understanding. In that regard, LuaJIT __completely fails__ one of the maxims from SICP (code should be written primarily for humans to understand). I therefore propose that this be one of the first issues addressed. The LuaJIT codebase needs to be refactored and extensively documented in such a manner as to facilitate rapid understanding by 3rd parties.
- camperman 10y ago
  I got the impression during my deep dive that it was elegantly designed and written code that solved problems that were a good two levels above my current experience. I don't think it's the place to go if you want to understand how Lua works or how JITs work. If you have a good working knowledge of those (I'm getting there) then you will be able to appreciate the codebase more.
  - armitron 10y ago
    One should not need good working experience of how Lua works or how JITs work in order to rapidly achieve a basic understanding of the codebase. I am being downvoted because people can't look past personal likes/dislikes and are conditioned to look uncritically at software that they like/depend on. Yet the obvious point I'm making here has nothing to do with the quality of LuaJIT as a tool that can solve problems. There are lots of examples of powerful, high-quality software that sits on an atrocious, impossible-to-understand codebase (e.g. pretty much anything by fabrice bellard, which ironically enough, some even consider as _good_ C) and while LuaJIT may not be at that level, it's not currently _good enough_. I think we need to move past personality-worship and start striving for quality where it matters. Read this thread and you'll find numerous posts referring to Mike Pall as an alien/god-of-coding and so on. This is not at all helpful. A software project that is positioned as something with staying power and a base for others to build on should be judged more on its ability to attract and maintain a community around it rather than its creator/profit that it generates/other superficial attributes. It is for this reason that a well-documented codebase that facilitates understanding is of _paramount importance_.
    - dream-on 10y ago
      Are there not many projects that cater to what your preferences? Why would you care to comment about this one? Maybe Google, Microsoft, Facebook or some other worshipped employer has a "well-documented codebase" project that "facilitates understanding" for you to work on. Sometimes software written by only one person turns out to be the best software available for the job, because it is the smallest and fastest. No one is forcing anyone to use it. We should be thankful for persons who do this work that they share it with us. On the contrary, there is a cargo cult mentality that pervades the large, popular languages where online discourse continually devolves into attempts to persuade people to use the popular language as a solution for every problem.
    - dang 10y ago
      > I am being downvoted because people can't Please don't go on about downvotes as a rhetorical device. That's a variant of the complaining that the site guidelines ask us all not to do: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.
    - camperman 10y ago
      Good points. Perhaps Luajit does need to be refactored into something more understandable to attract more people if it's going to survive in the long term. The major problem that I see is that we need another Mike Pall but with the added ability of writing code comprehensible by mere mortals.
      - dividuum 10y ago
        I'm not sure you can refactor away the complexity of LuaJIT. It will always end up somewhere as a complete jit'ing compiler with no external dependencies just is inherently a complex beast. What might make sense it to have a more accessible documentation on how LuaJIT works. There are blog posts, mailing list posts and source code comments that probably give an almost complete picture, but there is not central place to find it all.
    - mikemike 10y ago
      No. You DO need a good understanding of a computer language and of JIT compilers to understand the code base for any just-in-time compiler for that computer language. LuaJIT is not a toy compiler from a textbook. There's a lot of inherent complexity in a production compiler that employs advanced optimizations and needs to work on various CPU architectures and operating systems. This reflects in the code.
- PeCaN 10y ago
  Eh? LuaJIT has got to be one of the best pieces of C I've read, and the documentation while scattered (mailing lists and wiki) is pretty solid.
  - lukego 10y ago
    Agreed. LuaJIT is a lovely piece of code and very rewarding to read. The situation is fantastically luxurious compared with CMCUL where the new maintainers needed to acquire a dusty old DEC Alpha to run a non-reproducible binary image with which to write a cross-compiler to x86...
    - PeCaN 10y ago
      That's where SBCL came from?! Wow, they really did a great job with it. I've no idea how they manage to compile a dynamically-typed language so efficiently.
lukego 10y ago
Yet another encouraging note: PostgreSQL is another wildly successful project that was originally developed by some geniuses in Berkeley and then taken over by a community of mere mortals. The core developers seem to view their role as being humble maintainers who try to carry the torch and keep the project moving forward and solving problems without screwing it up too much along the way.

by lukego 10y ago

armitron 10y ago

SBCL is enormous compared to LuaJIT, true. But most of SBCL is written in Common Lisp which is extremely easy to understand (statically and at runtime). It's also not a JIT compiler (in the deterministic sense). LuaJIT source is terribly, terribly, terribly-written C code with minimal (and opaque) documentation that doesn't really facilitate human understanding. In that regard, LuaJIT __completely fails__ one of the maxims from SICP (code should be written primarily for humans to understand). I therefore propose that this be one of the first issues addressed. The LuaJIT codebase needs to be refactored and extensively documented in such a manner as to facilitate rapid understanding by 3rd parties.

camperman 10y ago
I got the impression during my deep dive that it was elegantly designed and written code that solved problems that were a good two levels above my current experience. I don't think it's the place to go if you want to understand how Lua works or how JITs work. If you have a good working knowledge of those (I'm getting there) then you will be able to appreciate the codebase more.
- armitron 10y ago
  One should not need good working experience of how Lua works or how JITs work in order to rapidly achieve a basic understanding of the codebase. I am being downvoted because people can't look past personal likes/dislikes and are conditioned to look uncritically at software that they like/depend on. Yet the obvious point I'm making here has nothing to do with the quality of LuaJIT as a tool that can solve problems. There are lots of examples of powerful, high-quality software that sits on an atrocious, impossible-to-understand codebase (e.g. pretty much anything by fabrice bellard, which ironically enough, some even consider as _good_ C) and while LuaJIT may not be at that level, it's not currently _good enough_. I think we need to move past personality-worship and start striving for quality where it matters. Read this thread and you'll find numerous posts referring to Mike Pall as an alien/god-of-coding and so on. This is not at all helpful. A software project that is positioned as something with staying power and a base for others to build on should be judged more on its ability to attract and maintain a community around it rather than its creator/profit that it generates/other superficial attributes. It is for this reason that a well-documented codebase that facilitates understanding is of _paramount importance_.
  - dream-on 10y ago
    Are there not many projects that cater to what your preferences? Why would you care to comment about this one? Maybe Google, Microsoft, Facebook or some other worshipped employer has a "well-documented codebase" project that "facilitates understanding" for you to work on. Sometimes software written by only one person turns out to be the best software available for the job, because it is the smallest and fastest. No one is forcing anyone to use it. We should be thankful for persons who do this work that they share it with us. On the contrary, there is a cargo cult mentality that pervades the large, popular languages where online discourse continually devolves into attempts to persuade people to use the popular language as a solution for every problem.
  - dang 10y ago
    > I am being downvoted because people can't Please don't go on about downvotes as a rhetorical device. That's a variant of the complaining that the site guidelines ask us all not to do: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.
  - camperman 10y ago
    Good points. Perhaps Luajit does need to be refactored into something more understandable to attract more people if it's going to survive in the long term. The major problem that I see is that we need another Mike Pall but with the added ability of writing code comprehensible by mere mortals.
    - dividuum 10y ago
      I'm not sure you can refactor away the complexity of LuaJIT. It will always end up somewhere as a complete jit'ing compiler with no external dependencies just is inherently a complex beast. What might make sense it to have a more accessible documentation on how LuaJIT works. There are blog posts, mailing list posts and source code comments that probably give an almost complete picture, but there is not central place to find it all.
  - mikemike 10y ago
    No. You DO need a good understanding of a computer language and of JIT compilers to understand the code base for any just-in-time compiler for that computer language. LuaJIT is not a toy compiler from a textbook. There's a lot of inherent complexity in a production compiler that employs advanced optimizations and needs to work on various CPU architectures and operating systems. This reflects in the code.
PeCaN 10y ago
Eh? LuaJIT has got to be one of the best pieces of C I've read, and the documentation while scattered (mailing lists and wiki) is pretty solid.
- lukego 10y ago
  Agreed. LuaJIT is a lovely piece of code and very rewarding to read. The situation is fantastically luxurious compared with CMCUL where the new maintainers needed to acquire a dusty old DEC Alpha to run a non-reproducible binary image with which to write a cross-compiler to x86...
  - PeCaN 10y ago
    That's where SBCL came from?! Wow, they really did a great job with it. I've no idea how they manage to compile a dynamically-typed language so efficiently.

lukego 10y ago

Yet another encouraging note: PostgreSQL is another wildly successful project that was originally developed by some geniuses in Berkeley and then taken over by a community of mere mortals. The core developers seem to view their role as being humble maintainers who try to carry the torch and keep the project moving forward and solving problems without screwing it up too much along the way.