11 ms·

GNU lightning: generates assembly language code at run-time

161 points by mpsq 6y ago

zekrioca 6y ago
Documentation was last updated in September, 2019
- brudgers 6y ago
  Stable software doesn't require modification. If a tool is intended to do one thing well, then it won't be constantly getting features. Or to put it another way, users have just been able to use it for almost two years without time spent upgrading to the latest release. Quicksort is still a useful algorithm after nearly sixty years. Merge sort after almost eighty.
  - zekrioca 6y ago
    Sure, I agree. However, it is also a sign that no new features nor capabilities have been added. > Quicksort is still a useful algorithm after nearly sixty years. Merge sort after almost eighty. I am not sure specific algorithms really compare to a whole suit of algorithms and libraries (i.e., a tool) composing very specific versions of a software like GNU lightning, but sure, both Quicksort and Mergesort are useful since their inception. Try installing Windows 3.1 (don't use VMs please) on your current computer and see if we can really live without updating things.. But I'd argue that if we follow your idea, we can say that the democratic concept is also useful since the 18th century. And still, it has to adapt throughout time and be somehow updated (e.g. allowing women and black people to vote or freely use public spaces like toilets and elevators).
    - deleted 6y ago
      [deleted]
  - 1vuio0pswjnm7 6y ago
    Love this comment.
  - vinceguidry 6y ago
    There's an algorithm and then there's it's implementation. So much more goes into implementation. That said, September 2019 is much much better than a lot of software I use.
    - deleted 6y ago
      [deleted]
  - MaxBarraclough 6y ago
    I like the GNU lightning project, but I wouldn't say it's so stable it doesn't need updating. Disassembly doesn't work properly on Ubuntu, which strikes me as a pretty serious problem, but which unfortunately they seem to be in no hurry to fix. [0] I think they'd do well to adopt a proper bug-tracker. Mailing lists are a good way to lose track of known issues. [0] https://lists.gnu.org/archive/html/lightning/2020-06/msg00005.html https://lists.gnu.org/archive/html/lightning/2020-06/msg0000...
    - deleted 6y ago
      [deleted]
    - brudgers 6y ago
      If it doesn’t work on Ubuntu that seems like a fault with Ubuntu. That’s where the breaking change is.
      - MaxBarraclough 6y ago
        I don't think the fault is with Ubuntu. configure scripts are responsible for coping with variations between different distros. lightning's configure script fails to accommodate the way Ubuntu installs the binutils-dev package. It works ok with (Red Hat based) Amazon Linux 2 on x86-64. It failed on Amazon Linux 2 on AArch64 though, with a seemingly unrelated issue, which was the second bug I reported. I don't have the specifics to hand. I also don't know autotools, so I'm not well positioned to write a fix.
        brudgers 6y ago
        Technically, the configure script is responsible in a non-ethical/moral sense. In terms of *nix spirit, it sounds like Ubuntu is not being generous in what it accepts.
        MaxBarraclough 6y ago
        > Technically, the configure script is responsible in a non-ethical/moral sense. I don't follow. Part of the role of a configure script is to accommodate the variations between distributions. If a configure script fails to work with a major distro, that generally means the configure script needs fixing. I don't see that ethics have anything to do with it. > In terms of *nix spirit, it sounds like Ubuntu is not being generous in what it accepts. Ubuntu likely isn't actively doing anything in the process here, it probably isn't in a position to accept or reject anything. Ubuntu presumably places certain files in places the configure script fails to check. This kind of variation between distros is annoying, but typical, and again it's part of the reason we use configure scripts in the first place. (Or modern alternatives like CMake.) If the issue turns out to be that Ubuntu has an issue with pkg-config (a package-discovery system with distro integration) failing to find binutils, then yes, the onus would be on Ubuntu to fix it. I don't think that's the case here though.
  - lifthrasiir 6y ago
    > Quicksort is still a useful algorithm after nearly sixty years. Merge sort after almost eighty. No one uses Quicksort as Hoare originally proposed (modern variants include 3-way quicksort and pattern-defeating quicksort), as much as merge sort as von Neumann originally proposed (modern variants include adaptive merge sort and Timsort). Both were tremendously updated to keep up with the ever-changing computer architecture and requirements.
peter_d_sherman 6y ago
https://www.gnu.org/software/lightning/ https://www.gnu.org/software/lightning/ >"GNU lightning is a library that generates assembly language code at run-time; it is very fast, making it ideal for Just-In-Time compilers, and it abstracts over the target CPU , as it exposes to the clients a standardized RISC instruction set inspired by the MIPS and SPARC chips." [...] >"GNU lightning is usable in complex code generation tasks. The available backends cover the aarch64, alpha, arm, hppa, ia64, mips, powerpc, risc-v, s390, sparc and x86 architectures." https://www.gnu.org/software/lightning/manual/lightning.html https://www.gnu.org/software/lightning/manual/lightning.html >"To be portable, GNU lightning abstracts over current architectures’ quirks and unorthogonalities. The interface that it exposes to is that of a standardized RISC architecture loosely based on the SPARC and MIPS chips. There are a few general-purpose registers (six, not including those used to receive and pass parameters between subroutines), and arithmetic operations involve three operands—either three registers or two registers and an arbitrarily sized immediate value. On one hand, this architecture is general enough that it is possible to generate pretty efficient code even on CISC architectures such as the Intel x86 or the Motorola 68k families. On the other hand, it matches real architectures closely enough that, most of the time, the compiler’s constant folding pass ends up generating code which assembles machine instructions without further tests." [...] >"3 GNU lightning’s instruction set GNU lightning’s instruction set was designed by deriving instructions that closely match those of most existing RISC architectures, or that can be easily syntesized if absent. Each instruction is composed of: an operation, like sub or mul most times, a register/immediate flag (r or i) an unsigned modifier (u), a type identifier or two, when applicable. Examples of legal mnemonics are addr (integer add, with three register operands) and muli (integer multiply, with two register operands and an immediate operand). Each instruction takes two or three operands; in most cases, one of them can be an immediate value instead of a register. Most GNU lightning integer operations are signed wordsize operations, with the exception of operations that convert types, or load or store values to/from memory. When applicable, the types and C types are as follow: _c signed char _uc unsigned char _s short _us unsigned short _i int _ui unsigned int _l long _f float _d double Most integer operations do not need a type modifier, and when loading or storing values to memory there is an alias to the proper operation using wordsize operands, that is, if ommited, the type is int on 32-bit architectures and long on 64-bit architectures. Note that lightning also expects sizeof(void*) to match the wordsize. When an unsigned operation result differs from the equivalent signed operation, there is a the _u modifier. There are at least seven integer registers, of which six are general-purpose, while the last is used to contain the frame pointer (FP). The frame pointer can be used to allocate and access local variables on the stack, using the allocai or allocar instruction. Of the general-purpose registers, at least three are guaranteed to be preserved across function calls (V0, V1 and V2) and at least three are not (R0, R1 and R2). Six registers are not very much, but this restriction was forced by the need to target CISC architectures which, like the x86 [PDS: I think despite this trade-off, that this was a good engineering decision(!), otherwise x86 could not be targeted, and that would destroy a huge swath of functionality!] , are poor of registers; anyway, backends can specify the actual number of available registers with the calls JIT_R_NUM (for caller-save registers) and JIT_V_NUM (for callee-save registers). There are at least six floating-point registers, named F0 to F5. These are usually caller-save and are separate from the integer registers on the supported architectures; on Intel architectures, in 32 bit mode if SSE2 is not available or use of X87 is forced, the register stack is mapped to a flat register file. As for the integer registers, the macro JIT_F_NUM yields the number of floating-point registers. The complete instruction set follows; as you can see, most non-memory operations only take integers (either signed or unsigned) as operands; this was done in order to reduce the instruction set [PDS: Opinion: Any way that the instruction set can be simplified is a good thing!] , and because most architectures only provide word and long word operations on registers. There are instructions that allow operands to be extended to fit a larger data type, both in a signed and in an unsigned way." [...] >"8 Acknowledgements As far as I know, the first general-purpose portable dynamic code generator is DCG, by Dawson R. Engler and T. A. Proebsting. Further work by Dawson R. Engler resulted in the VCODE system; unlike DCG, VCODE used no intermediate representation and directly inspired GNU lightning. Thanks go to Ian Piumarta, who kindly accepted to release his own program CCG under the GNU General Public License, thereby allowing GNU lightning to use the run-time assemblers he had wrote for CCG. CCG provides a way of dynamically assemble programs written in the underlying architecture’s assembly language. So it is not portable, yet very interesting. I also thank Steve Byrne for writing GNU Smalltalk, since GNU lightning was first developed as a tool to be used in GNU Smalltalk’s dynamic translator from bytecodes to native code." [...] PDS: Overall, looks like a great idea! Also, I think that future compiler and VM writers and FPGA soft-CPU authors -- should target this "abstracted instruction set"! Keywords: Instruction Set Architecture, ISA, x86, RISC, RISC-V, Abstraction, Abstract Instruction Set, Instruction Set Subset, Generic Compatibility Layer
riffraff 6y ago
I believe Guile and GNU Smalltalk used to run on lightning, do they still do? What other projects use GNU lightning these days?
- bjoli 6y ago
  Guile rund om a modified version of lightning 1 (called lightening for maximum confusion) due to the recent version being more heavy weight. It is a part of the recent JITted 3.0 release, which brought a lot better performance to the already completely OK 2.2 branch.
- sedachv 6y ago
  GNU CLISP had GNU Lightning added as an option for compiling its VM bytecode in 2008. IMO a bad decision - more maintenance burden; code can now have subtle different bugs whether run in the CLISP interpreter, CLISP bytecode VM, or CLISP bytecode compiled with GNU Lightning; generated code is much larger than bytecode (small memory use was the one unique thing CLISP has over other Common Lisp implementations); performance is still low compared to other Common Lisp compilers. Today libgccjit seems like a much better alternative for this kind of application than GNU Lightning.
snicker7 6y ago
GNU Guile (known primarily for its Scheme implementation) uses a modified version Lightning under the hood.
tyingq 6y ago
There's both a COPYING file that has the GPLv3 in it, and a COPYING.LESSER file with the LGPL in it. I can't tell what's meant by that. Dual licensed? Or portions GPLv3 and portions LGPL?
- e12e 6y ago
  Look at the actual files for copyright (or copyleft, if you will): http://git.savannah.gnu.org/cgit/lightning.git/tree/lib/lightning.c http://git.savannah.gnu.org/cgit/lightning.git/tree/lib/ligh... Looks like the libraryis lgpl,I couldn't find licensefor the docs themselves (I assume they use document license (COPYING.DOC)). There seemsto be at least one example program that is gpl: http://git.savannah.gnu.org/cgit/lightning.git/tree/doc/body.texi#n1671 http://git.savannah.gnu.org/cgit/lightning.git/tree/doc/body...
  - tyingq 6y ago
    Helpful, thanks. Feels like they should summarize it somewhere though, since GPLv3 and LGPL are wildly different.
    - gspr 6y ago
      The text of the LGPL relies on the text of the GPL.
- elvis70 6y ago
  It's under LGPL. The LGPL allows you to relicense your work under the GPL, if you want.
- bonzini 6y ago
  The LGPLv3 is implemented as a set of additional permission on top of the GPLv3. The library is LGPL.
- segfaultbuserr 6y ago
  Since GPLv3, {L,}GPL have been "grand unified" to simplify things. Like code reuse, LGPLv3 is a just small patch on top of GPLv3 that says "given Condition X, you don't have to follow Clause Y in GPLv3", it's literally just a few short paragraphs. Without the original GPLv3 text, LGPLv3 is meaningless, like a running an executable without its shared library. Hence, although many people don't realize it, technically every project that licenses itself under LGPLv3 must contain a copy of GPLv3.
analognoise 6y ago
Too bad it's GPL, is there something like this that's MIT, BSD, or Apache licensed?
- lucian1900 6y ago
  Cranelift is licensed Apache 2 https://github.com/bytecodealliance/wasmtime/tree/main/cranelift https://github.com/bytecodealliance/wasmtime/tree/main/crane...
  - analognoise 6y ago
    Awesome I'll check it out, thank you.
- neolog 6y ago
  If you want to benefit from free software you can share your own software by licensing it GPL.
  - analognoise 6y ago
    The MIT license predates the GPL. Real freedom has been there the whole time.
    - neolog 6y ago
      The MIT license encourages freeriding and impoverishes the free-software community, which reduces overall freedom.
      - analognoise 6y ago
        Weird that you'd say that - given that our license predates the GNU license and the fact that our community is way larger than the GNU one. It's impoverishing the free-software community by... existing for longer and having WAY more people involved in making free-software?
rjeli 6y ago
Why does it only have six registers? The end user probably doesn't want to write a register allocator.
- AlexAltea 6y ago
  Might not be the reason, but six registers make it trivial to write a backend for 32-bit x86 CPUs, since they offer 8 general-purpose registers minus 2 typically used for stack pointer and base (ESP, EBP respectively). That leaves 6 registers usable by compilers: EAX, EDX, ECX, EBX, ESI, EDI. To my knowledge, any other architecture targeted by GNU lightning offers more registers, so this magic number is effectively the largest number of registers one can offer where register allocation would become trivial for all architectures.
- tom_mellior 6y ago
  "Six registers are not very much, but this restriction was forced by the need to target CISC architectures which, like the x86, are poor of registers; anyway, backends can specify the actual number of available registers with the calls JIT_R_NUM (for caller-save registers) and JIT_V_NUM (for callee-save registers)." (https://www.gnu.org/software/lightning/manual/lightning.html#The-instruction-set https://www.gnu.org/software/lightning/manual/lightning.html...) There used to be a boardly similar library called libjit https://www.gnu.org/software/libjit/ https://www.gnu.org/software/libjit/ which was slightly higher level: It offered an arbitrary number of virtual registers. It took care of register allocation, I don't recall how sophisticated it was. The docs call it "global" allocation, i.e., taking the whole function into account at once, as opposed to local allocation that considers blocks in isolation.
  - cout 6y ago
    It wasn't very sophisticated when I used it 12 years ago, but it was "good enough". I know work has been done since then on the register allocator. No idea what the current state is.
dannyobrien 6y ago
Does anyone know how this compares with libgccjit ? What are the difference in their use cases? (I note that Guile picked lightning, and emacs chose libgccjit for instance for what seems to be similar roles.)
- rekado 6y ago
  Guile uses lightening (with an "e"), which is derived from GNU lightning: https://gitlab.com/wingo/lightening https://gitlab.com/wingo/lightening
  - bonzini 6y ago
    ... which is essentially the same as the first version of lightning before the maintainer changed and wrote an optimizer. The first version was literally just a header file emitting constant instruction patterns.
- moonchild 6y ago
  Libgccjit is based on regular gcc, which is a heavyweight compiler. The 'jit' is somewhat of a misnomer; you wouldn't want to actually use that for any of the same cases as a traditional jit (e.g. browsers). (LLVM jit has the same problems.) I think the main point of libgccjit is actually to provide a cleaner interface than GIMPLE to gcc.
  - sedachv 6y ago
    > you wouldn't want to actually use that for any of the same cases as a traditional jit Well the most high-profile project using libgccjit right now is Elisp (feature/native-comp GNU Emacs branch, which by all indications will eventually make it into the official release), and using GCC as a JIT compiler is how Embeddable Common Lisp has always worked.
    - aidenn0 6y ago
      I don't think ECL uses GCC as a JIT; most common lisps use an incremental AOT compiler, which is different. ECL also has an interpreter, but I'm pretty sure it doesn't use GCC.
orf 6y ago
Has had 6 commits in the last two years. From the same person. 2.1.0 was released 6 years ago, the latest 2.1.3 release came out two years ago. Releases are available from some random FTP server. On one hand this could be software that is completely done and requires no more maintenance. On the other hand it could be another random half-baked GNU project with a single champion who has lost interest but is still saddled with a purist/terrible new contributor onboarding experience dooms it to an inevitable grave. You can be the judge of that.
- rekado 6y ago
  "Random FTP server"? You mean the GNU FTP server?
  - orf 6y ago
    You speak of that like it’s some bastion of software delivery. You mean THE gnu ftp server?? It’s a random FTP server. Could be sourceforge for it all it matters. Doesn’t even support SFTP.
    - rekado 6y ago
      If you look closely you'll see that ftp.gnu.org is an HTTP server. But it seems that you just want to troll.
      - orf 6y ago
        And if you look closer you’ll see that it’s a FTP server with a HTTP interface (http, not https, by default mind you). ftp://ftp.gnu.org/
        mcosta 6y ago
        The link is http, https also works. Even if it is only http, so what? PGP signatures exists too.
        spicybright 6y ago
kwdc 6y ago
There was a similar JIT library written in C mentioned on HN awhile ago. Wasnt libjit or lightning. I forget what it was called. Seemed a lot simpler.
- cout 6y ago
  I'm not sure how one could get much simpler than libjit. The first time I read the documentation I had a working jit for a toy language five minutes later, and less than a week after that I was jit-compiling Ruby code (not the full language, obviously, just a subset). It really is a pleasure to use, since the API is based on normal C functions instead of macros.
  - kwdc 6y ago
    Simpler doesn't imoly more featured; as I recall that simpler approach had more rough edges. Still haven't found it.

161 points by mpsq 6y ago

zekrioca 6y ago

Documentation was last updated in September, 2019

brudgers 6y ago
Stable software doesn't require modification. If a tool is intended to do one thing well, then it won't be constantly getting features. Or to put it another way, users have just been able to use it for almost two years without time spent upgrading to the latest release. Quicksort is still a useful algorithm after nearly sixty years. Merge sort after almost eighty.
- zekrioca 6y ago
  Sure, I agree. However, it is also a sign that no new features nor capabilities have been added. > Quicksort is still a useful algorithm after nearly sixty years. Merge sort after almost eighty. I am not sure specific algorithms really compare to a whole suit of algorithms and libraries (i.e., a tool) composing very specific versions of a software like GNU lightning, but sure, both Quicksort and Mergesort are useful since their inception. Try installing Windows 3.1 (don't use VMs please) on your current computer and see if we can really live without updating things.. But I'd argue that if we follow your idea, we can say that the democratic concept is also useful since the 18th century. And still, it has to adapt throughout time and be somehow updated (e.g. allowing women and black people to vote or freely use public spaces like toilets and elevators).
  - deleted 6y ago
    [deleted]
- 1vuio0pswjnm7 6y ago
  Love this comment.
- vinceguidry 6y ago
  There's an algorithm and then there's it's implementation. So much more goes into implementation. That said, September 2019 is much much better than a lot of software I use.
  - deleted 6y ago
    [deleted]
- MaxBarraclough 6y ago
  I like the GNU lightning project, but I wouldn't say it's so stable it doesn't need updating. Disassembly doesn't work properly on Ubuntu, which strikes me as a pretty serious problem, but which unfortunately they seem to be in no hurry to fix. [0] I think they'd do well to adopt a proper bug-tracker. Mailing lists are a good way to lose track of known issues. [0] https://lists.gnu.org/archive/html/lightning/2020-06/msg00005.html https://lists.gnu.org/archive/html/lightning/2020-06/msg0000...
  - deleted 6y ago
    [deleted]
  - brudgers 6y ago
    If it doesn’t work on Ubuntu that seems like a fault with Ubuntu. That’s where the breaking change is.
    - MaxBarraclough 6y ago
      I don't think the fault is with Ubuntu. configure scripts are responsible for coping with variations between different distros. lightning's configure script fails to accommodate the way Ubuntu installs the binutils-dev package. It works ok with (Red Hat based) Amazon Linux 2 on x86-64. It failed on Amazon Linux 2 on AArch64 though, with a seemingly unrelated issue, which was the second bug I reported. I don't have the specifics to hand. I also don't know autotools, so I'm not well positioned to write a fix.
      - brudgers 6y ago
        Technically, the configure script is responsible in a non-ethical/moral sense. In terms of *nix spirit, it sounds like Ubuntu is not being generous in what it accepts.
        MaxBarraclough 6y ago
        > Technically, the configure script is responsible in a non-ethical/moral sense. I don't follow. Part of the role of a configure script is to accommodate the variations between distributions. If a configure script fails to work with a major distro, that generally means the configure script needs fixing. I don't see that ethics have anything to do with it. > In terms of *nix spirit, it sounds like Ubuntu is not being generous in what it accepts. Ubuntu likely isn't actively doing anything in the process here, it probably isn't in a position to accept or reject anything. Ubuntu presumably places certain files in places the configure script fails to check. This kind of variation between distros is annoying, but typical, and again it's part of the reason we use configure scripts in the first place. (Or modern alternatives like CMake.) If the issue turns out to be that Ubuntu has an issue with pkg-config (a package-discovery system with distro integration) failing to find binutils, then yes, the onus would be on Ubuntu to fix it. I don't think that's the case here though.
- lifthrasiir 6y ago
  > Quicksort is still a useful algorithm after nearly sixty years. Merge sort after almost eighty. No one uses Quicksort as Hoare originally proposed (modern variants include 3-way quicksort and pattern-defeating quicksort), as much as merge sort as von Neumann originally proposed (modern variants include adaptive merge sort and Timsort). Both were tremendously updated to keep up with the ever-changing computer architecture and requirements.

peter_d_sherman 6y ago

https://www.gnu.org/software/lightning/ https://www.gnu.org/software/lightning/ >"GNU lightning is a library that generates assembly language code at run-time; it is very fast, making it ideal for Just-In-Time compilers, and it abstracts over the target CPU , as it exposes to the clients a standardized RISC instruction set inspired by the MIPS and SPARC chips." [...] >"GNU lightning is usable in complex code generation tasks. The available backends cover the aarch64, alpha, arm, hppa, ia64, mips, powerpc, risc-v, s390, sparc and x86 architectures." https://www.gnu.org/software/lightning/manual/lightning.html https://www.gnu.org/software/lightning/manual/lightning.html >"To be portable, GNU lightning abstracts over current architectures’ quirks and unorthogonalities. The interface that it exposes to is that of a standardized RISC architecture loosely based on the SPARC and MIPS chips. There are a few general-purpose registers (six, not including those used to receive and pass parameters between subroutines), and arithmetic operations involve three operands—either three registers or two registers and an arbitrarily sized immediate value. On one hand, this architecture is general enough that it is possible to generate pretty efficient code even on CISC architectures such as the Intel x86 or the Motorola 68k families. On the other hand, it matches real architectures closely enough that, most of the time, the compiler’s constant folding pass ends up generating code which assembles machine instructions without further tests." [...] >"3 GNU lightning’s instruction set GNU lightning’s instruction set was designed by deriving instructions that closely match those of most existing RISC architectures, or that can be easily syntesized if absent. Each instruction is composed of: an operation, like sub or mul most times, a register/immediate flag (r or i) an unsigned modifier (u), a type identifier or two, when applicable. Examples of legal mnemonics are addr (integer add, with three register operands) and muli (integer multiply, with two register operands and an immediate operand). Each instruction takes two or three operands; in most cases, one of them can be an immediate value instead of a register. Most GNU lightning integer operations are signed wordsize operations, with the exception of operations that convert types, or load or store values to/from memory. When applicable, the types and C types are as follow: _c signed char _uc unsigned char _s short _us unsigned short _i int _ui unsigned int _l long _f float _d double Most integer operations do not need a type modifier, and when loading or storing values to memory there is an alias to the proper operation using wordsize operands, that is, if ommited, the type is int on 32-bit architectures and long on 64-bit architectures. Note that lightning also expects sizeof(void*) to match the wordsize. When an unsigned operation result differs from the equivalent signed operation, there is a the _u modifier. There are at least seven integer registers, of which six are general-purpose, while the last is used to contain the frame pointer (FP). The frame pointer can be used to allocate and access local variables on the stack, using the allocai or allocar instruction. Of the general-purpose registers, at least three are guaranteed to be preserved across function calls (V0, V1 and V2) and at least three are not (R0, R1 and R2). Six registers are not very much, but this restriction was forced by the need to target CISC architectures which, like the x86 [PDS: I think despite this trade-off, that this was a good engineering decision(!), otherwise x86 could not be targeted, and that would destroy a huge swath of functionality!] , are poor of registers; anyway, backends can specify the actual number of available registers with the calls JIT_R_NUM (for caller-save registers) and JIT_V_NUM (for callee-save registers). There are at least six floating-point registers, named F0 to F5. These are usually caller-save and are separate from the integer registers on the supported architectures; on Intel architectures, in 32 bit mode if SSE2 is not available or use of X87 is forced, the register stack is mapped to a flat register file. As for the integer registers, the macro JIT_F_NUM yields the number of floating-point registers. The complete instruction set follows; as you can see, most non-memory operations only take integers (either signed or unsigned) as operands; this was done in order to reduce the instruction set [PDS: Opinion: Any way that the instruction set can be simplified is a good thing!] , and because most architectures only provide word and long word operations on registers. There are instructions that allow operands to be extended to fit a larger data type, both in a signed and in an unsigned way." [...] >"8 Acknowledgements As far as I know, the first general-purpose portable dynamic code generator is DCG, by Dawson R. Engler and T. A. Proebsting. Further work by Dawson R. Engler resulted in the VCODE system; unlike DCG, VCODE used no intermediate representation and directly inspired GNU lightning. Thanks go to Ian Piumarta, who kindly accepted to release his own program CCG under the GNU General Public License, thereby allowing GNU lightning to use the run-time assemblers he had wrote for CCG. CCG provides a way of dynamically assemble programs written in the underlying architecture’s assembly language. So it is not portable, yet very interesting. I also thank Steve Byrne for writing GNU Smalltalk, since GNU lightning was first developed as a tool to be used in GNU Smalltalk’s dynamic translator from bytecodes to native code." [...] PDS: Overall, looks like a great idea! Also, I think that future compiler and VM writers and FPGA soft-CPU authors -- should target this "abstracted instruction set"! Keywords: Instruction Set Architecture, ISA, x86, RISC, RISC-V, Abstraction, Abstract Instruction Set, Instruction Set Subset, Generic Compatibility Layer

riffraff 6y ago

I believe Guile and GNU Smalltalk used to run on lightning, do they still do? What other projects use GNU lightning these days?

bjoli 6y ago
Guile rund om a modified version of lightning 1 (called lightening for maximum confusion) due to the recent version being more heavy weight. It is a part of the recent JITted 3.0 release, which brought a lot better performance to the already completely OK 2.2 branch.
sedachv 6y ago
GNU CLISP had GNU Lightning added as an option for compiling its VM bytecode in 2008. IMO a bad decision - more maintenance burden; code can now have subtle different bugs whether run in the CLISP interpreter, CLISP bytecode VM, or CLISP bytecode compiled with GNU Lightning; generated code is much larger than bytecode (small memory use was the one unique thing CLISP has over other Common Lisp implementations); performance is still low compared to other Common Lisp compilers. Today libgccjit seems like a much better alternative for this kind of application than GNU Lightning.

snicker7 6y ago

GNU Guile (known primarily for its Scheme implementation) uses a modified version Lightning under the hood.

tyingq 6y ago

There's both a COPYING file that has the GPLv3 in it, and a COPYING.LESSER file with the LGPL in it. I can't tell what's meant by that. Dual licensed? Or portions GPLv3 and portions LGPL?

e12e 6y ago
Look at the actual files for copyright (or copyleft, if you will): http://git.savannah.gnu.org/cgit/lightning.git/tree/lib/lightning.c http://git.savannah.gnu.org/cgit/lightning.git/tree/lib/ligh... Looks like the libraryis lgpl,I couldn't find licensefor the docs themselves (I assume they use document license (COPYING.DOC)). There seemsto be at least one example program that is gpl: http://git.savannah.gnu.org/cgit/lightning.git/tree/doc/body.texi#n1671 http://git.savannah.gnu.org/cgit/lightning.git/tree/doc/body...
- tyingq 6y ago
  Helpful, thanks. Feels like they should summarize it somewhere though, since GPLv3 and LGPL are wildly different.
  - gspr 6y ago
    The text of the LGPL relies on the text of the GPL.
elvis70 6y ago
It's under LGPL. The LGPL allows you to relicense your work under the GPL, if you want.
bonzini 6y ago
The LGPLv3 is implemented as a set of additional permission on top of the GPLv3. The library is LGPL.
segfaultbuserr 6y ago
Since GPLv3, {L,}GPL have been "grand unified" to simplify things. Like code reuse, LGPLv3 is a just small patch on top of GPLv3 that says "given Condition X, you don't have to follow Clause Y in GPLv3", it's literally just a few short paragraphs. Without the original GPLv3 text, LGPLv3 is meaningless, like a running an executable without its shared library. Hence, although many people don't realize it, technically every project that licenses itself under LGPLv3 must contain a copy of GPLv3.

analognoise 6y ago

Too bad it's GPL, is there something like this that's MIT, BSD, or Apache licensed?

lucian1900 6y ago
Cranelift is licensed Apache 2 https://github.com/bytecodealliance/wasmtime/tree/main/cranelift https://github.com/bytecodealliance/wasmtime/tree/main/crane...
- analognoise 6y ago
  Awesome I'll check it out, thank you.
neolog 6y ago
If you want to benefit from free software you can share your own software by licensing it GPL.
- analognoise 6y ago
  The MIT license predates the GPL. Real freedom has been there the whole time.
  - neolog 6y ago
    The MIT license encourages freeriding and impoverishes the free-software community, which reduces overall freedom.
    - analognoise 6y ago
      Weird that you'd say that - given that our license predates the GNU license and the fact that our community is way larger than the GNU one. It's impoverishing the free-software community by... existing for longer and having WAY more people involved in making free-software?

rjeli 6y ago

Why does it only have six registers? The end user probably doesn't want to write a register allocator.

AlexAltea 6y ago
Might not be the reason, but six registers make it trivial to write a backend for 32-bit x86 CPUs, since they offer 8 general-purpose registers minus 2 typically used for stack pointer and base (ESP, EBP respectively). That leaves 6 registers usable by compilers: EAX, EDX, ECX, EBX, ESI, EDI. To my knowledge, any other architecture targeted by GNU lightning offers more registers, so this magic number is effectively the largest number of registers one can offer where register allocation would become trivial for all architectures.
tom_mellior 6y ago
"Six registers are not very much, but this restriction was forced by the need to target CISC architectures which, like the x86, are poor of registers; anyway, backends can specify the actual number of available registers with the calls JIT_R_NUM (for caller-save registers) and JIT_V_NUM (for callee-save registers)." (https://www.gnu.org/software/lightning/manual/lightning.html#The-instruction-set https://www.gnu.org/software/lightning/manual/lightning.html...) There used to be a boardly similar library called libjit https://www.gnu.org/software/libjit/ https://www.gnu.org/software/libjit/ which was slightly higher level: It offered an arbitrary number of virtual registers. It took care of register allocation, I don't recall how sophisticated it was. The docs call it "global" allocation, i.e., taking the whole function into account at once, as opposed to local allocation that considers blocks in isolation.
- cout 6y ago
  It wasn't very sophisticated when I used it 12 years ago, but it was "good enough". I know work has been done since then on the register allocator. No idea what the current state is.

dannyobrien 6y ago

Does anyone know how this compares with libgccjit ? What are the difference in their use cases? (I note that Guile picked lightning, and emacs chose libgccjit for instance for what seems to be similar roles.)

rekado 6y ago
Guile uses lightening (with an "e"), which is derived from GNU lightning: https://gitlab.com/wingo/lightening https://gitlab.com/wingo/lightening
- bonzini 6y ago
  ... which is essentially the same as the first version of lightning before the maintainer changed and wrote an optimizer. The first version was literally just a header file emitting constant instruction patterns.
moonchild 6y ago
Libgccjit is based on regular gcc, which is a heavyweight compiler. The 'jit' is somewhat of a misnomer; you wouldn't want to actually use that for any of the same cases as a traditional jit (e.g. browsers). (LLVM jit has the same problems.) I think the main point of libgccjit is actually to provide a cleaner interface than GIMPLE to gcc.
- sedachv 6y ago
  > you wouldn't want to actually use that for any of the same cases as a traditional jit Well the most high-profile project using libgccjit right now is Elisp (feature/native-comp GNU Emacs branch, which by all indications will eventually make it into the official release), and using GCC as a JIT compiler is how Embeddable Common Lisp has always worked.
  - aidenn0 6y ago
    I don't think ECL uses GCC as a JIT; most common lisps use an incremental AOT compiler, which is different. ECL also has an interpreter, but I'm pretty sure it doesn't use GCC.

orf 6y ago

Has had 6 commits in the last two years. From the same person. 2.1.0 was released 6 years ago, the latest 2.1.3 release came out two years ago. Releases are available from some random FTP server. On one hand this could be software that is completely done and requires no more maintenance. On the other hand it could be another random half-baked GNU project with a single champion who has lost interest but is still saddled with a purist/terrible new contributor onboarding experience dooms it to an inevitable grave. You can be the judge of that.

rekado 6y ago
"Random FTP server"? You mean the GNU FTP server?
- orf 6y ago
  You speak of that like it’s some bastion of software delivery. You mean THE gnu ftp server?? It’s a random FTP server. Could be sourceforge for it all it matters. Doesn’t even support SFTP.
  - rekado 6y ago
    If you look closely you'll see that ftp.gnu.org is an HTTP server. But it seems that you just want to troll.
    - orf 6y ago
      And if you look closer you’ll see that it’s a FTP server with a HTTP interface (http, not https, by default mind you). ftp://ftp.gnu.org/
      - mcosta 6y ago
        The link is http, https also works. Even if it is only http, so what? PGP signatures exists too.
      - spicybright 6y ago

kwdc 6y ago

There was a similar JIT library written in C mentioned on HN awhile ago. Wasnt libjit or lightning. I forget what it was called. Seemed a lot simpler.

cout 6y ago
I'm not sure how one could get much simpler than libjit. The first time I read the documentation I had a working jit for a toy language five minutes later, and less than a week after that I was jit-compiling Ruby code (not the full language, obviously, just a subset). It really is a pleasure to use, since the API is based on normal C functions instead of macros.
- kwdc 6y ago
  Simpler doesn't imoly more featured; as I recall that simpler approach had more rough edges. Still haven't found it.