8 ms·

A New R6RS Scheme Compiler

182 points by widdershins 7y ago

neilv 7y ago
This looks like impressive progress towards a Scheme implementation for systems programming. I'll be interested to see how GC, interrupt-handling, etc. work. With an aarch64 backend, and some graphics&GUI work, it might also make sense atop PostmarketOS Linux kernel, for handhelds.[1] If one needs to heavily rework Linux userland anyway, and especially if motivated by enthusiasm, it's an opportunity to rethink, rather than automatically inherit decades of desktop stack. [1] Notes on approaching this incrementally at https://www.neilvandyke.org/postmarketos/ https://www.neilvandyke.org/postmarketos/ , though you could alternatively start by building up atop bare kernel&drivers, and Loko Scheme looks like a promising candidate for that.
exDM69 7y ago
The related article about using CPU alignment checking for "type checking" tagged pointers is quite interesting. I wonder if this technique has been used before (in lisp machines or something?). I'd be interested in hearing about practical matters like performance and CPU support too. https://weinholt.se/articles/alignment-check/ https://weinholt.se/articles/alignment-check/
- crististm 7y ago
  Lisp machines used tag bits
- rjsw 7y ago
  The SPARC CPU was designed to make use of this. Alignment checking was enabled by default, the tagged add/sub instructions required that the tag in the lower two bits for an integer was 0, tags for pointers were expected to be 3. Lisp Machines were word addressed so the technique wouldn't work for them.
pjmlp 7y ago
> It is written in R6RS Scheme and a wafer thin amount of assembly. Once it has been bootstrapped it can self-compile. There is no C code in the compiler or the runtime. > ...multiboot binaries for bare hardware Multiple brownie points combo! Congratulations on the work achieved.
- jacobush 7y ago
  PicoLisp also has a tiny assembler layer. (And two parallel implementations, one in Java, one in C.)
  - pjmlp 7y ago
    I am aware of it, thanks for bringing it up. The more the merrier.
scriptdevil 7y ago
There is definitely room for a scheme compiler. But the scheme ecosystem felt extremely fractured the last time I took a look. Chicken had a decent but aging repo. Chez had a great compiler (I prefer not compiling to C) but it's package ecosystem was not something well advertised if it did have one. Racket has too many dialects which might appeal to some but found the number of sub languages overwhelming for a casual schemer.
- craigsmansion 7y ago
  I always thought that was the best feature of scheme. Nobody agrees on anything, except mainly R5RS, keeping it from going down the path of "one implementation for everything, always" that most languages suffer from once they become a little popular. Scheme is one of the very few languages that has resisted accruing a lot of bloat throughout its existence (R6RS never was very popular because of that) People underestimate the upside of being able to paste any example snippet into your repl and have it work transparently in almost every scheme, without having to worry about libraries or eco-systems. For anything more exotic, you grab the nearest SRFI and squeeze it around until it works in the scheme you're working with. Scheme: it's so simple, any subgenius could use it!
- alfiedotwtf 7y ago
  I am just getting into Scheme, and have heard good things about Chez. Any pointers to what is standard out in the wild? What are the de-facto Scheme tools are people using these days?
  - thosakwe 7y ago
    I just got into Scheme this past month, but I started with GNU Guile, and really like it.
  - RhysU 7y ago
    https://akkuscm.org/ https://akkuscm.org/ is a seemingly vibrant package manager for Chez and others. It is also by the author of the article.
  - 7y ago
_ph_ 7y ago
Sounds like a very interesting project, but actually I belong to the crowd who would close the tab basically after reading the second paragraph. Which is a pity, because this sounds like an implementation which could be very nice.
zucker42 7y ago
About the AGPL license, does it extend to programs compiled with Loko? The author seems to not want it to with the comment about the license not extending to user space. My understanding, however, is that would iff the programs used a library provided by the AGPL compiler. Hence why gcc has the linking exception.
- _ph_ 7y ago
  Yes, unless there is a specific exception, the program would be covered by AGPL. Even if you just use it on the server, you have to release your programs code under the AGPL.
  - e12e 7y ago
    The compiled program. So you could give users the compiler (under AGPL) and source code to your program under a proprietary license. But you probably wouldn't, I guess. Any compiled artifacts would need to be distributed (if they are distributed) under AGPL. I'm a fan of Free software, but much prefer GPL/LGPL for compilers and runtimes/standard libraries.
    - mumblemumble 7y ago
      It depends. . . if the compiled program is 100% compiler output, then it's fine. If the compiled program ends up including bits of library code that ships with the compiler, then, in the absence of special exceptions, the output will also need to be AGPL-licensed. That's exactly why, for example, LLVM ships with a modified version of the Apache license: http://llvm.org/foundation/relicensing/LICENSE.txt http://llvm.org/foundation/relicensing/LICENSE.txt (see the bottom)
      - e12e 7y ago
        I'm having trouble seeing how this won't incorporate a substantial runtime in all builds? It's a full scheme system...
  - OskarS 7y ago
verisimilitudes 7y ago
All of my software that is not brief and trivial and so benefiting from it is licensed under the AGPLv3, although I don't use the or any later versions clause, as I don't trust licenses that have yet to be written, and the recent events make me feel all the superior for not having done so. I'm glad to see more AGPLv3 software. My opinion on a batch compiler is different from my opinion on an interactive environment and I'll share a method on how to make money with the AGPLv3: For a batch compiler, that is a language that doesn't need the compiler to run, write an AGPLv3 compiler without a linking exception and then demand sufficient funds be raised to release a version with the linking exception. This is then a way to write a Free Software compiler and still demand money for it, for many to actually want to use it. I wouldn't license something such as a Lisp compiler with the exception, however, as the implementation generally exists with the program and that removes some advantages of it being AGPLv3. Fortunately, it seems the author has used the AGPLv3 and any desired exception uses vague language not in the license, so it's legally irrelevant. I'd find it best if there were no exception for this program, but it's not my decision. In closing, this has made me want again to write my own Scheme implementation, so perhaps I'll do that at some point soon.
- nickpsecurity 7y ago
  I had a similar idea with it starting dual-licensed so I could straight-up sell it to companies. Optionally use a more permissive license if it hit a certain total. Each new version is done the same way so upgrades on the more-permissive version still get paid for. Plus, the "get what you pay for" companies might only buy a proprietary version.

182 points by widdershins 7y ago

This looks like impressive progress towards a Scheme implementation for systems programming. I'll be interested to see how GC, interrupt-handling, etc. work. With an aarch64 backend, and some graphics&GUI work, it might also make sense atop PostmarketOS Linux kernel, for handhelds.[1] If one needs to heavily rework Linux userland anyway, and especially if motivated by enthusiasm, it's an opportunity to rethink, rather than automatically inherit decades of desktop stack. [1] Notes on approaching this incrementally at https://www.neilvandyke.org/postmarketos/ https://www.neilvandyke.org/postmarketos/ , though you could alternatively start by building up atop bare kernel&drivers, and Loko Scheme looks like a promising candidate for that.

exDM69 7y ago

The related article about using CPU alignment checking for "type checking" tagged pointers is quite interesting. I wonder if this technique has been used before (in lisp machines or something?). I'd be interested in hearing about practical matters like performance and CPU support too. https://weinholt.se/articles/alignment-check/ https://weinholt.se/articles/alignment-check/

crististm 7y ago
Lisp machines used tag bits
rjsw 7y ago
The SPARC CPU was designed to make use of this. Alignment checking was enabled by default, the tagged add/sub instructions required that the tag in the lower two bits for an integer was 0, tags for pointers were expected to be 3. Lisp Machines were word addressed so the technique wouldn't work for them.

pjmlp 7y ago

> It is written in R6RS Scheme and a wafer thin amount of assembly. Once it has been bootstrapped it can self-compile. There is no C code in the compiler or the runtime. > ...multiboot binaries for bare hardware Multiple brownie points combo! Congratulations on the work achieved.

jacobush 7y ago
PicoLisp also has a tiny assembler layer. (And two parallel implementations, one in Java, one in C.)
- pjmlp 7y ago
  I am aware of it, thanks for bringing it up. The more the merrier.

scriptdevil 7y ago

There is definitely room for a scheme compiler. But the scheme ecosystem felt extremely fractured the last time I took a look. Chicken had a decent but aging repo. Chez had a great compiler (I prefer not compiling to C) but it's package ecosystem was not something well advertised if it did have one. Racket has too many dialects which might appeal to some but found the number of sub languages overwhelming for a casual schemer.

craigsmansion 7y ago
I always thought that was the best feature of scheme. Nobody agrees on anything, except mainly R5RS, keeping it from going down the path of "one implementation for everything, always" that most languages suffer from once they become a little popular. Scheme is one of the very few languages that has resisted accruing a lot of bloat throughout its existence (R6RS never was very popular because of that) People underestimate the upside of being able to paste any example snippet into your repl and have it work transparently in almost every scheme, without having to worry about libraries or eco-systems. For anything more exotic, you grab the nearest SRFI and squeeze it around until it works in the scheme you're working with. Scheme: it's so simple, any subgenius could use it!
alfiedotwtf 7y ago
I am just getting into Scheme, and have heard good things about Chez. Any pointers to what is standard out in the wild? What are the de-facto Scheme tools are people using these days?
- thosakwe 7y ago
  I just got into Scheme this past month, but I started with GNU Guile, and really like it.
- RhysU 7y ago
  https://akkuscm.org/ https://akkuscm.org/ is a seemingly vibrant package manager for Chez and others. It is also by the author of the article.
- 7y ago

_ph_ 7y ago

Sounds like a very interesting project, but actually I belong to the crowd who would close the tab basically after reading the second paragraph. Which is a pity, because this sounds like an implementation which could be very nice.

zucker42 7y ago

About the AGPL license, does it extend to programs compiled with Loko? The author seems to not want it to with the comment about the license not extending to user space. My understanding, however, is that would iff the programs used a library provided by the AGPL compiler. Hence why gcc has the linking exception.

_ph_ 7y ago
Yes, unless there is a specific exception, the program would be covered by AGPL. Even if you just use it on the server, you have to release your programs code under the AGPL.
- e12e 7y ago
  The compiled program. So you could give users the compiler (under AGPL) and source code to your program under a proprietary license. But you probably wouldn't, I guess. Any compiled artifacts would need to be distributed (if they are distributed) under AGPL. I'm a fan of Free software, but much prefer GPL/LGPL for compilers and runtimes/standard libraries.
  - mumblemumble 7y ago
    It depends. . . if the compiled program is 100% compiler output, then it's fine. If the compiled program ends up including bits of library code that ships with the compiler, then, in the absence of special exceptions, the output will also need to be AGPL-licensed. That's exactly why, for example, LLVM ships with a modified version of the Apache license: http://llvm.org/foundation/relicensing/LICENSE.txt http://llvm.org/foundation/relicensing/LICENSE.txt (see the bottom)
    - e12e 7y ago
      I'm having trouble seeing how this won't incorporate a substantial runtime in all builds? It's a full scheme system...
- OskarS 7y ago

verisimilitudes 7y ago

All of my software that is not brief and trivial and so benefiting from it is licensed under the AGPLv3, although I don't use the or any later versions clause, as I don't trust licenses that have yet to be written, and the recent events make me feel all the superior for not having done so. I'm glad to see more AGPLv3 software. My opinion on a batch compiler is different from my opinion on an interactive environment and I'll share a method on how to make money with the AGPLv3: For a batch compiler, that is a language that doesn't need the compiler to run, write an AGPLv3 compiler without a linking exception and then demand sufficient funds be raised to release a version with the linking exception. This is then a way to write a Free Software compiler and still demand money for it, for many to actually want to use it. I wouldn't license something such as a Lisp compiler with the exception, however, as the implementation generally exists with the program and that removes some advantages of it being AGPLv3. Fortunately, it seems the author has used the AGPLv3 and any desired exception uses vague language not in the license, so it's legally irrelevant. I'd find it best if there were no exception for this program, but it's not my decision. In closing, this has made me want again to write my own Scheme implementation, so perhaps I'll do that at some point soon.

nickpsecurity 7y ago
I had a similar idea with it starting dual-licensed so I could straight-up sell it to companies. Optionally use a more permissive license if it hit a certain total. Each new version is done the same way so upgrades on the more-permissive version still get paid for. Plus, the "get what you pay for" companies might only buy a proprietary version.