6 ms·

Autology: A Lisp with access to its own interpreter

136 points by simonpure 2y ago

timonoko 2y ago
Mind blown when you write better language using the language itself. Why I wasted so much time in assembly? Then you remember there are certain restrictions as regards to time travel.
sargstuff 2y ago
Lisp take on generating an embedded VM at runtime/ad hoc basis (vs. polymorphic coding)? How would use the most recent 'trie of lisp git diffs' differ from command to do a system call fork of lisp / duplicate lisp lists / apply 'diffs' to forked system call of lisp? Perhaps, lisp concurrency support without a system()/debugger.[1] (humor) To bad iLisp would be confused with other brand name(s). [1] : Evil Scheduler: Mastering Concurrency Through Interactive Debugging : http://aoli.al/blogs/deadlock-empire/ http://aoli.al/blogs/deadlock-empire/
lifthrasiir 2y ago
I think CosmicOS had the same idea [1] to seamlessly introduce special forms without defining the implicit interpreter in advance. [1] https://cosmicos.github.io/message.html#section18 https://cosmicos.github.io/message.html#section18
- johnisgood 2y ago
  What is "" and so forth? It does not even render properly here on HN after having copy pasted it from the website. Custom font or what? I will check later if no one replies.
  - ropejumper 2y ago
    https://imgur.com/a/5zDRli5 https://imgur.com/a/5zDRli5
    - deleted 2y ago
      [deleted]
  - lifthrasiir 2y ago
    The CosmicOS message has multiple possible encodings, including base 4 representations shown below (e.g. the message starts with 121001031211132233...). The very first page contains glyphs corresponding their compact graphic representations. I think there was some description about these, but I couldn't find them so I will just recall my understanding here. The base 4 representation uses four symbols 0--3, which are derived from textual tokens: --- 12 (0|1)+ 3 -- Names. A particular name `xxx` is only available after the `intro xxx;` declaration. Note that the name `intro` itself is the first token ever and never explicitly defined. 02 (0|1)+ 3 -- Binary numbers. Note that unary numbers used initially are encoded using a special function "unary" with ordinary binary numbers 0 (0203) and 1 (0213) as bits. 2 without preceding 0 or 1 -- Opening parenthesis. 3 without preceding 0 or 1 -- Closing parenthesis. 023 -- Auto-closing opening parenthesis, denoted `|` in the textual form. `(a | b | c d)` is same as `(a (b (c d)))`. 123 -- Single-item parentheses, denoted `$` in the textual form. `$x` means `(x)` and typically used in variable getters. 2233 -- Marks the end of the current expression. Each expression in the CosmicOS message should evaluate to true. --- Now, keep in mind that every name is effectively a specially marked number. I think each line segment in glyphs corresponds to a particular bit position, and numbers have short overline and underline to distinguish themselves from names.
deleted 2y ago
[deleted]
johnisgood 2y ago
These "inter-lingual" examples such as https://github.com/Kimbsy/autology/blob/main/resources/examples/inter-lingual-4.atl https://github.com/Kimbsy/autology/blob/main/resources/examp... are pretty cool. What is the use-case of this or why would one want to do this? Just trying to bounce off ideas.
- jampekka 2y ago
  Calling code across languages at least is very widespread. And typically requires custom, often clunky, solutions for all language pairs.
  - johnisgood 2y ago
    So is this like a "better FFI", in some sense of the term?
    - immibis 2y ago
      Looks like just automated FFI, putting some glue around your function and then invoking the appropriate compiler for that language. I don't really see the difference between (with-*i* "c" foo) and a macro that expands to (with-c "foo")
- Kimbsy 2y ago
  So the best thing I could come up with (assuming a theoretical perfect implementation that wasn't horribly slow and expensive) was the idea that since many languages have their own niches and categories of problems that they are best used to express/solve, you could conceivably work on a complex problem where different parts of the computation are most elegantly expressed in different languages. So having a metalanguage that allows you to switch syntax in a lexical scope would be useful!? Highly doubtful that this benefit would outweigh the egregious complexity cost of using a language like Autology, but a fun thought experiment.
  - johnisgood 2y ago
    It is definitely worth pursuing. :)
bandrami 2y ago
Isn't having access to the interpreter (and scanner, and parser, and compiler) kind of what makes something Lisp rather than something else with a lot of parentheses?
- drob518 2y ago
  Typically, Lisp is quite flexible, via macros and reader macros, but the smallest core is still fixed in function (the part that implements special forms, order of evaluation, etc., basically eval and apply). This makes even that core replaceable on a form by form basis. Interesting from a theoretical perspective but not very useful in practice, and performance killing as the author points out.
  - tines 2y ago
    You can already kind of replace that core on a form-by-form basis in plain Lisp by redefining EVAL and APPLY, and passing whatever forms you do not care about redefining to the original versions of them.
    - deleted 2y ago
      [deleted]
    - chabska 2y ago
      that requires dynamic scoping right? Wouldn't work in lexical scoping.
- agumonkey 2y ago
  Access to eval and a generic recursive data type as internal structure means the language is closed on itself, can introspect, manipulate, create more through the same means it's programmed (see norvig lispy with python [lists] instead of parens)
  - kazinator 2y ago
    But eval is opaque; it implements a particular dialect and that'w what you get when you recurse into it. It looks like Autology provides a way to customize the dialect that eval will process recursively. Interpreter extensions that handle subexpressions via eval will then work in the context of the altered dialect (that being provided by a parent interpreter extension itself).
ape4 2y ago
Since its written in Clojure which runs on the JVM why not a *jvm* variable also ;)
behnamoh 2y ago
so it's hot code reloading on steroids? really neat idea, I'm working on a similar Lisp-style language and will probably adopt this idea (and cite you of course). In my language, you can redefine any symbol, even numbers, so (def 10 12) is valid code (so is (def def 42), which breaks `def`!). I wonder what use cases there are for such extreme flexibility, aside from funs and games!
- anonzzzies 2y ago
  None; it's for fun and games. But maybe thats the most enjoyable use case anyway.
  - Kimbsy 2y ago
    I couldn't agree more
- Kimbsy 2y ago
  Please feel free to use any/all of it, it's really fun to play around with.
- v9v 2y ago
  Redefining 'def' is a trick used in Tcl to make "safe" configuration languages exposed to the end user. You define all of your configuration options as functions and then undefine everything else (including "proc" which corresponds to defun) allowing you to evaluate users' config files with this safe interpreter as code. I can't find a good source for this practice of manually undefining functions, but here is a link to relevant built-in functionality: https://www.tcl-lang.org/man/tcl8.6/TclCmd/safe.htm https://www.tcl-lang.org/man/tcl8.6/TclCmd/safe.htm
abeppu 2y ago
I haven't read any of the code to see how it compares, but this work reminds me of research from Nada Amin & Tiark Rompf on "towers of interpreters", where a stack of meta-circular interpreters uses some explicit staging operations that make it possible to specialize for the interpreter code in a way that removes overhead. https://www.cs.purdue.edu/homes/rompf/papers/amin-popl18.pdf https://www.cs.purdue.edu/homes/rompf/papers/amin-popl18.pdf
- cipherself 2y ago
  This in turn, is based on black[0] which is an extension to scheme, which in turn is inspired by [1] [0] http://pllab.is.ocha.ac.jp/~asai/Black/ http://pllab.is.ocha.ac.jp/~asai/Black/ [1] https://www.lirmm.fr/~dony/enseig/MR/notes-etudes/ReflectiveInterpreter.pdf https://www.lirmm.fr/~dony/enseig/MR/notes-etudes/Reflective...
- almostgotcaught 2y ago
  There's no relationship - tower of interpreters is related to lightweight modular staging https://dl.acm.org/doi/10.1145/1868294.1868314 https://dl.acm.org/doi/10.1145/1868294.1868314 This is just lisp or more broadly "I'm an interpreted language" since you know... Python has access to much of its interpreter runtime too . And if you think you can't do wild stuff in Python you can (hint: you can use ctypes on libpython to do way more than you think you can do).
souenzzo 2y ago
It's awesome to see the amount of ideas you can explore with 1kloc lines of clojure!
Cieric 2y ago
I was working on something similar to this a while ago, but the goal of it was experimenting with what a language that could modify it's own syntax would look like. I was going to write something as basic as possible and then have example scripts on transforming the syntax to other languages. I was probably just going to do brainfuck and c in the end, but I wanted something like that to be possible. I couldn't figure out how to make a language modify a tree structure for tweaking the ast of the interpreter, but I guess lisp fits the bill there.
- stevekemp 2y ago
  You could be inspired by FORTH and not have an AST at all ..
- rpcope1 2y ago
  This is basically just Forth or Factor.
jlarocco 2y ago
In Common Lisp, at least, it's pretty typical for the compiler or interpretter to have packages for interacting with its internals. For example, `sb-c`, `sb-vm`, and `sb-ext` packages in SBCL; and the `ccl` package with Clozure Common Lisp, etc. That ability also shows up in the inspector - it's possible to inspect types, classes, functions, etc. and inspect the internal data structures the compiler uses to represent them. With SBCL it's even possible to modify the "virtual ops" the compiler uses, and emit assembly code that SBCL doesn't support out of the box. It's a really convenient feature, and it would be nice if to see other languages pick it up.
- johnisgood 2y ago
  Are there any examples?
  - phoe-krk 2y ago
    https://pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-code-breadboard/ https://pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-co...
    - johnisgood 2y ago
      Thank you! Looks useful. "The Common Lisp Condition System" is a great book, by the way, thanks. :)
      - phoe-krk 2y ago
        Thanks! I hope it serves you well.
  - lisper 2y ago
    Here is a little code snippet for CCL that changes the compiler to accept a Scheme-like ((...) ...) syntax. https://flownet.com/ron/lisp/combination-hook.lisp https://flownet.com/ron/lisp/combination-hook.lisp
JoelMcCracken 2y ago
reminds me a little bit of the Kernel programming language
- bsder 2y ago
  Reminds me a lot, except that Kernel does away with "special forms". Link for "Kernel" because it's a lousy name to have to search for: https://web.cs.wpi.edu/~jshutt/kernel.html https://web.cs.wpi.edu/~jshutt/kernel.html One interesting bit that remains unappreciated about Kernel is that environments are reified and are "copy-on-write". So, if you make an environment change, it propagates forward but previous calls only see the previous environment unless explicitly give them access. It's a little unusual, but it works out pretty well. You can call the "ground environment" which is immutable and lets you compile things. You can access the "lexical environment" which works like normal. And you can call a "dynamic environment" that lets you capture things. It's interesting in that it contains the scope of "dynamic environments" and still allows things to be compiled (which was the big downside of dynamic environments circa 1970s). One downside I found is that the obvious implementation thrashes the hell out of your garbage collector. "Environments" really want a data structure that is more complicated and copy-friendly than "cons pairs".
  - sparkie 2y ago
    > One interesting bit that remains unappreciated about Kernel is that environments are reified and are "copy-on-write". So, if you make an environment change, it propagates forward but previous calls only see the previous environment unless explicitly give them access. That's not quite how it works. Environments are mutable, but only the locals of an environment that we have a direct reference to. We cannot mutate any of the bindings in the parents without a reference to the parents, and the language provides no facility to obtain references to the parents given only a reference to the child. We can however, explicitly pass around references to parent environments if we do have access to them, because environments are first-class, and we can move around references where we want. We can have a child mutate its grandparent environment, and those changes will be visible to the child's parent - but this is only possible if the child has a direct reference to the grandparent - which it would not have under normal circumstances - it has to be provided with that reference explicitly. We could for example, bind our top-level scope into a symbol in the top-level, such that any descendants of the top level can mutate any bindings in the top level. ($define! *globals* (get-current-environment)) ($define! x 0) ($define! print-x ($lambda () (write x))) ($define! foo ($lambda () ($set! *globals* x 1))) (foo) (print-x) Here we see that the lambda defined for `print-x` does not get an immutable copy of the top-level scope, because if that were the case, then the `x` printed by `print-x` would be a different `x` that foo mutates, and `print-x` would still print 0, but this should print 1. Obviously, we would not do this in practice because it would let us rebind `print-x` and `foo`, or even rebind `*globals*` to some another value. A safer way to do this would be to bind a setter for x in the top level. ($define! x 0) ($define! $set-x! (($vau () top-level ($vau (value) #ignore ($set! top-level x value))))) ($define! print-x ($lambda () (write x))) ($define! foo ($lambda () ($set-x! 1))) (foo) (print-x) This would allow any descendants of the top-level to mutate `x`, but not mutate any other binding in the top level. This works because the outermost `$vau` on $set-x!` is evaluated in the top level, where `$set-x!` is defined, so it receives the top level environment as its dynamic environment, which it binds to the symbol `top-level` in its local environment, and then returns an operative whose static environment is its own local environment. When `$set-x!` is called, it ignores its caller's dynamic environment, but its static environment contains the symbol `top-level`, bound to the top level environment. --- The immutable part of environments is the list of parents that any environment has. When we create an environment through say `(make-environment list-of-parents)`, the environment creates an immutable copy of the list of parents, which prevents whoever created the environment from being able to modify this list later. For example, we wouldn't want this ($define! parents (list some-environment)) ($define! env (make-environment parents)) ($set-car! parents some-other-environment) The caller mutates `parents` after creating `env`, but this will not affect `env`, because it created an immutable copy of `parents`. However, the creator of this list can mutate the bindings in the parent after creating the env. ($define! parents (list some-environment)) ($define! env (make-environment parents)) ($set! some-environment some-binding some-value) This would be reflected in `env`, whose `some-binding` would now resolve to `some-value`. But usually it will be the case that `some-environment` is encapsulated by the creator who will eventually return `env`, but no reference to `some-environment` - which will prevent any mutation of the bindings in the parents. --- There is no way to mutate the ground environment because every standard-environment is a child of the ground, but with no direct reference to the ground. The ground is just its parent. The definition of `make-kernel-standard-environment` is interesting, because it's just ($define! make-kernel-standard-environment ($lambda () (get-current-environment))) Because Kernel specifies that ground bindings are evaluated in the ground environment, and the lambda creates a child of the ground with no initial local bindings, then when we get that environment, we have an empty environment with ground as its parent. Although Kernel is specified using mutable environments, the Kernel report does make them optional. It doesn't really go into depth about how one would implement Kernel with immutable environments besides a brief note. If you do have immutable environments, then a fun challenge is how you would implement `make-kernel-standard-environment`. If using the definition above, then the environment returned by the call to `make-kernel-standard-environment` notably does not have the binding `make-kernel-standard-environment` in it (because the binding happens after).
bsima 2y ago
If you like this, you'll love Shen https://shenlanguage.org/ https://shenlanguage.org/
shadowgovt 2y ago
Interesting! Racket has the capacity to declare DSLs that are then applied at the file level, but this is even finer-grained.
Kimbsy 2y ago
Hi this is my project! I'm giving a talk on this in May for London Clojurians, and I'll also be talking about it at Lambda Days in Krakow in June. https://www.meetup.com/london-clojurians/events/306843409/ https://www.meetup.com/london-clojurians/events/306843409/ The project was initially inspired by Dr John Sturdy's thesis "A Lisp through the Looking Glass" which is all about interpreter towers and the ability to modify them in both directions. This is all just for fun, I've yet to think of anything truly useful you can do with this. If you have any cool ideas please let me know, they might even make it into my talk!
ludston 2y ago
You can actually get completely stupid with this in Common Lisp using something called a "Reader Macro" which lets you temporarily take complete control over the interpreter. For example, I have this joke project that defines a DSL for fizzbuzz: https://github.com/DanielKeogh/fizzbuzz https://github.com/DanielKeogh/fizzbuzz

136 points by simonpure 2y ago

Mind blown when you write better language using the language itself. Why I wasted so much time in assembly? Then you remember there are certain restrictions as regards to time travel.

sargstuff 2y ago

Lisp take on generating an embedded VM at runtime/ad hoc basis (vs. polymorphic coding)? How would use the most recent 'trie of lisp git diffs' differ from command to do a system call fork of lisp / duplicate lisp lists / apply 'diffs' to forked system call of lisp? Perhaps, lisp concurrency support without a system()/debugger.[1] (humor) To bad iLisp would be confused with other brand name(s). [1] : Evil Scheduler: Mastering Concurrency Through Interactive Debugging : http://aoli.al/blogs/deadlock-empire/ http://aoli.al/blogs/deadlock-empire/

lifthrasiir 2y ago

I think CosmicOS had the same idea [1] to seamlessly introduce special forms without defining the implicit interpreter in advance. [1] https://cosmicos.github.io/message.html#section18 https://cosmicos.github.io/message.html#section18

johnisgood 2y ago
What is "" and so forth? It does not even render properly here on HN after having copy pasted it from the website. Custom font or what? I will check later if no one replies.
- ropejumper 2y ago
  https://imgur.com/a/5zDRli5 https://imgur.com/a/5zDRli5
  - deleted 2y ago
    [deleted]
- lifthrasiir 2y ago
  The CosmicOS message has multiple possible encodings, including base 4 representations shown below (e.g. the message starts with 121001031211132233...). The very first page contains glyphs corresponding their compact graphic representations. I think there was some description about these, but I couldn't find them so I will just recall my understanding here. The base 4 representation uses four symbols 0--3, which are derived from textual tokens: --- 12 (0|1)+ 3 -- Names. A particular name `xxx` is only available after the `intro xxx;` declaration. Note that the name `intro` itself is the first token ever and never explicitly defined. 02 (0|1)+ 3 -- Binary numbers. Note that unary numbers used initially are encoded using a special function "unary" with ordinary binary numbers 0 (0203) and 1 (0213) as bits. 2 without preceding 0 or 1 -- Opening parenthesis. 3 without preceding 0 or 1 -- Closing parenthesis. 023 -- Auto-closing opening parenthesis, denoted `|` in the textual form. `(a | b | c d)` is same as `(a (b (c d)))`. 123 -- Single-item parentheses, denoted `$` in the textual form. `$x` means `(x)` and typically used in variable getters. 2233 -- Marks the end of the current expression. Each expression in the CosmicOS message should evaluate to true. --- Now, keep in mind that every name is effectively a specially marked number. I think each line segment in glyphs corresponds to a particular bit position, and numbers have short overline and underline to distinguish themselves from names.

deleted 2y ago

[deleted]

johnisgood 2y ago

These "inter-lingual" examples such as https://github.com/Kimbsy/autology/blob/main/resources/examples/inter-lingual-4.atl https://github.com/Kimbsy/autology/blob/main/resources/examp... are pretty cool. What is the use-case of this or why would one want to do this? Just trying to bounce off ideas.

jampekka 2y ago
Calling code across languages at least is very widespread. And typically requires custom, often clunky, solutions for all language pairs.
- johnisgood 2y ago
  So is this like a "better FFI", in some sense of the term?
  - immibis 2y ago
    Looks like just automated FFI, putting some glue around your function and then invoking the appropriate compiler for that language. I don't really see the difference between (with-*i* "c" foo) and a macro that expands to (with-c "foo")
Kimbsy 2y ago
So the best thing I could come up with (assuming a theoretical perfect implementation that wasn't horribly slow and expensive) was the idea that since many languages have their own niches and categories of problems that they are best used to express/solve, you could conceivably work on a complex problem where different parts of the computation are most elegantly expressed in different languages. So having a metalanguage that allows you to switch syntax in a lexical scope would be useful!? Highly doubtful that this benefit would outweigh the egregious complexity cost of using a language like Autology, but a fun thought experiment.
- johnisgood 2y ago
  It is definitely worth pursuing. :)

bandrami 2y ago

Isn't having access to the interpreter (and scanner, and parser, and compiler) kind of what makes something Lisp rather than something else with a lot of parentheses?

drob518 2y ago
Typically, Lisp is quite flexible, via macros and reader macros, but the smallest core is still fixed in function (the part that implements special forms, order of evaluation, etc., basically eval and apply). This makes even that core replaceable on a form by form basis. Interesting from a theoretical perspective but not very useful in practice, and performance killing as the author points out.
- tines 2y ago
  You can already kind of replace that core on a form-by-form basis in plain Lisp by redefining EVAL and APPLY, and passing whatever forms you do not care about redefining to the original versions of them.
  - deleted 2y ago
    [deleted]
  - chabska 2y ago
    that requires dynamic scoping right? Wouldn't work in lexical scoping.
agumonkey 2y ago
Access to eval and a generic recursive data type as internal structure means the language is closed on itself, can introspect, manipulate, create more through the same means it's programmed (see norvig lispy with python [lists] instead of parens)
- kazinator 2y ago
  But eval is opaque; it implements a particular dialect and that'w what you get when you recurse into it. It looks like Autology provides a way to customize the dialect that eval will process recursively. Interpreter extensions that handle subexpressions via eval will then work in the context of the altered dialect (that being provided by a parent interpreter extension itself).

ape4 2y ago

Since its written in Clojure which runs on the JVM why not a *jvm* variable also ;)

behnamoh 2y ago

so it's hot code reloading on steroids? really neat idea, I'm working on a similar Lisp-style language and will probably adopt this idea (and cite you of course). In my language, you can redefine any symbol, even numbers, so (def 10 12) is valid code (so is (def def 42), which breaks `def`!). I wonder what use cases there are for such extreme flexibility, aside from funs and games!

anonzzzies 2y ago
None; it's for fun and games. But maybe thats the most enjoyable use case anyway.
- Kimbsy 2y ago
  I couldn't agree more
Kimbsy 2y ago
Please feel free to use any/all of it, it's really fun to play around with.
v9v 2y ago
Redefining 'def' is a trick used in Tcl to make "safe" configuration languages exposed to the end user. You define all of your configuration options as functions and then undefine everything else (including "proc" which corresponds to defun) allowing you to evaluate users' config files with this safe interpreter as code. I can't find a good source for this practice of manually undefining functions, but here is a link to relevant built-in functionality: https://www.tcl-lang.org/man/tcl8.6/TclCmd/safe.htm https://www.tcl-lang.org/man/tcl8.6/TclCmd/safe.htm

abeppu 2y ago

I haven't read any of the code to see how it compares, but this work reminds me of research from Nada Amin & Tiark Rompf on "towers of interpreters", where a stack of meta-circular interpreters uses some explicit staging operations that make it possible to specialize for the interpreter code in a way that removes overhead. https://www.cs.purdue.edu/homes/rompf/papers/amin-popl18.pdf https://www.cs.purdue.edu/homes/rompf/papers/amin-popl18.pdf

cipherself 2y ago
This in turn, is based on black[0] which is an extension to scheme, which in turn is inspired by [1] [0] http://pllab.is.ocha.ac.jp/~asai/Black/ http://pllab.is.ocha.ac.jp/~asai/Black/ [1] https://www.lirmm.fr/~dony/enseig/MR/notes-etudes/ReflectiveInterpreter.pdf https://www.lirmm.fr/~dony/enseig/MR/notes-etudes/Reflective...
almostgotcaught 2y ago
There's no relationship - tower of interpreters is related to lightweight modular staging https://dl.acm.org/doi/10.1145/1868294.1868314 https://dl.acm.org/doi/10.1145/1868294.1868314 This is just lisp or more broadly "I'm an interpreted language" since you know... Python has access to much of its interpreter runtime too . And if you think you can't do wild stuff in Python you can (hint: you can use ctypes on libpython to do way more than you think you can do).

souenzzo 2y ago

It's awesome to see the amount of ideas you can explore with 1kloc lines of clojure!

Cieric 2y ago

I was working on something similar to this a while ago, but the goal of it was experimenting with what a language that could modify it's own syntax would look like. I was going to write something as basic as possible and then have example scripts on transforming the syntax to other languages. I was probably just going to do brainfuck and c in the end, but I wanted something like that to be possible. I couldn't figure out how to make a language modify a tree structure for tweaking the ast of the interpreter, but I guess lisp fits the bill there.

stevekemp 2y ago
You could be inspired by FORTH and not have an AST at all ..
rpcope1 2y ago
This is basically just Forth or Factor.

jlarocco 2y ago

In Common Lisp, at least, it's pretty typical for the compiler or interpretter to have packages for interacting with its internals. For example, `sb-c`, `sb-vm`, and `sb-ext` packages in SBCL; and the `ccl` package with Clozure Common Lisp, etc. That ability also shows up in the inspector - it's possible to inspect types, classes, functions, etc. and inspect the internal data structures the compiler uses to represent them. With SBCL it's even possible to modify the "virtual ops" the compiler uses, and emit assembly code that SBCL doesn't support out of the box. It's a really convenient feature, and it would be nice if to see other languages pick it up.

johnisgood 2y ago
Are there any examples?
- phoe-krk 2y ago
  https://pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-code-breadboard/ https://pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-co...
  - johnisgood 2y ago
    Thank you! Looks useful. "The Common Lisp Condition System" is a great book, by the way, thanks. :)
    - phoe-krk 2y ago
      Thanks! I hope it serves you well.
- lisper 2y ago
  Here is a little code snippet for CCL that changes the compiler to accept a Scheme-like ((...) ...) syntax. https://flownet.com/ron/lisp/combination-hook.lisp https://flownet.com/ron/lisp/combination-hook.lisp

JoelMcCracken 2y ago

reminds me a little bit of the Kernel programming language

bsder 2y ago
Reminds me a lot, except that Kernel does away with "special forms". Link for "Kernel" because it's a lousy name to have to search for: https://web.cs.wpi.edu/~jshutt/kernel.html https://web.cs.wpi.edu/~jshutt/kernel.html One interesting bit that remains unappreciated about Kernel is that environments are reified and are "copy-on-write". So, if you make an environment change, it propagates forward but previous calls only see the previous environment unless explicitly give them access. It's a little unusual, but it works out pretty well. You can call the "ground environment" which is immutable and lets you compile things. You can access the "lexical environment" which works like normal. And you can call a "dynamic environment" that lets you capture things. It's interesting in that it contains the scope of "dynamic environments" and still allows things to be compiled (which was the big downside of dynamic environments circa 1970s). One downside I found is that the obvious implementation thrashes the hell out of your garbage collector. "Environments" really want a data structure that is more complicated and copy-friendly than "cons pairs".
- sparkie 2y ago
  > One interesting bit that remains unappreciated about Kernel is that environments are reified and are "copy-on-write". So, if you make an environment change, it propagates forward but previous calls only see the previous environment unless explicitly give them access. That's not quite how it works. Environments are mutable, but only the locals of an environment that we have a direct reference to. We cannot mutate any of the bindings in the parents without a reference to the parents, and the language provides no facility to obtain references to the parents given only a reference to the child. We can however, explicitly pass around references to parent environments if we do have access to them, because environments are first-class, and we can move around references where we want. We can have a child mutate its grandparent environment, and those changes will be visible to the child's parent - but this is only possible if the child has a direct reference to the grandparent - which it would not have under normal circumstances - it has to be provided with that reference explicitly. We could for example, bind our top-level scope into a symbol in the top-level, such that any descendants of the top level can mutate any bindings in the top level. ($define! *globals* (get-current-environment)) ($define! x 0) ($define! print-x ($lambda () (write x))) ($define! foo ($lambda () ($set! *globals* x 1))) (foo) (print-x) Here we see that the lambda defined for `print-x` does not get an immutable copy of the top-level scope, because if that were the case, then the `x` printed by `print-x` would be a different `x` that foo mutates, and `print-x` would still print 0, but this should print 1. Obviously, we would not do this in practice because it would let us rebind `print-x` and `foo`, or even rebind `*globals*` to some another value. A safer way to do this would be to bind a setter for x in the top level. ($define! x 0) ($define! $set-x! (($vau () top-level ($vau (value) #ignore ($set! top-level x value))))) ($define! print-x ($lambda () (write x))) ($define! foo ($lambda () ($set-x! 1))) (foo) (print-x) This would allow any descendants of the top-level to mutate `x`, but not mutate any other binding in the top level. This works because the outermost `$vau` on $set-x!` is evaluated in the top level, where `$set-x!` is defined, so it receives the top level environment as its dynamic environment, which it binds to the symbol `top-level` in its local environment, and then returns an operative whose static environment is its own local environment. When `$set-x!` is called, it ignores its caller's dynamic environment, but its static environment contains the symbol `top-level`, bound to the top level environment. --- The immutable part of environments is the list of parents that any environment has. When we create an environment through say `(make-environment list-of-parents)`, the environment creates an immutable copy of the list of parents, which prevents whoever created the environment from being able to modify this list later. For example, we wouldn't want this ($define! parents (list some-environment)) ($define! env (make-environment parents)) ($set-car! parents some-other-environment) The caller mutates `parents` after creating `env`, but this will not affect `env`, because it created an immutable copy of `parents`. However, the creator of this list can mutate the bindings in the parent after creating the env. ($define! parents (list some-environment)) ($define! env (make-environment parents)) ($set! some-environment some-binding some-value) This would be reflected in `env`, whose `some-binding` would now resolve to `some-value`. But usually it will be the case that `some-environment` is encapsulated by the creator who will eventually return `env`, but no reference to `some-environment` - which will prevent any mutation of the bindings in the parents. --- There is no way to mutate the ground environment because every standard-environment is a child of the ground, but with no direct reference to the ground. The ground is just its parent. The definition of `make-kernel-standard-environment` is interesting, because it's just ($define! make-kernel-standard-environment ($lambda () (get-current-environment))) Because Kernel specifies that ground bindings are evaluated in the ground environment, and the lambda creates a child of the ground with no initial local bindings, then when we get that environment, we have an empty environment with ground as its parent. Although Kernel is specified using mutable environments, the Kernel report does make them optional. It doesn't really go into depth about how one would implement Kernel with immutable environments besides a brief note. If you do have immutable environments, then a fun challenge is how you would implement `make-kernel-standard-environment`. If using the definition above, then the environment returned by the call to `make-kernel-standard-environment` notably does not have the binding `make-kernel-standard-environment` in it (because the binding happens after).

bsima 2y ago

If you like this, you'll love Shen https://shenlanguage.org/ https://shenlanguage.org/

shadowgovt 2y ago

Interesting! Racket has the capacity to declare DSLs that are then applied at the file level, but this is even finer-grained.

Kimbsy 2y ago

Hi this is my project! I'm giving a talk on this in May for London Clojurians, and I'll also be talking about it at Lambda Days in Krakow in June. https://www.meetup.com/london-clojurians/events/306843409/ https://www.meetup.com/london-clojurians/events/306843409/ The project was initially inspired by Dr John Sturdy's thesis "A Lisp through the Looking Glass" which is all about interpreter towers and the ability to modify them in both directions. This is all just for fun, I've yet to think of anything truly useful you can do with this. If you have any cool ideas please let me know, they might even make it into my talk!

ludston 2y ago

You can actually get completely stupid with this in Common Lisp using something called a "Reader Macro" which lets you temporarily take complete control over the interpreter. For example, I have this joke project that defines a DSL for fizzbuzz: https://github.com/DanielKeogh/fizzbuzz https://github.com/DanielKeogh/fizzbuzz