9 ms·

ChrysaLisp

121 points by ngcc_hk 7y ago

smabie 7y ago
This is cool, but the code is... strange? https://github.com/vygr/ChrysaLisp/blob/master/gui/canvas/tga.vp https://github.com/vygr/ChrysaLisp/blob/master/gui/canvas/tg... The code doesn’t look like Lisp at all. Is this some sort of lower assembler language that Lisp is implemented on top of?
- dasmoth 7y ago
  See https://github.com/vygr/ChrysaLisp/blob/master/docs/VM.md https://github.com/vygr/ChrysaLisp/blob/master/docs/VM.md
- pdexter 7y ago
  https://github.com/vygr/ChrysaLisp/blob/master/apps/terminal/app.lisp https://github.com/vygr/ChrysaLisp/blob/master/apps/terminal... for example
- vygr 7y ago
  That code is using the C-Script assembler prototyping macros. It will produce VP assembler output, but lets you work without worrying about registers while you're getting stuff going. Have a read through the docs/ASSIGNMENT.md file to see how this all fits together.
- capableweb 7y ago
  Looks like imperative/sequential lisp to me. Got those parenthesis, quoting and some light-weight nesting. Anything in particular that sticks out to? Lack of recursion?
- TheAsprngHacker 7y ago
  As someone who isn't familiar with this Lisp dialect / DSL, something that stands out to me is the (def-struct ...) ... (def-struct-end), (def-method ...) ... (def-func-end), (switch) ... (endswitch), and (vpif ...) ... (endif). The appeal of s-expressions is that their nested structure is supposed to delineate the abstract syntax tree of the program, with the opening and closing parentheses denoting the boundaries of a construct. IMO this language's syntax isn't faithful to the spirit of Lisp.
  - vygr 7y ago
    The (def-struct) stuff is part of the assembler ! Not the Lisp. Same with (def-enum) etc. I am going to have to write a document to clearly state where these boundaries between the Lisp, the C-Script and the VP layers. I realise it's not that obvious sometimes. Chris
    - Koshkin 7y ago
      (def-func-end) and other ‘ends’ look pretty un-lispy to me. Helps readability, though.
pkphilip 7y ago
This is pretty amazing! would appreciate some more documentation on this and perhaps some benchmarks.
- vygr 7y ago
  There is a start on documentation in the docs/ folder. I know it's sparse at present I do aim to add more over time. The system can build itself from source after clean, on my 2014 Macbook in under 0.4s. It's not exactly a standard benchmark though, but it's not slow considering that's a Lisp (like) interpreter doing all the compiling and assembling !. Footprint at the moment for everything is 158KB. The snapshot.zip file contains 3 separate snapshots for the 3 ABIs supported currently.
classified 7y ago
Very confusing. It's not actually Lisp but some new language with superficially Lisp-like syntax. It doesn't say what it's for or why the hell anybody would want to use it.
- _pmf_ 7y ago
  > It's not actually Lisp but some new language with superficially Lisp-like syntax. So, like Clojure.
  - DannyB2 7y ago
    Why Clojure? Clojure has cons, car, cdr which you can use in the classic Lisp sense. The only thing you CAN'T do is have a dotted pair or dotted list in the last cons cell. One might argue that you cannot surgically alter data structures in Clojure. But you can! It's just that the function doing the alteration returns an entirely new data structure with the alteration in place -- but without the expected inefficiency of a copy operation. If you have an array of a billion elements, and alter one of the elements, you get back a new array with the alteration. But it is not a COPY of the entire array (with the expected time required to copy). The original array without the alteration also still exists -- but you don't have twice the memory usage now that there seem to be two slightly different arrays with these billion elements. Yet accessing or altering any element in the array has close to the performance you would expect of an actual array implementation.
    - MadWombat 7y ago
      Tail recursion? As far as I know, Clojure doesn't and cannot have it because JVM doesn't support it.
      - holtalanm 7y ago
        don't they get around that by compiling the tailrec to bytecode as a while loop? other languages supporting tail recursion on the jvm do that. I don't know enough about clojure to say that is how they do it, though.
nizmow 7y ago
Previously: https://news.ycombinator.com/item?id=15466124 https://news.ycombinator.com/item?id=15466124
tartoran 7y ago
Taken from the documenation: Lisp It's probably worth a few words specifically about the included Lisp and how it works, and how many rules it breaks ! The reason for doing the Lisp was to allow me to create an assembler to replace NASM, I was not concerned with sticking to 'the lisp way', or whatever your local Lisp guru says. No doubt I will have many demons in hell awaiting me... First of all there is no garbage collector by choice. All objects used by the Lisp are reference counted objects from the class library. Early on the Lisp was never going to have a problem with cycles because it had no way to create them, but as I developed the assembler I decided to introduce two functions, push and elem-set, that could create cycles. However the efficiency advantage in coding the assembler made me look the other way. There are now other ways that a cycle can be created, by naming an environment within its own scope, but again this was too good an efficiency feature to miss out on. So you do have to be careful not to create cycles, so think about how your code works. No tail recursion optimization ! There is a single looping function provided in native code, while, every other looping construct builds on this primitive. There are also two native primitives some! and each! that provide generic access to iterating over a slice of a sequence/s, while calling a function on the grouped elements. Standard some and each are built on these but they also allow other constructs to be built and gain the advantage of machine coded iteration. I try to stick to a functional approach in my Lisp code, and manipulate collections of things in a functional way with operations like map, filter, reduce, each etc. I've not found the lack of tail recursion a problem. All symbols live in the same environment, functions, macros, everything. The environment is a chain of hash maps. Each lambda gets a new hash map pushed onto the environment chain on invocation, and dereferenced on exit. The env function can be used to return the current hash map and optionally resize the number of buckets from the default of 1. This proves very effective for storing large numbers of symbols and objects for the assembler as well as creating caches. Make sure to setq the symbol you bind to the result of env to nil before returning from the function if you do this, else you will create a cycle that can't be freed. defq and bind always create entries in the top environment hash map. setq searches the environment chain to find an existing entry and sets that entry or fails with an error. This means setq can be used to write to symbols outside the scope of the current function. Some people don't like this, but used wisely it can be very powerful. Coming from an assembler background I prefer to have all the guns and knives available, so try not to shoot your foot off. There is no cons, cdr or car stuff. Lists are just vector objects and you use push, cat, slice etc to manipulate elements. Also an empty list does not evaluate to nil, it's just an error. Function and macro definitions are scoped and visible only within the scope of the declaring function. There is no global macro list. During macro expansion the environment chain is searched to see if a macro exists.
tartoran 7y ago
This is quite an intersting and approchable OS. Hope to see progress on it. Could this be run this in VMWare? Or is it specifically bound to a host OS?
- vygr 7y ago
  It's not bound to any host OS. I am using SDL to provided a simple abstraction to open a Window and obtain key and mouse input. I use very little of the SDL features, just a rectangle fill and rectangle bit blit. See the gui/ctx/class.inc for all those, everything else in the GUI, like the compositor and vector graphics library, widgets et al are done in VP on the CrysaLisp side. I have a small abstraction to the host OS via the sys/pii/functions, these just use the jump table you can see in the main.c file. Any host OS should be able to compile the main.c with little or no changes, but eventually there will be a native filesystem available from VP/Lisp. I will be doing direct bare metal hardware support, and there are things happening on that front but I can't talk about that here. But during my experimentation stage it was very useful to have it running hosted, and I'll keep the hosted version running going forward as it provides a nice dev environment. I do have a plan to get a version done that could boot under VMWare etc, but I've only got so much time on my hands and have to spread myself over the whole codebase as best I can. :)
DannyB2 7y ago
One interesting thing is the idea that app applications and utilities are in a virtual object code rather than native code. This reminds me of the UCSD p-System of the early 1980's. To port the OS all you had to port was the very small p-Machine emulator (PME). Once you had a bootable PME, everything else, the entire OS, utilities, and all applications instantly came along for the ride, without even being recompiled. I could only imagine a modern OS that would take an approach like this, but Hotspot JIT everything to native code.
- bitwize 7y ago
  Inferno could do this. Port the Dis VM and whamo, you had all of Inferno. On Windows, Linux, bare metal, x86, RISC... pick your poison.
- phyrex 7y ago
  I mean that’s basically the JVM, isn’t it?
  - antoncohen 7y ago
    There was JavaOS: https://en.wikipedia.org/wiki/JavaOS https://en.wikipedia.org/wiki/JavaOS
    - vygr 7y ago
      https://en.wikipedia.org/wiki/Virtual_Processor https://en.wikipedia.org/wiki/Virtual_Processor Taos articles link at the bottom, Byte, IEEE, Edge etc.
  - DannyB2 7y ago
    Yes. As a long time Java developer. But the UCSD p-System was much earlier (and more primitive) than Java. p-System never did JIT. Remember this was for machines with 64 K of memory. Not 64 MB. But 64 K! The p-Code was much more compact than native code. Interpreting most code was plenty fast for most things -- even on the slow (under 10 MHz) clock speeds of the era. Critical operations could be written in assembler. The p-System had its own assembler for native code. JVM is a whole other world. Adding GC is a whole new dimension in complexity. But decades of research have gone into the JVM making it the amazing runtime platform it is today.
vygr 7y ago
Thanks again folks for the interest. You catch me yet again when I’m at work but I’ll try answer question in an hour or so once I get home. Regards to all. Chris
- vygr 7y ago
  Just a quick comment till I can get home. There are three main layers of in use. The more Lisp like layer in apps/ and cmd/. The lowest level VP assembler and an assembler with stack variables and expression compiler I call C-Script. This is my hobby and a follow up to Taos that I did back in the 90’s. See ‘Virtual Processor’ page on Wikipedia for link articles of various scans from back then. Got to head into the car now.....
dang 7y ago
For the curious: 2019 https://news.ycombinator.com/item?id=18897817 https://news.ycombinator.com/item?id=18897817 2017 https://news.ycombinator.com/item?id=15466124 https://news.ycombinator.com/item?id=15466124 Related, on the TAOS operating system: 2018 https://news.ycombinator.com/item?id=16727565 https://news.ycombinator.com/item?id=16727565 2017 https://news.ycombinator.com/item?id=15527936 https://news.ycombinator.com/item?id=15527936 2015 https://news.ycombinator.com/item?id=9806607 https://news.ycombinator.com/item?id=9806607 (with lots of comments by original contributors)
zapperdapper 7y ago
It's real deja vu looking at this thread. Many of the questions/comments here are exactly the same ones that came up in the early 1990s with Taos on CiX, and then later with Elate/intent. :) Chris was far ahead of the curve - a heterogeneous, load-balancing, multi-processor, multi-tasking OS, with Object-Oriented Virtual Processor, byte code, load-time translation, support for multiple CPU types (including RISC/CISC/transputer and LE and BE), and tiny footprint message-passing kernel, when most people used MS-DOS 3.3 or DR-DOS 5.0, with Windows 95 still a gleam in Bill Gates's beady eye! Nearly 30 years on and he is still ahead of the curve! p.s. no one has mentioned it yet - try a Google search for "edge tao spyfish". That was 1995!! Not released, but still very interesting reading material.

121 points by ngcc_hk 7y ago

This is cool, but the code is... strange? https://github.com/vygr/ChrysaLisp/blob/master/gui/canvas/tga.vp https://github.com/vygr/ChrysaLisp/blob/master/gui/canvas/tg... The code doesn’t look like Lisp at all. Is this some sort of lower assembler language that Lisp is implemented on top of?

dasmoth 7y ago
See https://github.com/vygr/ChrysaLisp/blob/master/docs/VM.md https://github.com/vygr/ChrysaLisp/blob/master/docs/VM.md
pdexter 7y ago
https://github.com/vygr/ChrysaLisp/blob/master/apps/terminal/app.lisp https://github.com/vygr/ChrysaLisp/blob/master/apps/terminal... for example
vygr 7y ago
That code is using the C-Script assembler prototyping macros. It will produce VP assembler output, but lets you work without worrying about registers while you're getting stuff going. Have a read through the docs/ASSIGNMENT.md file to see how this all fits together.
capableweb 7y ago
Looks like imperative/sequential lisp to me. Got those parenthesis, quoting and some light-weight nesting. Anything in particular that sticks out to? Lack of recursion?
TheAsprngHacker 7y ago
As someone who isn't familiar with this Lisp dialect / DSL, something that stands out to me is the (def-struct ...) ... (def-struct-end), (def-method ...) ... (def-func-end), (switch) ... (endswitch), and (vpif ...) ... (endif). The appeal of s-expressions is that their nested structure is supposed to delineate the abstract syntax tree of the program, with the opening and closing parentheses denoting the boundaries of a construct. IMO this language's syntax isn't faithful to the spirit of Lisp.
- vygr 7y ago
  The (def-struct) stuff is part of the assembler ! Not the Lisp. Same with (def-enum) etc. I am going to have to write a document to clearly state where these boundaries between the Lisp, the C-Script and the VP layers. I realise it's not that obvious sometimes. Chris
  - Koshkin 7y ago
    (def-func-end) and other ‘ends’ look pretty un-lispy to me. Helps readability, though.

pkphilip 7y ago

This is pretty amazing! would appreciate some more documentation on this and perhaps some benchmarks.

vygr 7y ago
There is a start on documentation in the docs/ folder. I know it's sparse at present I do aim to add more over time. The system can build itself from source after clean, on my 2014 Macbook in under 0.4s. It's not exactly a standard benchmark though, but it's not slow considering that's a Lisp (like) interpreter doing all the compiling and assembling !. Footprint at the moment for everything is 158KB. The snapshot.zip file contains 3 separate snapshots for the 3 ABIs supported currently.

classified 7y ago

Very confusing. It's not actually Lisp but some new language with superficially Lisp-like syntax. It doesn't say what it's for or why the hell anybody would want to use it.

_pmf_ 7y ago
> It's not actually Lisp but some new language with superficially Lisp-like syntax. So, like Clojure.
- DannyB2 7y ago
  Why Clojure? Clojure has cons, car, cdr which you can use in the classic Lisp sense. The only thing you CAN'T do is have a dotted pair or dotted list in the last cons cell. One might argue that you cannot surgically alter data structures in Clojure. But you can! It's just that the function doing the alteration returns an entirely new data structure with the alteration in place -- but without the expected inefficiency of a copy operation. If you have an array of a billion elements, and alter one of the elements, you get back a new array with the alteration. But it is not a COPY of the entire array (with the expected time required to copy). The original array without the alteration also still exists -- but you don't have twice the memory usage now that there seem to be two slightly different arrays with these billion elements. Yet accessing or altering any element in the array has close to the performance you would expect of an actual array implementation.
  - MadWombat 7y ago
    Tail recursion? As far as I know, Clojure doesn't and cannot have it because JVM doesn't support it.
    - holtalanm 7y ago
      don't they get around that by compiling the tailrec to bytecode as a while loop? other languages supporting tail recursion on the jvm do that. I don't know enough about clojure to say that is how they do it, though.

nizmow 7y ago

Previously: https://news.ycombinator.com/item?id=15466124 https://news.ycombinator.com/item?id=15466124

tartoran 7y ago

Taken from the documenation: Lisp It's probably worth a few words specifically about the included Lisp and how it works, and how many rules it breaks ! The reason for doing the Lisp was to allow me to create an assembler to replace NASM, I was not concerned with sticking to 'the lisp way', or whatever your local Lisp guru says. No doubt I will have many demons in hell awaiting me... First of all there is no garbage collector by choice. All objects used by the Lisp are reference counted objects from the class library. Early on the Lisp was never going to have a problem with cycles because it had no way to create them, but as I developed the assembler I decided to introduce two functions, push and elem-set, that could create cycles. However the efficiency advantage in coding the assembler made me look the other way. There are now other ways that a cycle can be created, by naming an environment within its own scope, but again this was too good an efficiency feature to miss out on. So you do have to be careful not to create cycles, so think about how your code works. No tail recursion optimization ! There is a single looping function provided in native code, while, every other looping construct builds on this primitive. There are also two native primitives some! and each! that provide generic access to iterating over a slice of a sequence/s, while calling a function on the grouped elements. Standard some and each are built on these but they also allow other constructs to be built and gain the advantage of machine coded iteration. I try to stick to a functional approach in my Lisp code, and manipulate collections of things in a functional way with operations like map, filter, reduce, each etc. I've not found the lack of tail recursion a problem. All symbols live in the same environment, functions, macros, everything. The environment is a chain of hash maps. Each lambda gets a new hash map pushed onto the environment chain on invocation, and dereferenced on exit. The env function can be used to return the current hash map and optionally resize the number of buckets from the default of 1. This proves very effective for storing large numbers of symbols and objects for the assembler as well as creating caches. Make sure to setq the symbol you bind to the result of env to nil before returning from the function if you do this, else you will create a cycle that can't be freed. defq and bind always create entries in the top environment hash map. setq searches the environment chain to find an existing entry and sets that entry or fails with an error. This means setq can be used to write to symbols outside the scope of the current function. Some people don't like this, but used wisely it can be very powerful. Coming from an assembler background I prefer to have all the guns and knives available, so try not to shoot your foot off. There is no cons, cdr or car stuff. Lists are just vector objects and you use push, cat, slice etc to manipulate elements. Also an empty list does not evaluate to nil, it's just an error. Function and macro definitions are scoped and visible only within the scope of the declaring function. There is no global macro list. During macro expansion the environment chain is searched to see if a macro exists.

tartoran 7y ago

This is quite an intersting and approchable OS. Hope to see progress on it. Could this be run this in VMWare? Or is it specifically bound to a host OS?

vygr 7y ago
It's not bound to any host OS. I am using SDL to provided a simple abstraction to open a Window and obtain key and mouse input. I use very little of the SDL features, just a rectangle fill and rectangle bit blit. See the gui/ctx/class.inc for all those, everything else in the GUI, like the compositor and vector graphics library, widgets et al are done in VP on the CrysaLisp side. I have a small abstraction to the host OS via the sys/pii/functions, these just use the jump table you can see in the main.c file. Any host OS should be able to compile the main.c with little or no changes, but eventually there will be a native filesystem available from VP/Lisp. I will be doing direct bare metal hardware support, and there are things happening on that front but I can't talk about that here. But during my experimentation stage it was very useful to have it running hosted, and I'll keep the hosted version running going forward as it provides a nice dev environment. I do have a plan to get a version done that could boot under VMWare etc, but I've only got so much time on my hands and have to spread myself over the whole codebase as best I can. :)

DannyB2 7y ago

One interesting thing is the idea that app applications and utilities are in a virtual object code rather than native code. This reminds me of the UCSD p-System of the early 1980's. To port the OS all you had to port was the very small p-Machine emulator (PME). Once you had a bootable PME, everything else, the entire OS, utilities, and all applications instantly came along for the ride, without even being recompiled. I could only imagine a modern OS that would take an approach like this, but Hotspot JIT everything to native code.

bitwize 7y ago
Inferno could do this. Port the Dis VM and whamo, you had all of Inferno. On Windows, Linux, bare metal, x86, RISC... pick your poison.
phyrex 7y ago
I mean that’s basically the JVM, isn’t it?
- antoncohen 7y ago
  There was JavaOS: https://en.wikipedia.org/wiki/JavaOS https://en.wikipedia.org/wiki/JavaOS
  - vygr 7y ago
    https://en.wikipedia.org/wiki/Virtual_Processor https://en.wikipedia.org/wiki/Virtual_Processor Taos articles link at the bottom, Byte, IEEE, Edge etc.
- DannyB2 7y ago
  Yes. As a long time Java developer. But the UCSD p-System was much earlier (and more primitive) than Java. p-System never did JIT. Remember this was for machines with 64 K of memory. Not 64 MB. But 64 K! The p-Code was much more compact than native code. Interpreting most code was plenty fast for most things -- even on the slow (under 10 MHz) clock speeds of the era. Critical operations could be written in assembler. The p-System had its own assembler for native code. JVM is a whole other world. Adding GC is a whole new dimension in complexity. But decades of research have gone into the JVM making it the amazing runtime platform it is today.

vygr 7y ago

Thanks again folks for the interest. You catch me yet again when I’m at work but I’ll try answer question in an hour or so once I get home. Regards to all. Chris

vygr 7y ago
Just a quick comment till I can get home. There are three main layers of in use. The more Lisp like layer in apps/ and cmd/. The lowest level VP assembler and an assembler with stack variables and expression compiler I call C-Script. This is my hobby and a follow up to Taos that I did back in the 90’s. See ‘Virtual Processor’ page on Wikipedia for link articles of various scans from back then. Got to head into the car now.....

dang 7y ago

For the curious: 2019 https://news.ycombinator.com/item?id=18897817 https://news.ycombinator.com/item?id=18897817 2017 https://news.ycombinator.com/item?id=15466124 https://news.ycombinator.com/item?id=15466124 Related, on the TAOS operating system: 2018 https://news.ycombinator.com/item?id=16727565 https://news.ycombinator.com/item?id=16727565 2017 https://news.ycombinator.com/item?id=15527936 https://news.ycombinator.com/item?id=15527936 2015 https://news.ycombinator.com/item?id=9806607 https://news.ycombinator.com/item?id=9806607 (with lots of comments by original contributors)

zapperdapper 7y ago

It's real deja vu looking at this thread. Many of the questions/comments here are exactly the same ones that came up in the early 1990s with Taos on CiX, and then later with Elate/intent. :) Chris was far ahead of the curve - a heterogeneous, load-balancing, multi-processor, multi-tasking OS, with Object-Oriented Virtual Processor, byte code, load-time translation, support for multiple CPU types (including RISC/CISC/transputer and LE and BE), and tiny footprint message-passing kernel, when most people used MS-DOS 3.3 or DR-DOS 5.0, with Windows 95 still a gleam in Bill Gates's beady eye! Nearly 30 years on and he is still ahead of the curve! p.s. no one has mentioned it yet - try a Google search for "edge tao spyfish". That was 1995!! Not released, but still very interesting reading material.