6 ms·

uLisp – ARM Assembler in Lisp

172 points by eggy 7y ago

siraben 7y ago
I find Lisps to be very well suited towards this task, because of its homoiconicity and features like quasiquotation, effectively you have access to the best macro system possible. Related: Z80 assembler in Scheme https://github.com/siraben/zkeme80/blob/master/src/assembler.scm https://github.com/siraben/zkeme80/blob/master/src/assembler...
- speps 7y ago
  You might have an idea on this but how would you approach the opposite operation? Assembly to an in-memory representation, how does Lisp/Scheme handle that kind of manipulation?
  - siraben 7y ago
    Do you mean how opcodes are represented in Lisp-based assemblers?
    - speps 7y ago
      No I mean a disassembler, I can see how an assembler is nice to write in a language like that but I've never seen much byte manipulation examples. For example going from machine code to the list of opcodes for example.
      - siraben 7y ago
        That would be an interesting project. AFAIK Emacs has a bytecode disassembler[0] but I'm not aware of disassemblers for other instruction sets. [0] https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/disass.el https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-...
      - pjmlp 7y ago
        There you go, "CHIP-8 in Common Lisp: Disassembly" https://stevelosh.com/blog/2017/01/chip8-disassembly/ https://stevelosh.com/blog/2017/01/chip8-disassembly/
      - lispm 7y ago
        Every Common Lisp has a DISASSEMBLE function built-in. Compile a function and call the disassembler. For example SBCL https://github.com/sbcl/sbcl/blob/master/src/compiler/disassem.lisp https://github.com/sbcl/sbcl/blob/master/src/compiler/disass... and its ARM64 instructions: https://github.com/sbcl/sbcl/blob/master/src/compiler/arm64/insts.lisp https://github.com/sbcl/sbcl/blob/master/src/compiler/arm64/...
        dreamcompiler 7y ago
        Here's the times13 function in CCL on x86-64. (Note that #x68 is 13 shifted left 3 times to allow tag bits in the LS 3 bit positions. Also the below could be optimized a lot more but this is the default compilation.) ? (defun times13 (x) (* 13 x)) TIMES13 ? (disassemble 'times13) ;;; (defun times13 (x) (* 13 x)) (recover-fn-from-rip) ; [0] (cmpl ($ 8) (% nargs)) ; [7] (jne L41) ; [10] (pushq (% rbp)) ; [12] (movq (% rsp) (% rbp)) ; [13] (pushq (% arg_z)) ; [16] ;;; (* 13 x) (movl ($ #x68) (% arg_y.l)) ; [17] (movq (@ -8 (% rbp)) (% arg_z)) ; [22] (movq (% rbp) (% rsp)) ; [26] (popq (% rbp)) ; [29] (jmp (@ .SPBUILTIN-TIMES)) ; [30] ;;; #<no source text> (:align 3) L41 ; [@56] (uuo-error-wrong-number-of-args) ; [41] NIL
  - aidenn0 7y ago
    With the BYTE accessors and binary constants, I find bit and byte manipulation to be far easier in common-lisp than in C. e.g. set the 3 bytes starting at bit 2 of x to the binary value 110: (setf (ldb (byte 3 2) x) #b110) or set the 3rd byte (8 bits starting at bit 24) of x to 0xab: (setf (ldb (byte 8 24) x) #xab) The first example compiles down to the following 2 instructions on sbcl/x64: ; A4: 4883E1C6 AND RCX, -58 ; A8: 4883C930 OR RCX, 48 NOTE: The values are 1 bit shifted from what you might expect (binary 110 = 6; 6 << 2 == 24, not 48) because SBCL uses low-bit tagging to distinguish integers from pointers; working with arrays of unboxed machine-friendly sized integers (8,32,64) works as expected.
- bitwize 7y ago
  I once threw together a 6502 assembler (really a front end to x65) in Scheme to be able to write Commodore programs with powerful Lisp-like macro facilities.
  - kick 7y ago
    That reminds me of a great anecdote from an old thread: In 1983, Rescue on Fractalus and Ballblazer, the first releases from Lucasfilm Games, were built with a system written in Lisp. It was a 6502 assembler with Lisp syntax and a Lisp-style macro facility, written in Portable Standard Lisp, running in Berkeley Unix on a 4MB VAX 11/750. Unfortunately it was eventually abandoned because the developer had left and it was a bit of a strain on the VAX that was shared by the whole development team. Yes, I wrote it. Yes, it was my first non-academic programming job. Yes, the users complained about the parentheses, and the slowness. But, they also took advantage of the powerful macro facility. https://news.ycombinator.com/item?id=2475210 https://news.ycombinator.com/item?id=2475210
tieze 7y ago
Ah yea, I did something like this on top of Common Lisp, but not as interactive yet: https://github.com/stuij/armish https://github.com/stuij/armish My plan at the time was to use a serial protocol and interface with a Nintendo GBA/DS for this kind of interactive Lisp programming. It feels a bit dirty mixing implementation and target language like this, but it's great fun to program for.
lebuffon 7y ago
Nice move by uLisp. It's powerful to re-code time critical snippets of code in Assembler, while still having access to a high level language like Lisp for the complex stuff. It might interest people to know that Forth always offered this functionality and the ':' compiler lets one build ASM macros as well.
- lispm 7y ago
  It's not unusual. Lots of Lisp systems support inline assembler or inline C. Historically Lisp systems stored compiled code as LAP (Lisp Assembly Program) and loaded these files via its built-in assembler. The old Lisp 1.5 manual documents an in-memory assembler in Appendix C, Page 73 http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf http://www.softwarepreservation.org/projects/LISP/book/LISP%...
- bordercases 7y ago
  I was wondering why one person on Hackaday thought Forth was uniquely suited for exploration in embedded systems programming other than also being a stack language like Assembly. Thanks for pointing this out.
jimbokun 7y ago
Knowing nothing about uLisp beyond glancing at the code snippets, this seems like it would be a fun tool to use for a CS class on low level programming and compilers.
eggy 7y ago
The author also has a uLisp to C converter and sensor library in uLisp. Very useful and fun!
- eggy 7y ago
  I came upon uLisp over a few years ago, and was glad to see it show up for the ESP8266 chips. It makes programming these tiny controllers with WiFi enjoyable. Simple things like leaving the parameters blank in a WiFi connect function disconnects it from any current connection. It also blends in with my love of the minimal. I had been playing with Wasp Lisp [1,2] at the time when I found uLisp. [1] https://github.com/swdunlop/WaspVM https://github.com/swdunlop/WaspVM [2] https://github.com/doublec/WaspVM https://github.com/doublec/WaspVM
cmrdporcupine 7y ago
Can anybody speak to how hard uLisp would be to port to RISC-V? It looks pretty nifty. I have a little hobby computer I'm building around a RISC-V core, it might be nice to have it boot to this.
- lispm 7y ago
  I don't think it's too difficult. It's written in C and it has also been ported to ARM...
  - cmrdporcupine 7y ago
    Thanks, just read through the source (https://github.com/technoblogy/ulisp/blob/master/ulisp.ino https://github.com/technoblogy/ulisp/blob/master/ulisp.ino) and it looks do-able.
- johnsondavies 7y ago
  See Porting uLisp to a new platform: http://www.ulisp.com/show?2JZO http://www.ulisp.com/show?2JZO.
exdsq 7y ago
I asked a question on HN but didn't get any answers - this might be an easier forum to get one. I'm working on a hobbyist rocketry project where I'm helping build the flight computer - basically an Arduino Nano connected to modules like a GPS system, accelerator, gyroscope, etc. Are there any Lisp dialects that someone would recommend for an Arduino? The Nano is technically replaceable so if there are better boards that can use Lisp I'd be interested too!
- nerdponx 7y ago
  Not firsthand experience, but what about one of the Scheme dialects that compiles to C?
- mapcars 7y ago
  I don't think some specific lisp is required, the one which compiles to C should be good enough. Active example is https://gitlab.com/owl-lisp/owl https://gitlab.com/owl-lisp/owl
  - antoinealb 7y ago
    I don't know anything about owl-lisp, but you also need a lisp that does not do any runtime memory allocation, as those are usually problematic in embedded projects, especially in those with as little RAM as the nano.
    - pjmlp 7y ago
      If they have more memory than an IBM 704, it isn't that problematic.
      - exdsq 7y ago
        After installing uLisp I have 518 bytes of RAM and 1050 bytes of storage on my board - I'd kill for the sweet, sweet 18,432 bytes of an IBM 704! ;)
- cat199 7y ago
monocasa 7y ago
Neat reminds me of Game Oriented Assembly Lisp, the language used for a bunch of the Naughty Dog games. It included inline assemblers for the various processors in the PS2, and was used in a very procedural, macro heavy style that this seems to be encouraging. Neat! Here's an example of GOAL code with inline code for the vector processors. https://web.archive.org/web/20070127022728/http://lists.midnightryder.com/pipermail/sweng-gamedev-midnightryder.com/2005-August/003804.html https://web.archive.org/web/20070127022728/http://lists.midn...
kristianp 7y ago
The call function has no parameters. Does this mean you can only have one assembled function at a time?
- kristianp 7y ago
  Or can you use a closure to hold the assembled code as a context to a local instance of the call method?
- johnsondavies 7y ago
  Yes, it currently provides one assembler function, but I plan to extend it to support multiple named functions.

172 points by eggy 7y ago

I find Lisps to be very well suited towards this task, because of its homoiconicity and features like quasiquotation, effectively you have access to the best macro system possible. Related: Z80 assembler in Scheme https://github.com/siraben/zkeme80/blob/master/src/assembler.scm https://github.com/siraben/zkeme80/blob/master/src/assembler...

speps 7y ago
You might have an idea on this but how would you approach the opposite operation? Assembly to an in-memory representation, how does Lisp/Scheme handle that kind of manipulation?
- siraben 7y ago
  Do you mean how opcodes are represented in Lisp-based assemblers?
  - speps 7y ago
    No I mean a disassembler, I can see how an assembler is nice to write in a language like that but I've never seen much byte manipulation examples. For example going from machine code to the list of opcodes for example.
    - siraben 7y ago
      That would be an interesting project. AFAIK Emacs has a bytecode disassembler[0] but I'm not aware of disassemblers for other instruction sets. [0] https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/disass.el https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-...
    - pjmlp 7y ago
      There you go, "CHIP-8 in Common Lisp: Disassembly" https://stevelosh.com/blog/2017/01/chip8-disassembly/ https://stevelosh.com/blog/2017/01/chip8-disassembly/
    - lispm 7y ago
      Every Common Lisp has a DISASSEMBLE function built-in. Compile a function and call the disassembler. For example SBCL https://github.com/sbcl/sbcl/blob/master/src/compiler/disassem.lisp https://github.com/sbcl/sbcl/blob/master/src/compiler/disass... and its ARM64 instructions: https://github.com/sbcl/sbcl/blob/master/src/compiler/arm64/insts.lisp https://github.com/sbcl/sbcl/blob/master/src/compiler/arm64/...
      - dreamcompiler 7y ago
        Here's the times13 function in CCL on x86-64. (Note that #x68 is 13 shifted left 3 times to allow tag bits in the LS 3 bit positions. Also the below could be optimized a lot more but this is the default compilation.) ? (defun times13 (x) (* 13 x)) TIMES13 ? (disassemble 'times13) ;;; (defun times13 (x) (* 13 x)) (recover-fn-from-rip) ; [0] (cmpl ($ 8) (% nargs)) ; [7] (jne L41) ; [10] (pushq (% rbp)) ; [12] (movq (% rsp) (% rbp)) ; [13] (pushq (% arg_z)) ; [16] ;;; (* 13 x) (movl ($ #x68) (% arg_y.l)) ; [17] (movq (@ -8 (% rbp)) (% arg_z)) ; [22] (movq (% rbp) (% rsp)) ; [26] (popq (% rbp)) ; [29] (jmp (@ .SPBUILTIN-TIMES)) ; [30] ;;; #<no source text> (:align 3) L41 ; [@56] (uuo-error-wrong-number-of-args) ; [41] NIL
- aidenn0 7y ago
  With the BYTE accessors and binary constants, I find bit and byte manipulation to be far easier in common-lisp than in C. e.g. set the 3 bytes starting at bit 2 of x to the binary value 110: (setf (ldb (byte 3 2) x) #b110) or set the 3rd byte (8 bits starting at bit 24) of x to 0xab: (setf (ldb (byte 8 24) x) #xab) The first example compiles down to the following 2 instructions on sbcl/x64: ; A4: 4883E1C6 AND RCX, -58 ; A8: 4883C930 OR RCX, 48 NOTE: The values are 1 bit shifted from what you might expect (binary 110 = 6; 6 << 2 == 24, not 48) because SBCL uses low-bit tagging to distinguish integers from pointers; working with arrays of unboxed machine-friendly sized integers (8,32,64) works as expected.
bitwize 7y ago
I once threw together a 6502 assembler (really a front end to x65) in Scheme to be able to write Commodore programs with powerful Lisp-like macro facilities.
- kick 7y ago
  That reminds me of a great anecdote from an old thread: In 1983, Rescue on Fractalus and Ballblazer, the first releases from Lucasfilm Games, were built with a system written in Lisp. It was a 6502 assembler with Lisp syntax and a Lisp-style macro facility, written in Portable Standard Lisp, running in Berkeley Unix on a 4MB VAX 11/750. Unfortunately it was eventually abandoned because the developer had left and it was a bit of a strain on the VAX that was shared by the whole development team. Yes, I wrote it. Yes, it was my first non-academic programming job. Yes, the users complained about the parentheses, and the slowness. But, they also took advantage of the powerful macro facility. https://news.ycombinator.com/item?id=2475210 https://news.ycombinator.com/item?id=2475210

tieze 7y ago

Ah yea, I did something like this on top of Common Lisp, but not as interactive yet: https://github.com/stuij/armish https://github.com/stuij/armish My plan at the time was to use a serial protocol and interface with a Nintendo GBA/DS for this kind of interactive Lisp programming. It feels a bit dirty mixing implementation and target language like this, but it's great fun to program for.

lebuffon 7y ago

Nice move by uLisp. It's powerful to re-code time critical snippets of code in Assembler, while still having access to a high level language like Lisp for the complex stuff. It might interest people to know that Forth always offered this functionality and the ':' compiler lets one build ASM macros as well.

lispm 7y ago
It's not unusual. Lots of Lisp systems support inline assembler or inline C. Historically Lisp systems stored compiled code as LAP (Lisp Assembly Program) and loaded these files via its built-in assembler. The old Lisp 1.5 manual documents an in-memory assembler in Appendix C, Page 73 http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf http://www.softwarepreservation.org/projects/LISP/book/LISP%...
bordercases 7y ago
I was wondering why one person on Hackaday thought Forth was uniquely suited for exploration in embedded systems programming other than also being a stack language like Assembly. Thanks for pointing this out.

jimbokun 7y ago

Knowing nothing about uLisp beyond glancing at the code snippets, this seems like it would be a fun tool to use for a CS class on low level programming and compilers.

eggy 7y ago

The author also has a uLisp to C converter and sensor library in uLisp. Very useful and fun!

eggy 7y ago
I came upon uLisp over a few years ago, and was glad to see it show up for the ESP8266 chips. It makes programming these tiny controllers with WiFi enjoyable. Simple things like leaving the parameters blank in a WiFi connect function disconnects it from any current connection. It also blends in with my love of the minimal. I had been playing with Wasp Lisp [1,2] at the time when I found uLisp. [1] https://github.com/swdunlop/WaspVM https://github.com/swdunlop/WaspVM [2] https://github.com/doublec/WaspVM https://github.com/doublec/WaspVM

cmrdporcupine 7y ago

Can anybody speak to how hard uLisp would be to port to RISC-V? It looks pretty nifty. I have a little hobby computer I'm building around a RISC-V core, it might be nice to have it boot to this.

lispm 7y ago
I don't think it's too difficult. It's written in C and it has also been ported to ARM...
- cmrdporcupine 7y ago
  Thanks, just read through the source (https://github.com/technoblogy/ulisp/blob/master/ulisp.ino https://github.com/technoblogy/ulisp/blob/master/ulisp.ino) and it looks do-able.
johnsondavies 7y ago
See Porting uLisp to a new platform: http://www.ulisp.com/show?2JZO http://www.ulisp.com/show?2JZO.

exdsq 7y ago

I asked a question on HN but didn't get any answers - this might be an easier forum to get one. I'm working on a hobbyist rocketry project where I'm helping build the flight computer - basically an Arduino Nano connected to modules like a GPS system, accelerator, gyroscope, etc. Are there any Lisp dialects that someone would recommend for an Arduino? The Nano is technically replaceable so if there are better boards that can use Lisp I'd be interested too!

nerdponx 7y ago
Not firsthand experience, but what about one of the Scheme dialects that compiles to C?
mapcars 7y ago
I don't think some specific lisp is required, the one which compiles to C should be good enough. Active example is https://gitlab.com/owl-lisp/owl https://gitlab.com/owl-lisp/owl
- antoinealb 7y ago
  I don't know anything about owl-lisp, but you also need a lisp that does not do any runtime memory allocation, as those are usually problematic in embedded projects, especially in those with as little RAM as the nano.
  - pjmlp 7y ago
    If they have more memory than an IBM 704, it isn't that problematic.
    - exdsq 7y ago
      After installing uLisp I have 518 bytes of RAM and 1050 bytes of storage on my board - I'd kill for the sweet, sweet 18,432 bytes of an IBM 704! ;)
cat199 7y ago

monocasa 7y ago

Neat reminds me of Game Oriented Assembly Lisp, the language used for a bunch of the Naughty Dog games. It included inline assemblers for the various processors in the PS2, and was used in a very procedural, macro heavy style that this seems to be encouraging. Neat! Here's an example of GOAL code with inline code for the vector processors. https://web.archive.org/web/20070127022728/http://lists.midnightryder.com/pipermail/sweng-gamedev-midnightryder.com/2005-August/003804.html https://web.archive.org/web/20070127022728/http://lists.midn...

kristianp 7y ago

The call function has no parameters. Does this mean you can only have one assembled function at a time?

kristianp 7y ago
Or can you use a closure to hold the assembled code as a context to a local instance of the call method?
johnsondavies 7y ago
Yes, it currently provides one assembler function, but I plan to extend it to support multiple named functions.