11 ms·

Why I'm writing a Scheme implementation in 2025: Async Rust

198 points by maplant 2y ago

MathMonkeyMan 2y ago
Nice work! It's a good write-up, too. Hygienic macro expansion is one of the things I still haven't implemented before. I remember a [talk][1] where Matthew Flatt illustrates the idea of sets of scopes, and choosing the largest intersection. I see in your implementation there are sets of "marks" in syntax objects, is that what's going on there? I haven't played with rust, but when I do I'll be able to play with this scheme too. [1]: https://www.youtube.com/watch?v=Or_yKiI3Ha4 https://www.youtube.com/watch?v=Or_yKiI3Ha4
- maplant 2y ago
  I probably should have looked at this talk! Instead my reference was this PhD thesis[1] cited in the R6RS spec. I can’t say I read the whole thing, but my takeaway was to use a system of marks/antimarks. The basic idea is that at the point of macro expansion, the syntax object is “marked” (i.e. added to the set of marks) with a random integer. The environment of the macro is recorded at that point as well, along with the mark used. After expansion, the resulting syntax object is again marked with the same mark. When an identifier is marked twice with the same mark, the marks cancel each other out. The result of this is that identifier that are introduced into the expander and those introduced by the expander are distinguishable. [1]: https://www2.ccs.neu.edu/racket/pubs/dissertation-kohlbecker.pdf https://www2.ccs.neu.edu/racket/pubs/dissertation-kohlbecker...
  - lygaret 2y ago
    The talk goes line by line through this code [1]; I've been transcribing and taking notes on this for the last week, for a somewhat similar project actually. If you're interested, I've got a huge pile of papers and links collected here [2] that you might enjoy. I've read everything, but right now it's still a big ol' mush; there's a _ton_ of prior art! Enjoy, good hacking! [1]: https://github.com/mflatt/expander/tree/pico https://github.com/mflatt/expander/tree/pico [2]: https://github.com/lygaret/gigi?tab=readme-ov-file#reading https://github.com/lygaret/gigi?tab=readme-ov-file#reading
    - -__---____-ZXyw 2y ago
      Lovely pipe of papers and links, thanks for sharing!
  - bjoli 2y ago
    I have said this before, but it deserves repeating: When people say scheme is a simple language, they do not mean the hygienic macro system. Especially not something like psyntax/syntax-case. I am very impressed! The implementation is crazy clear.
    - maplant 2y ago
      Thank you! I’m very proud of the macro expander, so comments like yours are really invigorating and justify the work I put into it
leftyspook 2y ago
I think you make a pretty bad case for how embedding a Scheme interpreter is going to help with the pain points of async. Listing "stack traces full of tokio code" and then seemingly proposing to solve that by adding more glue to pollute the stack traces is especially weird.
- maplant 2y ago
  Not an interpreter! A compiler :-) Have you seen a stack trace originating from somewhere within tokio? Nearly all useful information is lost. My contention is that by isolating the functions that are required to be written in Rust and then doing orchestration, spawning, etc in Scheme the additional debug information at runtime will make the source of errors much more clear. I could be wrong! But hey there’s other reasons too. Being able to debug Rust functions dynamically is pretty cool, as well as being able to start/stop them like daemons.
  - giancarlostoro 2y ago
    On the other hand, you might wind up with an impressive Scheme implementation.
  - kanbankaren 2y ago
    luajit gets you almost all of scheme features except macros. Speed of luajit is very close to native C.
    - dapperdrake 2y ago
      Sounds like Greenspun's tenth rule.
      - kanbankaren 2y ago
        Well, unlike Lisp & Scheme, Lua and Luajit are used in production, millions of video game consoles, embedded systems, etc.
hardwaresofton 2y ago
Glad to see more language runtimes be available from Rust! Looking forward to scheme-rs being able to benefit from the safety and speed Rust allows
- dapperdrake 2y ago
  It’s about instrumenting Rust’s existing run-time.
  - hardwaresofton 2y ago
    Are we reading the same article? It's about this project: https://crates.io/crates/scheme-rs https://crates.io/crates/scheme-rs
    - dapperdrake 2y ago
      So? There is instrumentation for Async Rust being added. Just not in the Rust compiler and not in tokio. More in application level code. So as far as the user is concerned to "their" Rust run-time. A distinction that is mostly useless when programming lisp. Hence lisp. Lisp is about solving your problem and getting the "language" out of the way. So you add what is missing. As long as it works it doesn’t even matter where it’s added. Even C has a barebones runtime environment when compiled for a machine with an MMU and an OS, so not a microcontroller.
reikonomusha 2y ago
Coalton [1] is a Lisp language that has Hindley-Milner type inference with type classes. Its syntax actually resembles the prototype syntax from TFA pretty closely. The Coalton syntax-equivalent would be: (define-type (Option :a) (Some :a) None) (let ((x (Some 5))) (match x ((Some y) y) ((None) (random)))) (declare random (Distribution :a => Unit -> :a)) (define (random) (sample (thread-rng))) (define-class (Eq :t) (= (:t -> :t -> Boolean))) (define-instance (Eq Number) (define (= lhs rhs) (num-equal lhs rhs))) Of course types and classes like Option and Eq are built-in already. Coalton is based on Common Lisp (and allows free mixing with CL code) instead of Scheme however, though Coalton is a Lisp-1 and would feel relatively at-home to any Scheme programmer. [1] https://github.com/coalton-lang/coalton https://github.com/coalton-lang/coalton
- maplant 2y ago
  Very cool! I saw a number of projects pretty similar to this one, typed racket for example. I didn’t really talk about it too much but what I want to do is flip the direction. Coalton is based on CL, but what I would like is scheme to essentially be a set of Gouki function with the signature top … -> top. Then, i want to see how fast I can make the interaction between them be using different analysis like k-CFA
- anonzzzies 2y ago
  Coalton is great. Indeed often it's much easier to just implement on top of either SBCL or ChezScheme; they are very fast and chances are you are not going to make anything as or more performant, better documented etc. However, reimplementation in Rust is not too bad as that's were we need to be for small, fast, low level runtimes. If the goal was 'just' their own language, I feel just a an implementation on top of CL/SBCL + Coalton would've been faster to build, easier to maintain and probably far more performant. Less fun though!
markstos 2y ago
The Helix editor, a popular alternative to Vim, is going to implementing it's plugin system in a Scheme-like language. Helix is also written in Rust. https://github.com/helix-editor/helix/discussions/3806#discussioncomment-6686976 https://github.com/helix-editor/helix/discussions/3806#discu...
packetlost 2y ago
I'm more of a R7RS kinda guy but I appreciate this, especially the types. Scheme allows for extremely non-linear memory access which means you necessarily need dynamic memory management and garbage collection. IMO once you're to that spot, there's little reason to stick with Rust, though it being the implementation language is interesting in itself. That being said, there are battle tested Scheme implementations out there that have fantastic FFI APIs that would probably work just as well if you don't mind a small bit of C to glue things together.
mightyham 2y ago
This seems like a great idea and I support the effort. It was not clear to me on first read though that what was being proposed is not an extension to current async rust (compiling code to a series of state machines), but a completely alternative design utilizing a context switching runtime like Erlang or Go. If I interpreted that wrong please correct me. Part of me wonders, considering that rust is a systems programming language, how difficult would it be to write a runtime like that in rust so that there is no need to use a second language?
- whitten 2y ago
  So your goal for the runtime would just be a different runtime for Rust ?
  - dapperdrake 2y ago
    Unlikely. C and Rust have run-time environments. Just not as elaborate as the JVM for Java or a web browser for JS and WASM. It could only be about adding a second run-time environment to the same operating system process. That’s what the questions seems to be enquiring about. And it’s about instrumenting the Rust run-time environment with a REPL and human-friendly run-time data structures.
  - mightyham 2y ago
    To be clear, in this comment, I am referring to async in the broadest sense. I am specifically NOT referring to the Rust compiler keyword and runtimes (like tokio) that work with rust future state machines. If I understand OP correctly his current proposal is to create a different runtime for rust. That runtime happens to be written in scheme using his own compiler. It is meant to make writing async rust applications easier by having simple rust interop. Given the authors implicit argument that an async runtime with a stack-based context switching model can provide better debugging. Is it possible to write something like that in rust? That may be better for some use cases, as it wouldn't result in a multi-language project.
- dapperdrake 2y ago
  Doesn’t seem like it. It’s about interactive instrumentation of the runtime. Think lua in Haskell adding slight reflection and a REPL. In another comment I referenced Chris Kohlhepp’s write up.
api 2y ago
I wonder if people are in a way misusing Rust by trying to use it to build everything. It's designed to be a systems language, one for writing OSes, drivers, core system services, low level parsers, network protocols, compilers, runtimes, etc. There's no way a low-level systems language like this is going to be quite as ergonomic as a higher level more abstract garbage collected language.
- dapperdrake 2y ago
  Luckily, neither CPython, nor Lua, nor Perl, nor PHP, nor Ruby, nor Google Chrome are written in C. Oh, wait.
- hardwaresofton 2y ago
  IMO you've stumbled into one of the reasons Rust is amazing -- it can go to higher levels, for many good reasons. Whether you should is another question, and it will arguably never be as easy to write as python or javascript (to be fair it's also easy to write inconsistent code in those languages), but it can. I wouldn't argue that Rust is as ergonomic as other languages, but it has enough pieces to build DSLs that make it look as ergonomic as other languages. An example of this is the front-end frameworks that have been spun in Rust, like Leptos[0]. Rust probably has no business being on the frontend, but the code snippets are convincing and look pretty reasonable. If the macro machinery and type stuff never blows up/forces you to touch internals, then does anyone care that it's a system language underneath? [0]: https://book.leptos.dev/view/01_basic_component.html https://book.leptos.dev/view/01_basic_component.html
  - UncleEntity 2y ago
    > Rust probably has no business being on the frontend... I would argue it this were more common they would have a lot of incentive to iron out the rough spots. Well, hopefully not in the way that makes it easier to have you download a 100+ megabyte wasm bundle to read a static blogpost, but anyways. Personally, if I'm writing a script for an actual purpose (as opposed to just coding for coding's sake) I reach for python and if that's too slow I write a wrapper around some C code to speed things up. If I'm just messing around I use C(++) because that's what I know -- which is where rust could easily fit if the learning curve wasn't so high that I don't see the effort as worth it.
ilrwbwrkhv 2y ago
What would be really nice is a new lisp which actually allows for incredibly interactive programming similar to common lisp, but which targets a runtime like the v8 engine maybe. Because I think a lot of people are missing out on the Smalltalk / Common Lisp experience. Even Clojure and other lisps do not enable that level of interactive programming which the break loop in Common Lisp does.
- homarp 2y ago
  if you are curious about the breakloop as I was: https://news.ycombinator.com/item?id=35693916 https://news.ycombinator.com/item?id=35693916
  - ilrwbwrkhv 2y ago
    I actually built one of my earlier startups on Steelbank Common Lisp and I absolutely love the break loop. To build up a program by iterating on the break loop is just a magical experience and I think one can go even much further with a modern version of it where you can sort of jump around the stack and you can almost do a bunch of these things in common lisp as well but it is something that needs a little bit of work and it doesn't look and feel as modern as some of the other languages so I feel like therelisp as well but it is something that needs a little bit of work and it doesn't look and feel as modern as some of the other languages so I feel like there is an opportunity for a new language.
whitten 2y ago
What is a Tokio function ? What is CPS ?
- Y_Y 2y ago
  Tokio is a Rust library for async - https://tokio.rs/ https://tokio.rs/ CPS is Continuation Passing Style - https://www.youtube.com/watch?v=MbtkL5_f6-4 https://www.youtube.com/watch?v=MbtkL5_f6-4
  - mikevin 2y ago
    Is that the correct video for CPS?
    - Y_Y 2y ago
      Yes, via an old joke. See https://wiki.call-cc.org/chicken-compilation-process#cps-converted https://wiki.call-cc.org/chicken-compilation-process#cps-con... .
- bjoli 2y ago
  CPS style stands for continuation passing style. It is common for compiler to represent programs in a CPS style since it is a handy format for a compiler to reason about.
perrygeo 2y ago
Also a shout out to Steel (https://github.com/mattwparas/steel https://github.com/mattwparas/steel), another Scheme-on-Rust which is more active. The Rust community has given us at least two Scheme runtimes, several JS runtimes, but no Clojure interpreter yet that I'm aware of. There are a few zombie projects squatting on the name (seems common in Rust) but no viable releases.
- harrison_clarke 2y ago
  there's no reason you couldn't do a clojure in rust but the main implementation benefits from GC and making it easy to call host (java) methods. as far as i know, the core clojure data structures are kinda hard if you don't have a GC
  - packetlost 2y ago
    Lisps almost universally require or expect GC.
    - pjmlp 2y ago
      Yes, although the Common Lisp derived ones, and the old ones from Xerox, Symbolics, TI and co, also have other capabilities available, especially for the low level programming part, and all common data structures, not only lists.
    - TransAtlToonz 2y ago
      The only mature, production-used exception of which I can think is Naughty Dog's Game Oriented Object Lisp/Game Oriented Assembly Lisp. EDIT: I was wrong, wikipedia claims there are basic GC functions. I can't speak to the details, but I imagine that whatever they have would attempt to lift allocations out of the hot path/use arena allocation or some similar mechanism. That said, people have strong opinions on whether reference counting counts as GC. I say it does, but others are vehement that it does not. If it does not count as GC, I think Interlisp-D would qualify.
  - JonChesterfield 2y ago
valorzard 2y ago
Is the name Gouki a Street Fighter Reference? (For those not in the know, Gouki is the Japanese name for Akuma)
- maplant 2y ago
  Yup. He’s also my GitHub avatar. The idea behind the name is a reference to the language providing the user the maximal set of tools
  - valorzard 2y ago
    I see, so Scheme is the Satsui no Hado…
giancarlostoro 2y ago
To the author, if you're reading this: > But while I thing that async Rust is well designed At least one typo, thank you for not using an LLM to spit out an article. :)
IshKebab 2y ago
Why do people put up with the Lisp syntax? I get that it's super simple for computers to parse, but it's definitely not pleasant for humans to parse, and humans are the ones reading/writing code. I would have thought some very simple syntactic sugar would make it a lot nicer to work with. Has anyone done that?
- kazinator 2y ago
  > simple for computers to parse Syntaxes that are hard for computers to parse are also hard for humans to parse. But, nobody literally parses Lisp syntax: you rely on indentation, just like when you're writing C, Java, shell scripts. People working in Lisp like Lisp syntax; it's really easy to edit, very readable, very consistent. Anything new in the language, whether coming from a new release of your dialect or someone's new project, is a list with a new keyword, and predictable syntax! Syntax that works with your editor, tooling, and readability intuitions. If you work involves a lot of numerical formulas, the Lisp way of writing math isn't always the best. You might have documents for the code which use conventional math and someone going back and forth between the doc and the code may have to deal with the translation. Most software isn't math formulas. The language features that need ergonomics are those which address program organization. There are ways to get infix math if you really want it in that code. > Has anyone done that? Over and over again, starting in the 1960's. 1. 1960's: Defunct "Lisp 2" project run by John MacCarthy himself. Algol-like syntax generating Lisp code under the hood. 2. 1970s': CGOL by Vaughan Pratt (of Pratt Parser fame). 3. Others: - Sweet Expressions by David A. Wheeler (for Scheme, appearing as a SRFI). - Dylan - infix.cl module for Common Lisp - Racket language and its numerous #lang modules. The common thread in "syntactic skins" for Lisp is two fold: they all either died, or turned into separate languages which distanced themselves from Lisp. None of the above skins I mentioned received wide spread use and are defunct, except maybe Racket's various #lang things. However, Lisp has inspired languages with non-Lisp syntax: some of them with actual Lisp internals. For instance the language R known for statistical capabilities is built on a Lisp core. It has symbols, and cons-cell based lists terminated by NIL, which are used for creating expressions. The cons cells have CAR and CDR fields! The language Julia is boostrapped out of something called Femtolisp, which is still buried in there.
timewizard 2y ago
> I believe that it is a remarkably well designed language > and just overall a poor debugging experience make async Rust a frustrating experience for rapid iteration. > is the result of conscious trade-offs, trade-offs that I believe were chosen correctly. I don’t believe that async Rust necessarily needs to change at all, although I don’t doubt that the experience could be improved. That's rust for you. It's a remarkably well designed language. Except for all the ways it isn't. What's hilarious to me is none of these are even deep or hard to find issues. Right out of the gate.. cargo is a wart.. and async was designed inside out. Remarkable. Indeed. So, _of course_, let's use it to implement a _different_ language entirely. The niche is disappearing.
- maplant 2y ago
  Languages are about trade-offs.
  - timewizard 2y ago
    Compilers are. Languages are not.

198 points by maplant 2y ago

Nice work! It's a good write-up, too. Hygienic macro expansion is one of the things I still haven't implemented before. I remember a [talk][1] where Matthew Flatt illustrates the idea of sets of scopes, and choosing the largest intersection. I see in your implementation there are sets of "marks" in syntax objects, is that what's going on there? I haven't played with rust, but when I do I'll be able to play with this scheme too. [1]: https://www.youtube.com/watch?v=Or_yKiI3Ha4 https://www.youtube.com/watch?v=Or_yKiI3Ha4

maplant 2y ago
I probably should have looked at this talk! Instead my reference was this PhD thesis[1] cited in the R6RS spec. I can’t say I read the whole thing, but my takeaway was to use a system of marks/antimarks. The basic idea is that at the point of macro expansion, the syntax object is “marked” (i.e. added to the set of marks) with a random integer. The environment of the macro is recorded at that point as well, along with the mark used. After expansion, the resulting syntax object is again marked with the same mark. When an identifier is marked twice with the same mark, the marks cancel each other out. The result of this is that identifier that are introduced into the expander and those introduced by the expander are distinguishable. [1]: https://www2.ccs.neu.edu/racket/pubs/dissertation-kohlbecker.pdf https://www2.ccs.neu.edu/racket/pubs/dissertation-kohlbecker...
- lygaret 2y ago
  The talk goes line by line through this code [1]; I've been transcribing and taking notes on this for the last week, for a somewhat similar project actually. If you're interested, I've got a huge pile of papers and links collected here [2] that you might enjoy. I've read everything, but right now it's still a big ol' mush; there's a _ton_ of prior art! Enjoy, good hacking! [1]: https://github.com/mflatt/expander/tree/pico https://github.com/mflatt/expander/tree/pico [2]: https://github.com/lygaret/gigi?tab=readme-ov-file#reading https://github.com/lygaret/gigi?tab=readme-ov-file#reading
  - -__---____-ZXyw 2y ago
    Lovely pipe of papers and links, thanks for sharing!
- bjoli 2y ago
  I have said this before, but it deserves repeating: When people say scheme is a simple language, they do not mean the hygienic macro system. Especially not something like psyntax/syntax-case. I am very impressed! The implementation is crazy clear.
  - maplant 2y ago
    Thank you! I’m very proud of the macro expander, so comments like yours are really invigorating and justify the work I put into it

leftyspook 2y ago

I think you make a pretty bad case for how embedding a Scheme interpreter is going to help with the pain points of async. Listing "stack traces full of tokio code" and then seemingly proposing to solve that by adding more glue to pollute the stack traces is especially weird.

maplant 2y ago
Not an interpreter! A compiler :-) Have you seen a stack trace originating from somewhere within tokio? Nearly all useful information is lost. My contention is that by isolating the functions that are required to be written in Rust and then doing orchestration, spawning, etc in Scheme the additional debug information at runtime will make the source of errors much more clear. I could be wrong! But hey there’s other reasons too. Being able to debug Rust functions dynamically is pretty cool, as well as being able to start/stop them like daemons.
- giancarlostoro 2y ago
  On the other hand, you might wind up with an impressive Scheme implementation.
- kanbankaren 2y ago
  luajit gets you almost all of scheme features except macros. Speed of luajit is very close to native C.
  - dapperdrake 2y ago
    Sounds like Greenspun's tenth rule.
    - kanbankaren 2y ago
      Well, unlike Lisp & Scheme, Lua and Luajit are used in production, millions of video game consoles, embedded systems, etc.

hardwaresofton 2y ago

Glad to see more language runtimes be available from Rust! Looking forward to scheme-rs being able to benefit from the safety and speed Rust allows

dapperdrake 2y ago
It’s about instrumenting Rust’s existing run-time.
- hardwaresofton 2y ago
  Are we reading the same article? It's about this project: https://crates.io/crates/scheme-rs https://crates.io/crates/scheme-rs
  - dapperdrake 2y ago
    So? There is instrumentation for Async Rust being added. Just not in the Rust compiler and not in tokio. More in application level code. So as far as the user is concerned to "their" Rust run-time. A distinction that is mostly useless when programming lisp. Hence lisp. Lisp is about solving your problem and getting the "language" out of the way. So you add what is missing. As long as it works it doesn’t even matter where it’s added. Even C has a barebones runtime environment when compiled for a machine with an MMU and an OS, so not a microcontroller.

reikonomusha 2y ago

Coalton [1] is a Lisp language that has Hindley-Milner type inference with type classes. Its syntax actually resembles the prototype syntax from TFA pretty closely. The Coalton syntax-equivalent would be: (define-type (Option :a) (Some :a) None) (let ((x (Some 5))) (match x ((Some y) y) ((None) (random)))) (declare random (Distribution :a => Unit -> :a)) (define (random) (sample (thread-rng))) (define-class (Eq :t) (= (:t -> :t -> Boolean))) (define-instance (Eq Number) (define (= lhs rhs) (num-equal lhs rhs))) Of course types and classes like Option and Eq are built-in already. Coalton is based on Common Lisp (and allows free mixing with CL code) instead of Scheme however, though Coalton is a Lisp-1 and would feel relatively at-home to any Scheme programmer. [1] https://github.com/coalton-lang/coalton https://github.com/coalton-lang/coalton

maplant 2y ago
Very cool! I saw a number of projects pretty similar to this one, typed racket for example. I didn’t really talk about it too much but what I want to do is flip the direction. Coalton is based on CL, but what I would like is scheme to essentially be a set of Gouki function with the signature top … -> top. Then, i want to see how fast I can make the interaction between them be using different analysis like k-CFA
anonzzzies 2y ago
Coalton is great. Indeed often it's much easier to just implement on top of either SBCL or ChezScheme; they are very fast and chances are you are not going to make anything as or more performant, better documented etc. However, reimplementation in Rust is not too bad as that's were we need to be for small, fast, low level runtimes. If the goal was 'just' their own language, I feel just a an implementation on top of CL/SBCL + Coalton would've been faster to build, easier to maintain and probably far more performant. Less fun though!

markstos 2y ago

The Helix editor, a popular alternative to Vim, is going to implementing it's plugin system in a Scheme-like language. Helix is also written in Rust. https://github.com/helix-editor/helix/discussions/3806#discussioncomment-6686976 https://github.com/helix-editor/helix/discussions/3806#discu...

packetlost 2y ago

I'm more of a R7RS kinda guy but I appreciate this, especially the types. Scheme allows for extremely non-linear memory access which means you necessarily need dynamic memory management and garbage collection. IMO once you're to that spot, there's little reason to stick with Rust, though it being the implementation language is interesting in itself. That being said, there are battle tested Scheme implementations out there that have fantastic FFI APIs that would probably work just as well if you don't mind a small bit of C to glue things together.

mightyham 2y ago

This seems like a great idea and I support the effort. It was not clear to me on first read though that what was being proposed is not an extension to current async rust (compiling code to a series of state machines), but a completely alternative design utilizing a context switching runtime like Erlang or Go. If I interpreted that wrong please correct me. Part of me wonders, considering that rust is a systems programming language, how difficult would it be to write a runtime like that in rust so that there is no need to use a second language?

whitten 2y ago
So your goal for the runtime would just be a different runtime for Rust ?
- dapperdrake 2y ago
  Unlikely. C and Rust have run-time environments. Just not as elaborate as the JVM for Java or a web browser for JS and WASM. It could only be about adding a second run-time environment to the same operating system process. That’s what the questions seems to be enquiring about. And it’s about instrumenting the Rust run-time environment with a REPL and human-friendly run-time data structures.
- mightyham 2y ago
  To be clear, in this comment, I am referring to async in the broadest sense. I am specifically NOT referring to the Rust compiler keyword and runtimes (like tokio) that work with rust future state machines. If I understand OP correctly his current proposal is to create a different runtime for rust. That runtime happens to be written in scheme using his own compiler. It is meant to make writing async rust applications easier by having simple rust interop. Given the authors implicit argument that an async runtime with a stack-based context switching model can provide better debugging. Is it possible to write something like that in rust? That may be better for some use cases, as it wouldn't result in a multi-language project.
dapperdrake 2y ago
Doesn’t seem like it. It’s about interactive instrumentation of the runtime. Think lua in Haskell adding slight reflection and a REPL. In another comment I referenced Chris Kohlhepp’s write up.

api 2y ago

I wonder if people are in a way misusing Rust by trying to use it to build everything. It's designed to be a systems language, one for writing OSes, drivers, core system services, low level parsers, network protocols, compilers, runtimes, etc. There's no way a low-level systems language like this is going to be quite as ergonomic as a higher level more abstract garbage collected language.

dapperdrake 2y ago
Luckily, neither CPython, nor Lua, nor Perl, nor PHP, nor Ruby, nor Google Chrome are written in C. Oh, wait.
hardwaresofton 2y ago
IMO you've stumbled into one of the reasons Rust is amazing -- it can go to higher levels, for many good reasons. Whether you should is another question, and it will arguably never be as easy to write as python or javascript (to be fair it's also easy to write inconsistent code in those languages), but it can. I wouldn't argue that Rust is as ergonomic as other languages, but it has enough pieces to build DSLs that make it look as ergonomic as other languages. An example of this is the front-end frameworks that have been spun in Rust, like Leptos[0]. Rust probably has no business being on the frontend, but the code snippets are convincing and look pretty reasonable. If the macro machinery and type stuff never blows up/forces you to touch internals, then does anyone care that it's a system language underneath? [0]: https://book.leptos.dev/view/01_basic_component.html https://book.leptos.dev/view/01_basic_component.html
- UncleEntity 2y ago
  > Rust probably has no business being on the frontend... I would argue it this were more common they would have a lot of incentive to iron out the rough spots. Well, hopefully not in the way that makes it easier to have you download a 100+ megabyte wasm bundle to read a static blogpost, but anyways. Personally, if I'm writing a script for an actual purpose (as opposed to just coding for coding's sake) I reach for python and if that's too slow I write a wrapper around some C code to speed things up. If I'm just messing around I use C(++) because that's what I know -- which is where rust could easily fit if the learning curve wasn't so high that I don't see the effort as worth it.

ilrwbwrkhv 2y ago

What would be really nice is a new lisp which actually allows for incredibly interactive programming similar to common lisp, but which targets a runtime like the v8 engine maybe. Because I think a lot of people are missing out on the Smalltalk / Common Lisp experience. Even Clojure and other lisps do not enable that level of interactive programming which the break loop in Common Lisp does.

homarp 2y ago
if you are curious about the breakloop as I was: https://news.ycombinator.com/item?id=35693916 https://news.ycombinator.com/item?id=35693916
- ilrwbwrkhv 2y ago
  I actually built one of my earlier startups on Steelbank Common Lisp and I absolutely love the break loop. To build up a program by iterating on the break loop is just a magical experience and I think one can go even much further with a modern version of it where you can sort of jump around the stack and you can almost do a bunch of these things in common lisp as well but it is something that needs a little bit of work and it doesn't look and feel as modern as some of the other languages so I feel like therelisp as well but it is something that needs a little bit of work and it doesn't look and feel as modern as some of the other languages so I feel like there is an opportunity for a new language.

whitten 2y ago

What is a Tokio function ? What is CPS ?

Y_Y 2y ago
Tokio is a Rust library for async - https://tokio.rs/ https://tokio.rs/ CPS is Continuation Passing Style - https://www.youtube.com/watch?v=MbtkL5_f6-4 https://www.youtube.com/watch?v=MbtkL5_f6-4
- mikevin 2y ago
  Is that the correct video for CPS?
  - Y_Y 2y ago
    Yes, via an old joke. See https://wiki.call-cc.org/chicken-compilation-process#cps-converted https://wiki.call-cc.org/chicken-compilation-process#cps-con... .
bjoli 2y ago
CPS style stands for continuation passing style. It is common for compiler to represent programs in a CPS style since it is a handy format for a compiler to reason about.

perrygeo 2y ago

Also a shout out to Steel (https://github.com/mattwparas/steel https://github.com/mattwparas/steel), another Scheme-on-Rust which is more active. The Rust community has given us at least two Scheme runtimes, several JS runtimes, but no Clojure interpreter yet that I'm aware of. There are a few zombie projects squatting on the name (seems common in Rust) but no viable releases.

harrison_clarke 2y ago
there's no reason you couldn't do a clojure in rust but the main implementation benefits from GC and making it easy to call host (java) methods. as far as i know, the core clojure data structures are kinda hard if you don't have a GC
- packetlost 2y ago
  Lisps almost universally require or expect GC.
  - pjmlp 2y ago
    Yes, although the Common Lisp derived ones, and the old ones from Xerox, Symbolics, TI and co, also have other capabilities available, especially for the low level programming part, and all common data structures, not only lists.
  - TransAtlToonz 2y ago
    The only mature, production-used exception of which I can think is Naughty Dog's Game Oriented Object Lisp/Game Oriented Assembly Lisp. EDIT: I was wrong, wikipedia claims there are basic GC functions. I can't speak to the details, but I imagine that whatever they have would attempt to lift allocations out of the hot path/use arena allocation or some similar mechanism. That said, people have strong opinions on whether reference counting counts as GC. I say it does, but others are vehement that it does not. If it does not count as GC, I think Interlisp-D would qualify.
- JonChesterfield 2y ago

valorzard 2y ago

Is the name Gouki a Street Fighter Reference? (For those not in the know, Gouki is the Japanese name for Akuma)

maplant 2y ago
Yup. He’s also my GitHub avatar. The idea behind the name is a reference to the language providing the user the maximal set of tools
- valorzard 2y ago
  I see, so Scheme is the Satsui no Hado…

giancarlostoro 2y ago

To the author, if you're reading this: > But while I thing that async Rust is well designed At least one typo, thank you for not using an LLM to spit out an article. :)

IshKebab 2y ago

Why do people put up with the Lisp syntax? I get that it's super simple for computers to parse, but it's definitely not pleasant for humans to parse, and humans are the ones reading/writing code. I would have thought some very simple syntactic sugar would make it a lot nicer to work with. Has anyone done that?

kazinator 2y ago
> simple for computers to parse Syntaxes that are hard for computers to parse are also hard for humans to parse. But, nobody literally parses Lisp syntax: you rely on indentation, just like when you're writing C, Java, shell scripts. People working in Lisp like Lisp syntax; it's really easy to edit, very readable, very consistent. Anything new in the language, whether coming from a new release of your dialect or someone's new project, is a list with a new keyword, and predictable syntax! Syntax that works with your editor, tooling, and readability intuitions. If you work involves a lot of numerical formulas, the Lisp way of writing math isn't always the best. You might have documents for the code which use conventional math and someone going back and forth between the doc and the code may have to deal with the translation. Most software isn't math formulas. The language features that need ergonomics are those which address program organization. There are ways to get infix math if you really want it in that code. > Has anyone done that? Over and over again, starting in the 1960's. 1. 1960's: Defunct "Lisp 2" project run by John MacCarthy himself. Algol-like syntax generating Lisp code under the hood. 2. 1970s': CGOL by Vaughan Pratt (of Pratt Parser fame). 3. Others: - Sweet Expressions by David A. Wheeler (for Scheme, appearing as a SRFI). - Dylan - infix.cl module for Common Lisp - Racket language and its numerous #lang modules. The common thread in "syntactic skins" for Lisp is two fold: they all either died, or turned into separate languages which distanced themselves from Lisp. None of the above skins I mentioned received wide spread use and are defunct, except maybe Racket's various #lang things. However, Lisp has inspired languages with non-Lisp syntax: some of them with actual Lisp internals. For instance the language R known for statistical capabilities is built on a Lisp core. It has symbols, and cons-cell based lists terminated by NIL, which are used for creating expressions. The cons cells have CAR and CDR fields! The language Julia is boostrapped out of something called Femtolisp, which is still buried in there.

timewizard 2y ago

> I believe that it is a remarkably well designed language > and just overall a poor debugging experience make async Rust a frustrating experience for rapid iteration. > is the result of conscious trade-offs, trade-offs that I believe were chosen correctly. I don’t believe that async Rust necessarily needs to change at all, although I don’t doubt that the experience could be improved. That's rust for you. It's a remarkably well designed language. Except for all the ways it isn't. What's hilarious to me is none of these are even deep or hard to find issues. Right out of the gate.. cargo is a wart.. and async was designed inside out. Remarkable. Indeed. So, _of course_, let's use it to implement a _different_ language entirely. The niche is disappearing.

maplant 2y ago
Languages are about trade-offs.
- timewizard 2y ago
  Compilers are. Languages are not.