14 ms·

Steel – An embeddable and extensible Scheme dialect

245 points by MaximilianEmel 3y ago

darthrupert 3y ago
This is just perfection on first glance, exactly what I've been looking for to augment my Rust projects that need a command/configuration language. And with a working repl, too, apparently. I hope the second and third glances will be good too.
_nalply 3y ago
Helix Editor plans to use Steel as a plugin language [0]. [0]: https://github.com/helix-editor/helix/pull/8675 https://github.com/helix-editor/helix/pull/8675
- agumonkey 3y ago
  emacsers looking deep from afar
  - NeutralForest 3y ago
    I would use Helix in the terminal if it supported Emacs keybindings tbh but I don't want to relearn another set of keybindings. Still I'd be interested in what it becomes.
    - goku12 3y ago
      The keybindings is one of the major selling points of helix. It uses kakoune style object-verb action (like vim visual mode) by default with multiple selections. If you're comfortable with emacs bindings then you're better off with a lightweight emacs alternative.
      - NeutralForest 3y ago
        Fair enough
      - Hugsun 3y ago
        As an evil user, this is potentially huge to me. Emacs happens to have the best balance of easy to setup configuration (relatively), powerful package ecosystem, and proper hackability of all editors I've found. It's not very fast though and has some conventions that feel archaic.
        goku12 3y ago
        I'm also an Emacs evil user (neovim too). I think the kakoune editing model has the potential to surpass even the vim/evil model. Its default object-verb order makes it easy to preview and change selection before proceeding with the action. That's not possible with vim's normal mode. Vim does have the visual model. But then kakoune model also uses multiple cursors, making it more powerful. I really wanted to try kakoune model in Emacs. But the package needs a bit more updates. Another issue I have with evil is that it changes a lot of Emacs' default bindings, making it hard to do certain tasks. Some operations simply don't work at all. The kakoune package doesn't do this - at least not in insert mode. > It's not very fast though and has some conventions that feel archaic Sadly, multithreading is an afterthought for Emacs. There is just too much legacy stuff to make it easy. The language design is also from another era. The default dynamic binding feels very alien when almost every language everyone knows is lexical binding by default. On the other hand, scheme feel very modern due to very careful language design. But the effort to switch Emacs to scheme didn't find much steam. There is one aspect where none of the new editors (hx with scheme and nvim with lua)can match Emacs. Emacs is entirely written in elisp with the C parts acting merely as libraries. The extension language for other editors is just an addition to their core editing code.
        klibertp 3y ago
        > Sadly, multithreading is an afterthought for Emacs It is, but it's usable. I'm actually amazed that, even after three major versions, the built-in threading is not used by the community. Yes, the threads currently are not usable for number crunching in the background. And yes, there are bugs, and trying to do many things from the background thread doesn't work, sometimes in unexpected ways. You can still block the main thread from the background thread since some things block the event loop, no matter where they were started. But, the threads do give you independent control flows. Whatever you cannot do in the background, you can offload to the main thread with a timer and a queue of lambdas. The built-in threads are very, very bare-bones - it's around 15 functions, for threads, mutexes, and condition variables. They are very limited by their "mostly cooperative" nature. However, with a bit of sugar, they are usable for at least one thing: async processes and network communication. In a background thread, you can "block" to wait for a child process to do something. It's natural and requires no macrology (async.el...). The same is true for network communication. You can block and wait for a response while the rest of Emacs does whatever. With just two functions, you can write code without blocking as if you used `call-process`. Sequential actions - call this, wait for it to finish, call that, wait for it to finish, etc. - can now be coded in a sequential way, without having to worry about callbacks, sentinels, and a poor-man FSM implementation that invariably appears in Elisp that doesn't use threads. The threads built into Emacs, currently, are closer to green threads or coroutines, functionally, than to OS-level threads. But that's still a huge help in a bunch of important and pervasive scenarios. It's really strange that nobody seems to realize this. With threads (as they are), the Continuation Passing Style compiler macro (in generator.el), and dynamic modules (for actual parallelism where needed) Emacs now has everything it needs to make it non-blocking by default. Of course, that would entail rewriting everything on top of these abstractions, so it's unrealistic - but for new code and packages? I think we're just one package (along the lines of dash, s, etc.) away from convenient concurrency and parallelism in Emacs. The problem, of course, is that someone needs to design and code that package...
    - schemescape 3y ago
      Out of curiosity, if you’re familiar with Emacs, what’s your motivation for wanting to use Helix? Built-in LSP support?
      - ParetoOptimal 3y ago
        I use it despite a very complete emaxs setup because: - curiosity - seeing if I'm missing anything - haven't had time to get tree sitter stuff configured, and wanted to see how good it is - see if I want to move back to meow from vanilla (tons of previous vim, evil experience)
      - NeutralForest 3y ago
        - Curiosity - Speed, even though emacs with the emacsclient is pretty much instant - Font support, for some reason the terminal with Emacs sometimes doesn't render as well as it should - Some keybindings issues in the terminal like Ctrl-Backspace not working properly. - See if there are good workflows/defaults I could use in Emacs
- nikolay 3y ago
  That's ridiculous!
  - icommentedtoday 3y ago
    ?
  - bobbyskelton41 3y ago
    I saw your comment on the thread. A. Most WebAssembly implementations are bigger than Helix itself. B. WebAssembly is immature. C. Lisp is perfectly fine. Even if you don't like it, it's not the end of the world if you have to use it to configure Helix.
    - nikolay 3y ago
      A. It's true that Wasmer is bigger, but even my phone has 16GB of RAM and 1TB of storage space. B. There are probably more computers running WebAssembly loads today than Lisp ones. C. Lisp is a mature language, almost too mature. I used it extensively in the '80s and it served me well then, but that was 40 years ago. At least have the decency to use a more modern language like Lua, which is what Neovim uses.
      - bobbyskelton41 3y ago
        A. Sure, but it isn't sufficiently beneficial for the cost. B. WebAssembly is immature for developing a plugin system because of the lack of a sufficient ABI: https://github.com/WebAssembly/component-model https://github.com/WebAssembly/component-model C. There aren't any other languages that meet the criteria. Lua was a no-go from the start. The maintainers did not like the language, and it necessitated adding more C code to Helix which could complicate building even further. https://github.com/helix-editor/helix/discussions/3806#discussioncomment-6686976 https://github.com/helix-editor/helix/discussions/3806#discu...
        nikolay 3y ago
        A Lisp, a weird dialect of Lisp, is not better than Lua. Why use Rune [0]?! [0]: https://rune-rs.github.io/ https://rune-rs.github.io/
sirsuki 3y ago
When I tried to introduce s-expressions to a DSL my co-workers nearly lynched me. The parenthesis were so violently hated I sunk into a deep hole and still haven’t came back out of it.
- capableweb 3y ago
  It is really fun (sad) to see this, I've seen it so many times myself too. You show how something would be with s-expressions, compared to something, and all they can focus on is how many parenthesis there are. But when you sit down and count, they're the same amount as the code was when it wasn't s-expressions, just in different locations. And when you remove the parenthesis, they can kind of understand the code, kind of.
  - fwip 3y ago
    One option that might be suitable for a DSL, is implicit parens based on whitespace. A newline opens a new paren, and the paren closes when it reaches another line with the same indentation e.g: (defun factorial (x) (if (zerop x) 1 (* x (factorial (- x 1))))) could be rewritten as defun factorial (x) if (zerop x) 1 * x (factorial (- x 1))
    - alwaysbeconsing 3y ago
      Whitespace significance might be the choice even more controversial than Lisp parens. For myself, I appreciate both, but those that do not, really do not.
      - capableweb 3y ago
        Probably there are more Python users out there than combined users of all lisp-like languages, which I guess would mean people are less scared of white-space significance than s-expressions :)
    - nsajko 3y ago
      Existing SRFI: SRFI 49 "Indentation-sensitive syntax": https://srfi.schemers.org/srfi-49/srfi-49.html https://srfi.schemers.org/srfi-49/srfi-49.html
oldpersonintx 3y ago
I want this to succeed!! please do not let the word "srfi" ever appear in the packages list...naming libraries with obscure numbers no one remembers was a terrible terrible idea that all schemes seem to perpetuate
- mattparas 3y ago
  At the moment I only have 1 SRFI package, and its just to check against compatibility :) - my plan is to wrap the SRFI packages with friendly names and just include the metadata in the package spec so someone searching for it can find it easily.
- zelphirkalt 3y ago
  It is a standard. What do you expect? But there are some or many Schemes that offer their own libs on top or below SRFI implementations, which then have readable names.
  - adastra22 3y ago
    I expect people to willing not conform when nonconformance improves the art.
- aag 3y ago
  SRFI editor here. The numeric designations are there because, among other reasons, there is sometimes more than one SRFI for a particular general idea. But there is nothing stopping libraries from having more than one name, including a semantically meaningful one. In fact, there has been a SRFI standardizing how that is done since 2008 [1]. [1] https://srfi.schemers.org/srfi-97/srfi-97.html https://srfi.schemers.org/srfi-97/srfi-97.html
mattparas 3y ago
This is a surprise! Steel is my project, happy to answer any questions anyone might have
- dataangel 3y ago
  Any chance of seeing the things I miss the most from Racket in other langs? 1. Parameters and Syntax Parameters (Syntax parameters make macros more powerful) 2. Turing complete macros (not just syntax-case) 3. Typed Racket I almost used https://gamelisp.rs/ https://gamelisp.rs/ for a project but the nightly feature it needs broke and it's no longer maintained, glad to see something similar arise! You might want to consider adopting their choice of VecDeque as a list replacement, I think it makes a lot more sense than naive linked lists on modern machines.
  - mattparas 3y ago
    To answer each of these: 1. I do support parameters now, syntax parameters not yet. I would like to! But Racket has a pretty hefty head start on me so it'll take some time. 2. Right now I have syntax-rules macros, I also have defmacro style macros that get used internally in the kernel during expansion, but haven't yet opened them up for user space yet. Syntax case will be coming soon hopefully. 3. The odds of me being able to come up with an implementation to match typed racket pound for pound is pretty low. I have toyed with using contracts as types (where possible), with medium/promising success in certain situations. I have a soft spot for racket and have been modeling behavior after it, however it will take time to be able to create a macro system powerful enough to match it. It wouldn't be impossible to create an alternative syntax to just lower to steel after type checking, but I haven't put time into that. On the list type - the list in use currently is an unrolled linked list https://github.com/mattwparas/im-lists https://github.com/mattwparas/im-lists, which I've found to yield much better performance for iteration than the naive linked lists. When possible, the vm does some in place mutation on the lists as well when consing, which helps performance as well. I also can hot swap for a vlist, but at the moment have stuck with unrolled linked lists.
    - mst 3y ago
falcor84 3y ago
Rust is gradually overcoming C and Fortran and Greenspun's tenth rule [0] is apparently inviolable. [0] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule
- deleted 3y ago
  [deleted]
ghc 3y ago
Should have named it Steele since it's dragging Rust users half-way to Lisp :).
- wolverine876 3y ago
  Yes! I just posted https://news.ycombinator.com/item?id=38510054 https://news.ycombinator.com/item?id=38510054
- bitwize 3y ago
  Why would you want to do that? Lisp is dynamically typed and has a GC.
  - Jtsummers 3y ago
    It's a joke that you seem to have missed. > We were not out to win over the Lisp programmers; we were after the C++ programmers. We managed to drag a lot of them about halfway to Lisp. Guy Steele on Java
    - bitwize 3y ago
      I'm familiar with the Steele quote. The manner in which C++ programmers were "dragged halfway to Lisp" concerns primarily manual memory management and all the bugginess attending thereto (Java came out before the STL and smart pointers were widely adopted). Concerns about memory safety were a significant part of the impetus for developing Java in the first place. Steele was responding to complaints that he had turned his back on Lisp, by countering that he was instead bringing the C++ crowd closer to Lisp with a C++-like language that had Lisp-like memory management. But the reason why Rust is such revolutionary computer science and, quite possibly, the most interesting thing to happen to PL design in decades is because with safe Rust you get all the memory-safety advantages of Java or Lisp, without a GC because the borrow checker statically guarantees object lifetimes. So Rust programmers don't need to be dragged halfway to Lisp the way C++ programmers were in the mid-90s, because Rust has the same memory-safety guarantees with none of the drawbacks of GC.
CooCooCaCha 3y ago
How does Steel handle garbage collection? Would it be possible to manually control the garbage collector? For example, a game that runs the GC at the end of every frame.
- mattparas 3y ago
  Immutable values are reference counted, so for most code, things will be dropped when they exit scope. For captured mutable values, there is a fairly mundane mark and sweep collector. It is possible to manually control the garbage collector, however I have not optimized it for that kind of workload. If you were embedding Steel in a game, I don't think it would be explicitly necessary to tune the GC as long as you aren't using a ton of mutation. If you were using a lot of mutation and still wanted a relatively performant GC collection at the end of every frame, then the underlying GC implementation would have to be changed or swapped to a different one (which is not impossible - I just only have one GC implemented)
  - mst 3y ago
    You might find the Recycler algorithm interesting - nim's ORC collector is ARC + Recycler and seems to be working out rather well.
    - mattparas 3y ago
      Thanks for the recommendation, with some brief poking around it does seem promising! It looks like the algorithm is this? https://github.com/fitzgen/bacon-rajan-cc https://github.com/fitzgen/bacon-rajan-cc The link to the paper seems dead unfortunately from this blog post https://nim-lang.org/blog/2020/12/08/introducing-orc.html https://nim-lang.org/blog/2020/12/08/introducing-orc.html I could see how it works as a drop in replacement for Rc
      - zozbot234 3y ago
        The bacon-rajan-cc link says it's stop-the-world. Samsara seems to be fully concurrent.
wolverine876 3y ago
I wonder where the name came from. Being HN, here is my nitpicking imperative: Names are important. Our inheritance is wit, self-awareness, and irony; names that puncture ego and power and that appeal to joy: C, C++, GNU, Rust, Google, Yahoo!, Vim, Git, awk, etc. Others are beautiful, evocative images, like Apple and Amazon. Names communicate our culture and ideals to each other and to the next generation. Careless, thoughtless names like Microsoft, IBM, etc. (ok it's ironic and self-deprecating, but without self-awareness or wit!), etc. should be hated and banned. Egotistical BS, especially Tolkien plagiarists who assert they are supernatural, should be tarred and feathered and paraded around town (with wit and irony). (Plenty of names fall in some middle ground.) If Steel is just a derivative of 'Rust' [edit: it is not, see the response below], it misses the self-awareness. Someone naming their development product - designed to build great structures - 'Rust' is engaging in a little self-deprication, joy, and self-awareness. Naming the derivative project 'Steel' possibly misses all that; there's a reason the original wasn't named Iron or Steel or Carbon Fiber. But maybe there's more to the name.
- pasabagi 3y ago
  I don't think it's particularly careless: SBCL is short for 'Steel Bank Common Lisp', which is a reference to how the CMU university, named after Carnegie Mellon, could be respectively substituted with 'Steel' (Carnegie) and 'Bank' (Mellon).
  - girvo 3y ago
    I, somehow, never put together the SB -> CM link! That's awesome, thanks for sharing
- mattparas 3y ago
  So here is the history: 1. Guy Steele (along with Gerald Sussman) created scheme, and Steel is close to Steele, just drop the e. 2. Steel is a scheme, and I observed that scheme names have a tendency to be named things crime related: Scheme, Racket (racketeering), Larceny, etc - Not a scientific analysis at all, but I found it funny at the time that Steel sounds like "steal". 3. You made the observation, Steel sounds like something that would be associated with Rust. Beyond that, I just liked the name. No SEO involved, and arguably probably should have picked something more searchable, but I didn't start making it with the intention of there being a lot of users, it started as a project for school.
I_am_tiberius 3y ago
It sounds great but what is it exactly and what can I use it for? Is there an equivalent of this in other languages?
- mattparas 3y ago
  Hopefully the linked README provides a general overview (I know I need to write some more documentation!), but Steel is an implementation of the scheme programming language (not entirely compliant yet, but aiming for R5RS and R7RS compliance). It can be used as a standalone language via the interpreter/repl (like Python or Racket), or it can be embedded inside applications, like Lua. There are hundreds (thousands, probably) of embeddable languages, each with their own flavor - see a list compiled here for example https://github.com/dbohdan/embedded-scripting-languages https://github.com/dbohdan/embedded-scripting-languages Use cases are generally for either configuration, scripting, or plugins - so scripting in games, or adding extensions to your text editor without having to use FFI or RPC + serializing a bunch of data. The advantage it has over using dynamic libraries (in general) is it runs in the same process, and can access the internal data structures directly without a lot of ceremony involved. The downside is that it is typically not as fast as native code unless a JIT is involved. Javascript is an example of an embedded scripting, where the browser is the host application.
  - I_am_tiberius 3y ago
    Understood. Thank you for the summary!
tucnak 3y ago
Is it possible to avoid rust completely when programming using Steel?
- mattparas 3y ago
  Yep - you can write standalone steel code without interacting with Rust at all, just interacting with the interpreter. I've done the first few days of the advent of code in Steel without needing to touch any Rust. Now, I will say that Steel has gotten visibility faster than I've been able to keep up with, so you might find a native function missing, or something that you would like to use that isn't implemented yet - at which point you either need to implement it yourself or open up an issue for someone to get to :)
  - schemescape 3y ago
    Solving Advent of Code problems in a language you implemented yourself should earn you an extra good star or something. Nice work, and cool project!
nerdponx 3y ago
Which Scheme is this implementing? R7RS? Any SRFIs? Is there a library reference document?
- mattparas 3y ago
  Starting with R5RS compliance, then once that is achieved moving on to R7RS. I borrowed the R5RS test suite from chibi here - https://github.com/mattwparas/steel/blob/master/cogs/r5rs.scm https://github.com/mattwparas/steel/blob/master/cogs/r5rs.sc... - only a few of these tests don't yet pass. Something like 135 pass, 4 fail, 20 skipped since I haven't implemented the primitives yet. I've only tested against a few SRFIs so far, but am also attempting to run the R7RS benchmark suite https://ecraven.github.io/r7rs-benchmarks/ https://ecraven.github.io/r7rs-benchmarks/ - So far you can view the progress here https://github.com/mattwparas/steel/tree/master/r7rs-benchmarks https://github.com/mattwparas/steel/tree/master/r7rs-benchma... There are some more that aren't yet checked in. I plan to get a document up with the exact state of compliance soon. The biggest difference right now is that, like Racket, Steel lists are immutable, so there I need a compatibility layer when running portable scheme.
  - nerdponx 3y ago
    Thanks! Was this originally a "just because" project, or do you expect it to offer some improvements or advantages over existing embeddable Schemes like Chibi (as you mentioned)?
    - mattparas 3y ago
      Originally started it as a school project, which then during the pandemic morphed into something to work on while cooped up. It is really a passion project, working on it is fun! There are some interesting design things I wanted to explore, like how to get good performance out of safe Rust, using unrolled linked lists or vlists instead of naive linked lists, using contracts, etc. Chibi is an impressive scheme implementation, and it will take a long time before Steel can hit the same level of compliance as Chibi. There is not a Chibi equivalent in native Rust that I am aware of. There are other embedded scripting languages for Rust that are pleasant - but no schemes of the maturity of Chibi. So in that regard, I'm hoping to offer a compelling scheme in native Rust to make integration with Rust applications relatively easy and painless. I also don't have a particularly strong need to be 100% completely compliant with the scheme specs. The plan is to have compatibility layers so that portable scheme code can be used, however there are things about scheme that I think Racket (for example) improved on, and I'd like to explore that as well as much as I can.
perrygeo 3y ago
Fantastic project! If you're looking to embed a DSL/full programming language into your application, this seems right on point. Rhai and RustPython are two other options if you don't dig parentheses.
SushiHippie 3y ago
Can someone explain these scheme and lisp languages to me? Every time I look at these languages, I can't grasp what you can use them for. And why one would use them. I always feel like I'm missing something. In the example scripts and code snippets, I can see that you can define functions, that you can use lists, mathematical operations, you can build some algorithms, you can print text, but it never goes further than that. I've only used languages like Python, Rust, C, Java, JavaScript, and they all have a very similar vibe, you have a std lib, which can interact with many things, you can build UIs, networking libraries and all that. And I could probably start using any language that is "similar" to these. But I could never use one of these scheme/lisp languages, as I can't really grasp them. Sorry, this comment is all over the place, because I can't really explain what's going on in my head when I see languages like this. I'd call myself a proficient programmer, but every time I look at these languages, it feels like as I've never seen code once in my lifetime. Any help or hint at what I'm missing, is appreciated.
- deleted 3y ago
  [deleted]
- femiagbabiaka 3y ago
  Basically the differences are in the concepts you'll use to write code. Lisps themselves are very different from each other, but just like the languages you're used to, many lisp distributions have standard libraries that can be called, and those building blocks can be used to build applications or whatever else. In this case specifically, Steel provides the facility to call Rust functions within a Steel program: https://github.com/mattwparas/steel https://github.com/mattwparas/steel. So, although I haven't used Steel, it looks like the advantage you'd get from using it is the opportunity to take advantage of features it provides like transducers and contracts, which are feature common to some other Lisps as well. So, just like choosing any other language, it boils down to a series of tradeoffs.
- tmtvl 3y ago
  My first introduction to Lisp-like programming languages were the SICP videos with Abelson and Sussman so to me Lisps are just programming languages like any other but with a nicer syntax and great support for interactive programming.
silent_cal 3y ago
Maybe a dumb question but does embeddable mean embeddable in web browsers? If so it would be awesome to include this in a web-based version of SICP.

This is just perfection on first glance, exactly what I've been looking for to augment my Rust projects that need a command/configuration language. And with a working repl, too, apparently. I hope the second and third glances will be good too.

_nalply 3y ago

Helix Editor plans to use Steel as a plugin language [0]. [0]: https://github.com/helix-editor/helix/pull/8675 https://github.com/helix-editor/helix/pull/8675

agumonkey 3y ago
emacsers looking deep from afar
- NeutralForest 3y ago
  I would use Helix in the terminal if it supported Emacs keybindings tbh but I don't want to relearn another set of keybindings. Still I'd be interested in what it becomes.
  - goku12 3y ago
    The keybindings is one of the major selling points of helix. It uses kakoune style object-verb action (like vim visual mode) by default with multiple selections. If you're comfortable with emacs bindings then you're better off with a lightweight emacs alternative.
    - NeutralForest 3y ago
      Fair enough
    - Hugsun 3y ago
      As an evil user, this is potentially huge to me. Emacs happens to have the best balance of easy to setup configuration (relatively), powerful package ecosystem, and proper hackability of all editors I've found. It's not very fast though and has some conventions that feel archaic.
      - goku12 3y ago
        I'm also an Emacs evil user (neovim too). I think the kakoune editing model has the potential to surpass even the vim/evil model. Its default object-verb order makes it easy to preview and change selection before proceeding with the action. That's not possible with vim's normal mode. Vim does have the visual model. But then kakoune model also uses multiple cursors, making it more powerful. I really wanted to try kakoune model in Emacs. But the package needs a bit more updates. Another issue I have with evil is that it changes a lot of Emacs' default bindings, making it hard to do certain tasks. Some operations simply don't work at all. The kakoune package doesn't do this - at least not in insert mode. > It's not very fast though and has some conventions that feel archaic Sadly, multithreading is an afterthought for Emacs. There is just too much legacy stuff to make it easy. The language design is also from another era. The default dynamic binding feels very alien when almost every language everyone knows is lexical binding by default. On the other hand, scheme feel very modern due to very careful language design. But the effort to switch Emacs to scheme didn't find much steam. There is one aspect where none of the new editors (hx with scheme and nvim with lua)can match Emacs. Emacs is entirely written in elisp with the C parts acting merely as libraries. The extension language for other editors is just an addition to their core editing code.
        klibertp 3y ago
        > Sadly, multithreading is an afterthought for Emacs It is, but it's usable. I'm actually amazed that, even after three major versions, the built-in threading is not used by the community. Yes, the threads currently are not usable for number crunching in the background. And yes, there are bugs, and trying to do many things from the background thread doesn't work, sometimes in unexpected ways. You can still block the main thread from the background thread since some things block the event loop, no matter where they were started. But, the threads do give you independent control flows. Whatever you cannot do in the background, you can offload to the main thread with a timer and a queue of lambdas. The built-in threads are very, very bare-bones - it's around 15 functions, for threads, mutexes, and condition variables. They are very limited by their "mostly cooperative" nature. However, with a bit of sugar, they are usable for at least one thing: async processes and network communication. In a background thread, you can "block" to wait for a child process to do something. It's natural and requires no macrology (async.el...). The same is true for network communication. You can block and wait for a response while the rest of Emacs does whatever. With just two functions, you can write code without blocking as if you used `call-process`. Sequential actions - call this, wait for it to finish, call that, wait for it to finish, etc. - can now be coded in a sequential way, without having to worry about callbacks, sentinels, and a poor-man FSM implementation that invariably appears in Elisp that doesn't use threads. The threads built into Emacs, currently, are closer to green threads or coroutines, functionally, than to OS-level threads. But that's still a huge help in a bunch of important and pervasive scenarios. It's really strange that nobody seems to realize this. With threads (as they are), the Continuation Passing Style compiler macro (in generator.el), and dynamic modules (for actual parallelism where needed) Emacs now has everything it needs to make it non-blocking by default. Of course, that would entail rewriting everything on top of these abstractions, so it's unrealistic - but for new code and packages? I think we're just one package (along the lines of dash, s, etc.) away from convenient concurrency and parallelism in Emacs. The problem, of course, is that someone needs to design and code that package...
  - schemescape 3y ago
    Out of curiosity, if you’re familiar with Emacs, what’s your motivation for wanting to use Helix? Built-in LSP support?
    - ParetoOptimal 3y ago
      I use it despite a very complete emaxs setup because: - curiosity - seeing if I'm missing anything - haven't had time to get tree sitter stuff configured, and wanted to see how good it is - see if I want to move back to meow from vanilla (tons of previous vim, evil experience)
    - NeutralForest 3y ago
      - Curiosity - Speed, even though emacs with the emacsclient is pretty much instant - Font support, for some reason the terminal with Emacs sometimes doesn't render as well as it should - Some keybindings issues in the terminal like Ctrl-Backspace not working properly. - See if there are good workflows/defaults I could use in Emacs
nikolay 3y ago
That's ridiculous!
- icommentedtoday 3y ago
  ?
- bobbyskelton41 3y ago
  I saw your comment on the thread. A. Most WebAssembly implementations are bigger than Helix itself. B. WebAssembly is immature. C. Lisp is perfectly fine. Even if you don't like it, it's not the end of the world if you have to use it to configure Helix.
  - nikolay 3y ago
    A. It's true that Wasmer is bigger, but even my phone has 16GB of RAM and 1TB of storage space. B. There are probably more computers running WebAssembly loads today than Lisp ones. C. Lisp is a mature language, almost too mature. I used it extensively in the '80s and it served me well then, but that was 40 years ago. At least have the decency to use a more modern language like Lua, which is what Neovim uses.
    - bobbyskelton41 3y ago
      A. Sure, but it isn't sufficiently beneficial for the cost. B. WebAssembly is immature for developing a plugin system because of the lack of a sufficient ABI: https://github.com/WebAssembly/component-model https://github.com/WebAssembly/component-model C. There aren't any other languages that meet the criteria. Lua was a no-go from the start. The maintainers did not like the language, and it necessitated adding more C code to Helix which could complicate building even further. https://github.com/helix-editor/helix/discussions/3806#discussioncomment-6686976 https://github.com/helix-editor/helix/discussions/3806#discu...
      - nikolay 3y ago
        A Lisp, a weird dialect of Lisp, is not better than Lua. Why use Rune [0]?! [0]: https://rune-rs.github.io/ https://rune-rs.github.io/

sirsuki 3y ago

When I tried to introduce s-expressions to a DSL my co-workers nearly lynched me. The parenthesis were so violently hated I sunk into a deep hole and still haven’t came back out of it.

capableweb 3y ago
It is really fun (sad) to see this, I've seen it so many times myself too. You show how something would be with s-expressions, compared to something, and all they can focus on is how many parenthesis there are. But when you sit down and count, they're the same amount as the code was when it wasn't s-expressions, just in different locations. And when you remove the parenthesis, they can kind of understand the code, kind of.
- fwip 3y ago
  One option that might be suitable for a DSL, is implicit parens based on whitespace. A newline opens a new paren, and the paren closes when it reaches another line with the same indentation e.g: (defun factorial (x) (if (zerop x) 1 (* x (factorial (- x 1))))) could be rewritten as defun factorial (x) if (zerop x) 1 * x (factorial (- x 1))
  - alwaysbeconsing 3y ago
    Whitespace significance might be the choice even more controversial than Lisp parens. For myself, I appreciate both, but those that do not, really do not.
    - capableweb 3y ago
      Probably there are more Python users out there than combined users of all lisp-like languages, which I guess would mean people are less scared of white-space significance than s-expressions :)
  - nsajko 3y ago
    Existing SRFI: SRFI 49 "Indentation-sensitive syntax": https://srfi.schemers.org/srfi-49/srfi-49.html https://srfi.schemers.org/srfi-49/srfi-49.html

oldpersonintx 3y ago

I want this to succeed!! please do not let the word "srfi" ever appear in the packages list...naming libraries with obscure numbers no one remembers was a terrible terrible idea that all schemes seem to perpetuate

mattparas 3y ago
At the moment I only have 1 SRFI package, and its just to check against compatibility :) - my plan is to wrap the SRFI packages with friendly names and just include the metadata in the package spec so someone searching for it can find it easily.
zelphirkalt 3y ago
It is a standard. What do you expect? But there are some or many Schemes that offer their own libs on top or below SRFI implementations, which then have readable names.
- adastra22 3y ago
  I expect people to willing not conform when nonconformance improves the art.
aag 3y ago
SRFI editor here. The numeric designations are there because, among other reasons, there is sometimes more than one SRFI for a particular general idea. But there is nothing stopping libraries from having more than one name, including a semantically meaningful one. In fact, there has been a SRFI standardizing how that is done since 2008 [1]. [1] https://srfi.schemers.org/srfi-97/srfi-97.html https://srfi.schemers.org/srfi-97/srfi-97.html

mattparas 3y ago

This is a surprise! Steel is my project, happy to answer any questions anyone might have

dataangel 3y ago
Any chance of seeing the things I miss the most from Racket in other langs? 1. Parameters and Syntax Parameters (Syntax parameters make macros more powerful) 2. Turing complete macros (not just syntax-case) 3. Typed Racket I almost used https://gamelisp.rs/ https://gamelisp.rs/ for a project but the nightly feature it needs broke and it's no longer maintained, glad to see something similar arise! You might want to consider adopting their choice of VecDeque as a list replacement, I think it makes a lot more sense than naive linked lists on modern machines.
- mattparas 3y ago
  To answer each of these: 1. I do support parameters now, syntax parameters not yet. I would like to! But Racket has a pretty hefty head start on me so it'll take some time. 2. Right now I have syntax-rules macros, I also have defmacro style macros that get used internally in the kernel during expansion, but haven't yet opened them up for user space yet. Syntax case will be coming soon hopefully. 3. The odds of me being able to come up with an implementation to match typed racket pound for pound is pretty low. I have toyed with using contracts as types (where possible), with medium/promising success in certain situations. I have a soft spot for racket and have been modeling behavior after it, however it will take time to be able to create a macro system powerful enough to match it. It wouldn't be impossible to create an alternative syntax to just lower to steel after type checking, but I haven't put time into that. On the list type - the list in use currently is an unrolled linked list https://github.com/mattwparas/im-lists https://github.com/mattwparas/im-lists, which I've found to yield much better performance for iteration than the naive linked lists. When possible, the vm does some in place mutation on the lists as well when consing, which helps performance as well. I also can hot swap for a vlist, but at the moment have stuck with unrolled linked lists.
  - mst 3y ago

falcor84 3y ago

Rust is gradually overcoming C and Fortran and Greenspun's tenth rule [0] is apparently inviolable. [0] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule

deleted 3y ago
[deleted]

ghc 3y ago

Should have named it Steele since it's dragging Rust users half-way to Lisp :).

wolverine876 3y ago
Yes! I just posted https://news.ycombinator.com/item?id=38510054 https://news.ycombinator.com/item?id=38510054
bitwize 3y ago
Why would you want to do that? Lisp is dynamically typed and has a GC.
- Jtsummers 3y ago
  It's a joke that you seem to have missed. > We were not out to win over the Lisp programmers; we were after the C++ programmers. We managed to drag a lot of them about halfway to Lisp. Guy Steele on Java
  - bitwize 3y ago
    I'm familiar with the Steele quote. The manner in which C++ programmers were "dragged halfway to Lisp" concerns primarily manual memory management and all the bugginess attending thereto (Java came out before the STL and smart pointers were widely adopted). Concerns about memory safety were a significant part of the impetus for developing Java in the first place. Steele was responding to complaints that he had turned his back on Lisp, by countering that he was instead bringing the C++ crowd closer to Lisp with a C++-like language that had Lisp-like memory management. But the reason why Rust is such revolutionary computer science and, quite possibly, the most interesting thing to happen to PL design in decades is because with safe Rust you get all the memory-safety advantages of Java or Lisp, without a GC because the borrow checker statically guarantees object lifetimes. So Rust programmers don't need to be dragged halfway to Lisp the way C++ programmers were in the mid-90s, because Rust has the same memory-safety guarantees with none of the drawbacks of GC.

CooCooCaCha 3y ago

How does Steel handle garbage collection? Would it be possible to manually control the garbage collector? For example, a game that runs the GC at the end of every frame.

mattparas 3y ago
Immutable values are reference counted, so for most code, things will be dropped when they exit scope. For captured mutable values, there is a fairly mundane mark and sweep collector. It is possible to manually control the garbage collector, however I have not optimized it for that kind of workload. If you were embedding Steel in a game, I don't think it would be explicitly necessary to tune the GC as long as you aren't using a ton of mutation. If you were using a lot of mutation and still wanted a relatively performant GC collection at the end of every frame, then the underlying GC implementation would have to be changed or swapped to a different one (which is not impossible - I just only have one GC implemented)
- mst 3y ago
  You might find the Recycler algorithm interesting - nim's ORC collector is ARC + Recycler and seems to be working out rather well.
  - mattparas 3y ago
    Thanks for the recommendation, with some brief poking around it does seem promising! It looks like the algorithm is this? https://github.com/fitzgen/bacon-rajan-cc https://github.com/fitzgen/bacon-rajan-cc The link to the paper seems dead unfortunately from this blog post https://nim-lang.org/blog/2020/12/08/introducing-orc.html https://nim-lang.org/blog/2020/12/08/introducing-orc.html I could see how it works as a drop in replacement for Rc
    - zozbot234 3y ago
      The bacon-rajan-cc link says it's stop-the-world. Samsara seems to be fully concurrent.

wolverine876 3y ago

I wonder where the name came from. Being HN, here is my nitpicking imperative: Names are important. Our inheritance is wit, self-awareness, and irony; names that puncture ego and power and that appeal to joy: C, C++, GNU, Rust, Google, Yahoo!, Vim, Git, awk, etc. Others are beautiful, evocative images, like Apple and Amazon. Names communicate our culture and ideals to each other and to the next generation. Careless, thoughtless names like Microsoft, IBM, etc. (ok it's ironic and self-deprecating, but without self-awareness or wit!), etc. should be hated and banned. Egotistical BS, especially Tolkien plagiarists who assert they are supernatural, should be tarred and feathered and paraded around town (with wit and irony). (Plenty of names fall in some middle ground.) If Steel is just a derivative of 'Rust' [edit: it is not, see the response below], it misses the self-awareness. Someone naming their development product - designed to build great structures - 'Rust' is engaging in a little self-deprication, joy, and self-awareness. Naming the derivative project 'Steel' possibly misses all that; there's a reason the original wasn't named Iron or Steel or Carbon Fiber. But maybe there's more to the name.

pasabagi 3y ago
I don't think it's particularly careless: SBCL is short for 'Steel Bank Common Lisp', which is a reference to how the CMU university, named after Carnegie Mellon, could be respectively substituted with 'Steel' (Carnegie) and 'Bank' (Mellon).
- girvo 3y ago
  I, somehow, never put together the SB -> CM link! That's awesome, thanks for sharing
mattparas 3y ago
So here is the history: 1. Guy Steele (along with Gerald Sussman) created scheme, and Steel is close to Steele, just drop the e. 2. Steel is a scheme, and I observed that scheme names have a tendency to be named things crime related: Scheme, Racket (racketeering), Larceny, etc - Not a scientific analysis at all, but I found it funny at the time that Steel sounds like "steal". 3. You made the observation, Steel sounds like something that would be associated with Rust. Beyond that, I just liked the name. No SEO involved, and arguably probably should have picked something more searchable, but I didn't start making it with the intention of there being a lot of users, it started as a project for school.

I_am_tiberius 3y ago

It sounds great but what is it exactly and what can I use it for? Is there an equivalent of this in other languages?

mattparas 3y ago
Hopefully the linked README provides a general overview (I know I need to write some more documentation!), but Steel is an implementation of the scheme programming language (not entirely compliant yet, but aiming for R5RS and R7RS compliance). It can be used as a standalone language via the interpreter/repl (like Python or Racket), or it can be embedded inside applications, like Lua. There are hundreds (thousands, probably) of embeddable languages, each with their own flavor - see a list compiled here for example https://github.com/dbohdan/embedded-scripting-languages https://github.com/dbohdan/embedded-scripting-languages Use cases are generally for either configuration, scripting, or plugins - so scripting in games, or adding extensions to your text editor without having to use FFI or RPC + serializing a bunch of data. The advantage it has over using dynamic libraries (in general) is it runs in the same process, and can access the internal data structures directly without a lot of ceremony involved. The downside is that it is typically not as fast as native code unless a JIT is involved. Javascript is an example of an embedded scripting, where the browser is the host application.
- I_am_tiberius 3y ago
  Understood. Thank you for the summary!

tucnak 3y ago

Is it possible to avoid rust completely when programming using Steel?

mattparas 3y ago
Yep - you can write standalone steel code without interacting with Rust at all, just interacting with the interpreter. I've done the first few days of the advent of code in Steel without needing to touch any Rust. Now, I will say that Steel has gotten visibility faster than I've been able to keep up with, so you might find a native function missing, or something that you would like to use that isn't implemented yet - at which point you either need to implement it yourself or open up an issue for someone to get to :)
- schemescape 3y ago
  Solving Advent of Code problems in a language you implemented yourself should earn you an extra good star or something. Nice work, and cool project!

nerdponx 3y ago

Which Scheme is this implementing? R7RS? Any SRFIs? Is there a library reference document?

mattparas 3y ago
Starting with R5RS compliance, then once that is achieved moving on to R7RS. I borrowed the R5RS test suite from chibi here - https://github.com/mattwparas/steel/blob/master/cogs/r5rs.scm https://github.com/mattwparas/steel/blob/master/cogs/r5rs.sc... - only a few of these tests don't yet pass. Something like 135 pass, 4 fail, 20 skipped since I haven't implemented the primitives yet. I've only tested against a few SRFIs so far, but am also attempting to run the R7RS benchmark suite https://ecraven.github.io/r7rs-benchmarks/ https://ecraven.github.io/r7rs-benchmarks/ - So far you can view the progress here https://github.com/mattwparas/steel/tree/master/r7rs-benchmarks https://github.com/mattwparas/steel/tree/master/r7rs-benchma... There are some more that aren't yet checked in. I plan to get a document up with the exact state of compliance soon. The biggest difference right now is that, like Racket, Steel lists are immutable, so there I need a compatibility layer when running portable scheme.
- nerdponx 3y ago
  Thanks! Was this originally a "just because" project, or do you expect it to offer some improvements or advantages over existing embeddable Schemes like Chibi (as you mentioned)?
  - mattparas 3y ago
    Originally started it as a school project, which then during the pandemic morphed into something to work on while cooped up. It is really a passion project, working on it is fun! There are some interesting design things I wanted to explore, like how to get good performance out of safe Rust, using unrolled linked lists or vlists instead of naive linked lists, using contracts, etc. Chibi is an impressive scheme implementation, and it will take a long time before Steel can hit the same level of compliance as Chibi. There is not a Chibi equivalent in native Rust that I am aware of. There are other embedded scripting languages for Rust that are pleasant - but no schemes of the maturity of Chibi. So in that regard, I'm hoping to offer a compelling scheme in native Rust to make integration with Rust applications relatively easy and painless. I also don't have a particularly strong need to be 100% completely compliant with the scheme specs. The plan is to have compatibility layers so that portable scheme code can be used, however there are things about scheme that I think Racket (for example) improved on, and I'd like to explore that as well as much as I can.

perrygeo 3y ago

Fantastic project! If you're looking to embed a DSL/full programming language into your application, this seems right on point. Rhai and RustPython are two other options if you don't dig parentheses.

SushiHippie 3y ago

Can someone explain these scheme and lisp languages to me? Every time I look at these languages, I can't grasp what you can use them for. And why one would use them. I always feel like I'm missing something. In the example scripts and code snippets, I can see that you can define functions, that you can use lists, mathematical operations, you can build some algorithms, you can print text, but it never goes further than that. I've only used languages like Python, Rust, C, Java, JavaScript, and they all have a very similar vibe, you have a std lib, which can interact with many things, you can build UIs, networking libraries and all that. And I could probably start using any language that is "similar" to these. But I could never use one of these scheme/lisp languages, as I can't really grasp them. Sorry, this comment is all over the place, because I can't really explain what's going on in my head when I see languages like this. I'd call myself a proficient programmer, but every time I look at these languages, it feels like as I've never seen code once in my lifetime. Any help or hint at what I'm missing, is appreciated.

deleted 3y ago
[deleted]
femiagbabiaka 3y ago
Basically the differences are in the concepts you'll use to write code. Lisps themselves are very different from each other, but just like the languages you're used to, many lisp distributions have standard libraries that can be called, and those building blocks can be used to build applications or whatever else. In this case specifically, Steel provides the facility to call Rust functions within a Steel program: https://github.com/mattwparas/steel https://github.com/mattwparas/steel. So, although I haven't used Steel, it looks like the advantage you'd get from using it is the opportunity to take advantage of features it provides like transducers and contracts, which are feature common to some other Lisps as well. So, just like choosing any other language, it boils down to a series of tradeoffs.
tmtvl 3y ago
My first introduction to Lisp-like programming languages were the SICP videos with Abelson and Sussman so to me Lisps are just programming languages like any other but with a nicer syntax and great support for interactive programming.

silent_cal 3y ago

Maybe a dumb question but does embeddable mean embeddable in web browsers? If so it would be awesome to include this in a web-based version of SICP.