10 ms·

Schemesh: Fusion between Unix shell and Lisp REPL

• dreamcompiler 2y ago
  I'm curious as to how this differs from an older project that seems to solve the same problem: https://github.com/scheme/scsh https://github.com/scheme/scsh
  • cosmos0072 2y ago
    This is a common question :) Schemesh is intended as an interactive shell and REPL: it supports line editing, autocompletion, searchable history, aliases, builtins, a customizable prompt, and automatic loading of `~/.config/schemesh/repl_init.ss`. Most importantly, it has job control (CTRL+Z, `fg`, `bg` etc.) and recognizes and extends Unix shell syntax for starting, redirecting and composing jobs. An example: find (lisp-expression-returning-some-string) -type f | xargs ls 2>/dev/null >> ls.log & Scsh has none of the above features. As stated in https://scsh.net/docu/html/man-Z-H-2.html#node_sec_1.4 https://scsh.net/docu/html/man-Z-H-2.html#node_sec_1.4 > Scsh, in the current release, is primarily designed for the writing of shell scripts -- programming. It is not a very comfortable system for interactive command use: the current release lacks job control, command-line editing, a terse, convenient command syntax, and it does not read in an initialisation file analogous to .login or .profile
    • sshine 2y ago
      I really like how you don’t sacrifice complete command-line first shell feel, and escaping into a sane language with real datastructures is literally one character away. Rather than the tclsh way of saying “we’ll just make the Lisp seem really shelly” which is a dud to anyone who is not a lisper. Now, it’d be really cool if schemesh had a TUI library at the maturity level of Ratatui. So... it sacrifices sub-shell syntax with parentheses being hijacked for Scheme. Have you also lost $(...) shell interpolation as the saner alternative to `...`?
      • cosmos0072 2y ago
        It does not sacrifice sub-shell syntax: it is fully supported, I just had to rename it from ( ... ) to [ ... ] to avoid conflicts with ( ... ) that switches to lisp syntax Also, both $(...) and `...` shell interpolation are fully supported. The only thing I intentionally sacrificed is shell flow control: schemesh shell syntax does not have the builtins 'case' 'for' 'if' 'while' etc. In practically all examples I tried, escaping to Scheme for loops and conditional code works better: it avoids the usual pitfalls related to shell string splitting, and usually results in more readable code too, at least for a Lisper Note: there's also some additional parsing logic to distinguish between sub-shell syntax [ ... ] and wildcard patterns that use [ ... ] as well
        • sshine 2y ago
          Then [ shadows the sh executable aliased from ‘test’ so that you can no longer do [ some conditional ] && ... But have to write test some conditional && ...
          • cosmos0072 2y ago
            That's true. In shells, "test" and "[" are often used after "if", as for example if test -f "some_file" do_something fi if [ "$FOO" != "" ] do_something_else fi Schemesh does not have a shell builtin "if", you switch to Scheme for that: (if (file-regular? "some_file') (sh-run {do_something})) Thus the need for "test" and its alias "[" is reduced. Also, "test" implements a mini-language full of one-letter operators: `-f FILE` `COND1 -a COND2` `COND1 -o COND2` etc. I really don't miss it, as I find the equivalent in Scheme to be more readable - and of course more general (file-regular? "FILE") (and COND1 COND2) (or COND1 COND2) etc.
    • nemoniac 2y ago
      > This is a common question :) It'd be really great if you could put your answer in the readme. It was the first question that came to my mind when looking at your project. I'm looking forward to trying out schemesh!
  • lisper 2y ago
    SCSH is a shell embedded in Scheme, i.e. it's a Scheme library that lets you easily create unix processes in Scheme. Schemesh is Scheme embedded in a shell, i.e. it's a shell that lets you easily call Scheme code. For example, if you type: ls in schemesh you will execute the "ls" command and get a directory listing, whereas in scsh you will get the value of a variable named "ls". [UPDATE] Also, as cosmos0072 notes in a sibling comment, schemesh has shell-like features like line editing, autocompletion, searchable history, aliases, builtins, etc.
• behnamoh 2y ago
  speaking of shell pipelines, what is the "right" way of implementing pipes? - Elixir: data |> process(12) puts data as the FIRST arg of process (before 12). - Gleam: data |> process(12, _) puts data as the "hole" arg ("_") of process. So far so good, but these approaches are mainly just more convenient function calls - i.e., they don't have fancy error checking in them. Then you have Haskell: - Haskell: >>= "binds" actions to guarantee execution order (even for actions that don't depend on the previous action's output!). This is more fancy because it uses monads to encapsulate the computations at each step, and can shortcircuit on errors.
  • dan-robertson 2y ago
    I’m not sure that |> operators are the right analogy, but fwiw: Clojure does either first or last position depending on the operator, and it offers lightweight lambdas similar to your second option The natural choice for a language like Haskell is final position: the rhs of the |> will be partially applied an |> has type a -> (a -> b) -> b In R, things go in the last slot I think but most arguments on the right hand side would be passed as keywords so the ‘last’ slot would often be the first argument. The whole point of Unix pipes is that execution is parallel so I’m not totally sure I get your point about guaranteeing execution order.
  • isr 2y ago
    I think you're conflating "chaining function calls together" (aka "threading function calls") with unix pipelines, which are all about running separate programs in parallel & connecting their io streams together (with the kernel regulating the flow of data between them). Threading functions together is basically about being able to write read -> eval -> print -> loop rather than: (loop (print (eval (read))))
• rednafi 2y ago
  So many shells, so little time.
• susam 2y ago
  Excellent project! Thanks for sharing it here! I'd just like to share my joy in using the Emacs shell (Eshell), which I find to be a wonderful fusion of the Unix shell and a Lisp REPL. You can enter commands with or without parentheses. For example: ~ $ concat "foo" "bar" foobar ~ $ (concat "foo" "bar") foobar Or ~ $ * 123 456 56088 ~ $ (* 123 456) 56088 Eshell comes with several built-in commands implemented as Elisp functions. For example: ~ $ which cd echo ls which eshell/cd is a byte-compiled Lisp function in ‘em-dirs.el’. eshell/echo is a byte-compiled Lisp function in ‘em-basic.el’. eshell/ls is a byte-compiled Lisp function in ‘em-ls.el’. eshell/which is a byte-compiled Lisp function in ‘esh-cmd.el’. ~ $ ls -l /etc/h* -rw-r--r-- 1 root wheel 446 2025-02-15 20:43 /etc/hosts -rw-r--r-- 1 root wheel 0 2024-10-01 2024 /etc/ hosts.equiv ~ $ (eshell/ls "-l" (eshell-extended-glob "/etc/h*")) -rw-r--r-- 1 root wheel 446 2025-02-15 20:43 /etc/hosts -rw-r--r-- 1 root wheel 0 2024-10-01 2024 /etc/hosts.equiv Of course, you can still run external commands like usual: ~ $ which curl jq python3 rustc /usr/bin/curl /opt/homebrew/bin/jq /opt/homebrew/bin/python3 /Users/susam/.cargo/bin/rustc ~ $ python3 -c "print('hello')" hello ~ $ curl -sS https://hacker-news.firebaseio.com/v0/item/43061183.json | jq -r .title Schemesh: Fusion between Unix shell and Lisp REPL Since TRAMP is an integral part of Emacs, you can switch between the local shell and remote shells transparently with simple 'cd' commands. For example: ~ $ echo local > /tmp/foo.txt ~ $ echo remote > /ssh:susam@susam.net:/tmp/foo.txt ~ $ cd /tmp/ /tmp $ cat foo.txt local /tmp $ hostname mac.local ~ $ cd /ssh:susam@susam.net:/tmp/ /ssh:susam@susam.net:/tmp $ hostname susam.net /ssh:susam@susam.net:/tmp $ cat foo.txt remote In the second command, I redirected a file to a remote file system with the usual '>' redirection operator. Notice how, in the sixth command, I switched from my local shell to a remote shell with a simple 'cd' command. With Eshell and TRAMP, working across multiple remote systems becomes transparent, seamless, and effortless! Best of all, I still have the full power of Emacs at my fingertips, making Eshell an incredibly smooth and powerful experience!
  • cosmos0072 2y ago
    Eshell looks really powerful :) Does it also have job control, and jobs as first-class objects? In schemesh, you can do things like find / -xdev -type f | ls -l --sort=size CTRL+Z bg 1 and also (define j {git log}) (display j) (display (sh-run/string j))
• whartung 2y ago
  Can this shell do this (in some form): (lisp expression) | <unix command> | (lisp expression)
  • cosmos0072 2y ago
    [REWRITTEN FOR CLARITY] Yes, although the current syntax is cumbersome - I am thinking how to improve it. The first part is easy. If you want to run something like (lisp-expr1-produces-a-string) | grep foo the current solution is echo (lisp-expr1-produces-a-string) | grep foo The second part, i.e. feeding a command's output into a Scheme function, is more cumbersome. If you want to run echo (lisp-expr1) | grep foo | (lisp-expr2) the current solution requires (sh-run/string job), namely: (lisp-expr2-accepts-string-arg (sh-run/string {echo (lisp-expr1) | grep foo})) If instead you have a (lisp-expr2...) that reads from an integer file descriptor passed as argument - not a Scheme I/O port - you can write (lisp-expr2-accepts-integer-file-descriptor (sh-start/fd-stdout {echo (lisp-expr1) | grep foo})) [UPDATE] There is also a function (sh-redirect job redirection-args ...) - it can add arbitrary redirections to a job, including pipes, but it's quite low-level and verbose to use
    • cosmos0072 2y ago
      I found another, possibly simpler solution. The functions (sh-fd-stdin) (sh-fd-stdout) and (sh-fd-stderr) return the integer file descriptors that a schemesh builtin should use to perform I/O. With them, you can do true (lambda () (lisp-expr-writes-to-fd (sh-fd-stdout))) | grep foo | true (lambda () (lisp-expr-reads-from-fd (sh-fd-stdin))) It should work :)
      • pama 2y ago
        Could this be abstracted enough with the right macros to make a subset of useful lisp commands play well with the shell? It could be a powerful way to extend the shell for interactive use.
        • cosmos0072 2y ago
• codr7 2y ago
  Lisp makes total sense as a shell scripting language to me. Being able to switch back/forth without leaving the prompt is nice.
• elviejo 2y ago
  How does this project compare to RaSH: Racket sell? Rash for me seemed like the perfect blend of lisp (racket) and external commands. https://youtu.be/yXcwK3XNU3Y?si=v0LuWWkqfoHvkaHl https://youtu.be/yXcwK3XNU3Y?si=v0LuWWkqfoHvkaHl
  • Bogdanp 2y ago
    Rash is excellent. I use it in production for a process that copies large parquet files on the daily by shelling out to the aws command, then processes them by shelling out to another command and then pushes the processed data over HTTP to another server using a Racket HTTP client[1]. My only complaint would be that the docs could use some cleaning up. [1]: https://docs.racket-lang.org/http-easy/index.html https://docs.racket-lang.org/http-easy/index.html
    • willghatch 2y ago
      The docs are terrible, and I've always meant to go back and improve them... but it's never quite been a priority and I've never actually summoned the motivation to do it... But it's always nice to hear when someone uses it and likes it despite that!
  • cosmos0072 2y ago
    Rash and schemesh start from similar ideas: create a shell scriptable in some dialect of Lisp. Rash has several limitations, sometimes due to design choices, that schemesh solves: 1. no job control 2. multi-line editing is limited 3. from what I understand, shell syntax is available only at REPL top level. Once you switch to Lisp syntax with `(`, you can return to shell syntax only with `)`. Thus means you cannot embed shell syntax inside Lisp syntax, i.e. you cannot do `(define j {find -type f | less})` 4. shell commands are Lisp functions, not Lisp objects. Inspecting and redirecting them after they have been created is difficult 5. Rash is written in Racket, which has larger RAM footprint than schemesh running on vanilla Chez Scheme: at startup, ~160MB vs. ~32MB 6. Racket/Rash support for multi-language at REPL is limited: once you do `#lang racket`, you cannot go back to `#lang rash`
    • 2y ago
• netbioserror 2y ago
  For Janet fans, there's janetsh (https://github.com/andrewchambers/janetsh https://github.com/andrewchambers/janetsh). Seems very elegant indeed.
  • andrewchambers 2y ago
    Author here - thanks for linking - Unfortunately I didn't have time to continue on this for a long while. I am certainly happy if people get inspiration from any ideas and continue on with it.
• tombert 2y ago
  I tried doing one of these too! With Chicken Scheme. I got pretty far, I abandoned the project when the computer I was working on was destroyed, and I hadn't committed and pushed the majority of the work. I'm glad that someone is actually doing it though; a Scheme shell always seemed like it could have a lot of potential for scripting.
• owl57 2y ago
  By the way, that's a regionally cool name. I read it at first as "shemesh", that means "Sun" in Hebrew.
• ews 2y ago
  This is amazing, did you think about packing this for guix?
  • -__---____-ZXyw 2y ago
    https://guix-hpc.gitlabpages.inria.fr/guix-packager/ https://guix-hpc.gitlabpages.inria.fr/guix-packager/ try copy-pasting in here yourself and see what happens :)
• j1mr10rd4n 2y ago
  LISPy REPLs are awesome. Babashka is another amazing tool for interacting with a shell with clojure (or a very close dialect thereof). https://babashka.org https://babashka.org
• a2code 2y ago
  Does it support other scheme implementations? Or at least any plans to do so?
  • cosmos0072 2y ago
    Porting to a different Scheme implementation requires some effort: schemesh needs a good, bidirectional C FFI and an (eval) that allows any Scheme form, including definitions. For creating a single `schemesh` executable with the usual shell-compatible options and arguments, the Scheme implementation also needs to be linkable as a library from C: Chez Scheme provides a `kernel.o` or `libkernel.a` library that you can link into C code, then call the C functions Sscheme_init(), Sregister_boot_file() and finally Scall0(some_scheme_repl_procedure) or Sscheme_start()
• hnlmorg 2y ago
  As an author of an alternative shell myself, I really have to commend the effort and design of this one. It’s one of the more impressive and genuinely interesting shells I’ve seen for a while.
• Levitating 2y ago
  I am curious if this could also be implemented as a command named ( Similar to [ and [[
  • cosmos0072 2y ago
    You mean a command or builtin `(` inside a traditional shell as bash or zsh? It would be quite limited: internal status would not persist between invocations, and it would only be able to exchange unstructured data (a stream of bytes) with the shell
• cess11 2y ago
  I don't know, I kind of like having a boundary that separates my programming language from the shell, like Picolisp in/out/call, Elixir System.shell, PHP shell_exec, racket/port and so on. And I do an awful lot of shelling out, usually as a poor person's FFI or concurrency, or to just interact with a chain of shell pipes.
• deleted 2y ago
  [deleted]
• melvinroest 2y ago
  This is the first time I'm curious to potentially actually learn Lisp. Using Lisp as a shell language is where it's at. Where's my book The Little Schemer? :D