4 ms·

Shallow and composable is something we all experience when using UNIX tooling. GUIs are where this all falls apart as they are literal islands that don’t commu

Shallow and composable is something we all experience when using UNIX tooling. GUIs are where this all falls apart as they are literal islands that don’t communicate with each other in a composable manner. I’ve been experimenting with some GUI-meets-shell-pipeline ideas with a tool I’ve been working on call guish. https://github.com/williamcotton/guish https://github.com/williamcotton/guish I’m curious to know if anyone knows of any similar tools or approaches to composable GUIs!

wryl 2y ago
This is really, really cool! Definitely going to share this around, I love tools like this. They bring so much visibility into dark spaces.
- williamcotton 2y ago
  Thanks! I started on it a week ago so it is just barely presentable at this point. I expect there are many glitches and bugs that I am completely unaware of! The parser is off-the-shelf and seems robust. The AST-to-bash aspect is a bit messy. Wrapping arguments with spaces in them with either " or ' is somewhat of an unsolved problem. Like, it's hard to tell if someone wants to embed a bash variable and thus wants " or if they are using something like awk that uses $var itself and thus need '. We'll see how it goes!
  - wryl 2y ago
    It's still really impressive! Comparisons were thrown around to nushell, which is also a really impressive project. I can see the vision and I hope to follow along! :)
marcosdumay 2y ago
Yes, shallow, wide, and composable. That's how our abstractions should be. But the larger problem is not GUIs. GUIs are a problem, but they are necessarily at the top of the abstraction stack, so the problem doesn't compose any further. (What interestingly means they are so much of a problem that they aren't anymore.) The elephant in the room nowadays are distributed systems.
- williamcotton 2y ago
  I would love to hear your thoughts on how distributed systems lead to deep, narrow and isolated abstractions. I can think of some obvious reasons but it seems like you’ve put more thought into this than I!
  - marcosdumay 2y ago
    I haven't put anywhere near enough thought to decide on something simple. There are two main dynamics that I noticed: - Each distributed component has a high minimal cost to create and maintain, what leads people to consolidate them into more complex, fewer ones; - You'll invariably need to interact with some component that doesn't share your values. So you need security, contracts (the legal kind), consumer protection, etc... and that creeps into your abstraction. Outside of that, there's one thing that isn't inherent but does happen every single time is information erasure. Distributed interfaces are way less detailed than the ones on monolithic systems.
chubot 2y ago
This looks pretty cool! I agree -- a big reason I like Unix, and find it productive, is that it's "shallow and composable". But GUIs have long been a weakness. I guess because there are many "global" concerns -- whether a UI is nice isn't a "modular" property. Personally I would more of a UI, but still retain automation That is, the property of shell where I can just saved what I typed in a file, and then run it later And modify it and run it again Or I can copy and paste a command, send it to my friend in an e-mail, etc. --- As context, I've been working on a from-scratch shell for many years, and it has a "headless mode" meant for GUIs. There are real demos that other people have written, but nobody's working on it at the moment. Screenshots: https://www.oilshell.org/blog/2023/12/screencasts.html#headless-protocol-oils-web_shell https://www.oilshell.org/blog/2023/12/screencasts.html#headl... https://www.oilshell.org/blog/tags.html?tag=headless#headless https://www.oilshell.org/blog/tags.html?tag=headless#headles... More links here - https://github.com/oilshell/oil/wiki/Interactive-Shell https://github.com/oilshell/oil/wiki/Interactive-Shell - some interesting (inactive) projects like Xiki If you find the need for a compatible shell that's divorced from the terminal, or a NEW shell that is, feel free to let me know (by e-mail or https://oilshell.zulipchat.com https://oilshell.zulipchat.com ) Basically we need to people to test out the headless protocol and tell us how it can be improved. I think we should make a shell GUI that HAS a terminal, but IS NOT a terminal -- which looks like it has some relation to what you're building Right now we're working mostly on the new YSH language, but I'd like to revive the GUI work too ... I'm not an experienced UI programmer, so it would be nice to have some different viewpoints :) --- Also, I'm a big fan of ggplot, so I'm glad you included it there. Actually ggplot is exactly where I miss having graphics in shell!
- williamcotton 2y ago
  I’m going to play around a little bit with this headless mode and see if I can come up with anything interesting! Is there a place y’all hang out or a way you communicate? I foresee needing some guidance in places.
  - chubot 2y ago
    Oh yes feel free to join https://oilshell.zulipchat.com/ https://oilshell.zulipchat.com/ -- there is a whole #shell-gui channel . We are mainly developing OSH and YSH now, but it would be nice to work on the interactive shell too The key idea is that OSH and YSH can run "headless" with the "FANOS" protocol (file descriptors and netstrings over unix domain socket), which has a tiny implementations in C and Python The reason we use FD passing with Unix sockets is so that anything that uses terminal still works: ls --color pip install cargo build All these programs will output ANSI color if `isatty(stdout)` is true (roughly speaking). Most people didn't quite get that -- they thought we should use "nREPL" or something -- but my opinion is that you really need a specific protocol to communicate with a shell, because a shell spawns other programs you still want to work. --- Here is a pure Python 3 implementation of the recv() side of FANOS in only ~100 lines https://github.com/oilshell/oil/blob/master/client/py_fanos.py https://github.com/oilshell/oil/blob/master/client/py_fanos.... It uses the recvmsg() binding . I'm pretty sure node.js has that too? i.e. it has Unix domain socket support --- Anyway, thanks for taking a look - we're interested in feedback!
brabel 2y ago
> I’m curious to know if anyone knows of any similar tools or approaches to composable GUIs! I believe the emacs architecture is the answer. emacs is not just a CLI, it's not just a GUI, it's a wonderful (but archaic) matchup of both, where they integrate seamlessly. I am currently working on some ideas on this as well, my idea even looks superficially similar to yours, but I would say you should look into how this works in emacs as I feel like your approach is not really very "composable" (though I haven't looked very deep, sorry). EDIT: perhaps this may also inspire you, in case you don't know it: https://gtoolkit.com/ https://gtoolkit.com/ (Smalltalk environment where literally everything is programmable, just like in emacs but kind of the other way around: the GUI is language itself, not a result of its commands).
- williamcotton 2y ago
  I feel like your approach is not really very "composable" It is as composable as a shell pipeline because all it does is make shell pipelines!
  - brabel 2y ago
    I was thinking of the GUI itself: how can you compose GUI components?
m3047 2y ago
I'm a fan of KNIME (https://www.knime.com/ https://www.knime.com/) for those "Sunday Driver" pipelines that I use heavily for a few days or a week and then shelve for months at a time. It uses Python / Pandas as its code layer.
cben 2y ago
Some projects in a similar space: https://hisham.hm/userland/ https://hisham.hm/userland/ https://arcan-fe.com/2021/04/12/introducing-pipeworld/ https://arcan-fe.com/2021/04/12/introducing-pipeworld/ http://conal.net/papers/Eros/ http://conal.net/papers/Eros/

wryl 2y ago

This is really, really cool! Definitely going to share this around, I love tools like this. They bring so much visibility into dark spaces.

williamcotton 2y ago
Thanks! I started on it a week ago so it is just barely presentable at this point. I expect there are many glitches and bugs that I am completely unaware of! The parser is off-the-shelf and seems robust. The AST-to-bash aspect is a bit messy. Wrapping arguments with spaces in them with either " or ' is somewhat of an unsolved problem. Like, it's hard to tell if someone wants to embed a bash variable and thus wants " or if they are using something like awk that uses $var itself and thus need '. We'll see how it goes!
- wryl 2y ago
  It's still really impressive! Comparisons were thrown around to nushell, which is also a really impressive project. I can see the vision and I hope to follow along! :)

marcosdumay 2y ago

Yes, shallow, wide, and composable. That's how our abstractions should be. But the larger problem is not GUIs. GUIs are a problem, but they are necessarily at the top of the abstraction stack, so the problem doesn't compose any further. (What interestingly means they are so much of a problem that they aren't anymore.) The elephant in the room nowadays are distributed systems.

williamcotton 2y ago
I would love to hear your thoughts on how distributed systems lead to deep, narrow and isolated abstractions. I can think of some obvious reasons but it seems like you’ve put more thought into this than I!
- marcosdumay 2y ago
  I haven't put anywhere near enough thought to decide on something simple. There are two main dynamics that I noticed: - Each distributed component has a high minimal cost to create and maintain, what leads people to consolidate them into more complex, fewer ones; - You'll invariably need to interact with some component that doesn't share your values. So you need security, contracts (the legal kind), consumer protection, etc... and that creeps into your abstraction. Outside of that, there's one thing that isn't inherent but does happen every single time is information erasure. Distributed interfaces are way less detailed than the ones on monolithic systems.

chubot 2y ago

This looks pretty cool! I agree -- a big reason I like Unix, and find it productive, is that it's "shallow and composable". But GUIs have long been a weakness. I guess because there are many "global" concerns -- whether a UI is nice isn't a "modular" property. Personally I would more of a UI, but still retain automation That is, the property of shell where I can just saved what I typed in a file, and then run it later And modify it and run it again Or I can copy and paste a command, send it to my friend in an e-mail, etc. --- As context, I've been working on a from-scratch shell for many years, and it has a "headless mode" meant for GUIs. There are real demos that other people have written, but nobody's working on it at the moment. Screenshots: https://www.oilshell.org/blog/2023/12/screencasts.html#headless-protocol-oils-web_shell https://www.oilshell.org/blog/2023/12/screencasts.html#headl... https://www.oilshell.org/blog/tags.html?tag=headless#headless https://www.oilshell.org/blog/tags.html?tag=headless#headles... More links here - https://github.com/oilshell/oil/wiki/Interactive-Shell https://github.com/oilshell/oil/wiki/Interactive-Shell - some interesting (inactive) projects like Xiki If you find the need for a compatible shell that's divorced from the terminal, or a NEW shell that is, feel free to let me know (by e-mail or https://oilshell.zulipchat.com https://oilshell.zulipchat.com ) Basically we need to people to test out the headless protocol and tell us how it can be improved. I think we should make a shell GUI that HAS a terminal, but IS NOT a terminal -- which looks like it has some relation to what you're building Right now we're working mostly on the new YSH language, but I'd like to revive the GUI work too ... I'm not an experienced UI programmer, so it would be nice to have some different viewpoints :) --- Also, I'm a big fan of ggplot, so I'm glad you included it there. Actually ggplot is exactly where I miss having graphics in shell!

williamcotton 2y ago
I’m going to play around a little bit with this headless mode and see if I can come up with anything interesting! Is there a place y’all hang out or a way you communicate? I foresee needing some guidance in places.
- chubot 2y ago
  Oh yes feel free to join https://oilshell.zulipchat.com/ https://oilshell.zulipchat.com/ -- there is a whole #shell-gui channel . We are mainly developing OSH and YSH now, but it would be nice to work on the interactive shell too The key idea is that OSH and YSH can run "headless" with the "FANOS" protocol (file descriptors and netstrings over unix domain socket), which has a tiny implementations in C and Python The reason we use FD passing with Unix sockets is so that anything that uses terminal still works: ls --color pip install cargo build All these programs will output ANSI color if `isatty(stdout)` is true (roughly speaking). Most people didn't quite get that -- they thought we should use "nREPL" or something -- but my opinion is that you really need a specific protocol to communicate with a shell, because a shell spawns other programs you still want to work. --- Here is a pure Python 3 implementation of the recv() side of FANOS in only ~100 lines https://github.com/oilshell/oil/blob/master/client/py_fanos.py https://github.com/oilshell/oil/blob/master/client/py_fanos.... It uses the recvmsg() binding . I'm pretty sure node.js has that too? i.e. it has Unix domain socket support --- Anyway, thanks for taking a look - we're interested in feedback!

brabel 2y ago

> I’m curious to know if anyone knows of any similar tools or approaches to composable GUIs! I believe the emacs architecture is the answer. emacs is not just a CLI, it's not just a GUI, it's a wonderful (but archaic) matchup of both, where they integrate seamlessly. I am currently working on some ideas on this as well, my idea even looks superficially similar to yours, but I would say you should look into how this works in emacs as I feel like your approach is not really very "composable" (though I haven't looked very deep, sorry). EDIT: perhaps this may also inspire you, in case you don't know it: https://gtoolkit.com/ https://gtoolkit.com/ (Smalltalk environment where literally everything is programmable, just like in emacs but kind of the other way around: the GUI is language itself, not a result of its commands).

williamcotton 2y ago
I feel like your approach is not really very "composable" It is as composable as a shell pipeline because all it does is make shell pipelines!
- brabel 2y ago
  I was thinking of the GUI itself: how can you compose GUI components?

m3047 2y ago

I'm a fan of KNIME (https://www.knime.com/ https://www.knime.com/) for those "Sunday Driver" pipelines that I use heavily for a few days or a week and then shelve for months at a time. It uses Python / Pandas as its code layer.

cben 2y ago

Some projects in a similar space: https://hisham.hm/userland/ https://hisham.hm/userland/ https://arcan-fe.com/2021/04/12/introducing-pipeworld/ https://arcan-fe.com/2021/04/12/introducing-pipeworld/ http://conal.net/papers/Eros/ http://conal.net/papers/Eros/