4 ms·

Thanks for posting this. I’ve been thinking about the design philosophy behind this tool we use every day, and this was interesting to read. Does anyone recom

Thanks for posting this. I’ve been thinking about the design philosophy behind this tool we use every day, and this was interesting to read. Does anyone recommend any other papers or writings, classic or new, that shed light on how shells have evolved and where they might be going in the future?

chubot 5y ago
There are Usenix papers on bash and ksh -- I archived the ksh one here: http://www.oilshell.org/archive/ksh-usenix.pdf http://www.oilshell.org/archive/ksh-usenix.pdf The surprising thing to me is that ksh was Lua-like in terms of being embeddable and Perl-/Python-like in terms of having GUI bindings(!) and being a rich language. Though I don't think it has garbage collection and thus was fundamentally more limited in its data structures than Perl/Python/JS/Ruby/PHP, etc. My viewpoint is that shell should be more like the dynamic languages that won -- which are basically those 5. I elaborated on that here: https://old.reddit.com/r/ProgrammingLanguages/comments/n1y2pd/lowlevel_and_sandboxed_scripting_all_in_a_single/gwkcl4l/ https://old.reddit.com/r/ProgrammingLanguages/comments/n1y2p... Probably one of the longest documents is a 182 page Master's thesis from the author of the Oh shell, linked here: https://github.com/oilshell/oil/wiki/Alternative-Shells https://github.com/oilshell/oil/wiki/Alternative-Shells There are many sections, but one of them goes into history, and other sections explore alternatives. ----- Ken Thompson's original paper on shell just popped up: https://news.ycombinator.com/item?id=25235450 https://news.ycombinator.com/item?id=25235450 (it was scanned and revived) There are also interviews with Stephen Bourne around. I would say that those 4 shells form the "dominant" strain: Thompson shell -> Bourne Shell -> ksh -> bash, although there are many offshoots. ---- If you want some fun reading that's not about the shell specifically but about Unix and language-oriented interfaces, check out "In the Beginning Was the Command Line" by Neal Stephenson (search HN archives, it has appeared many many times). Also I recommend "The Art of Unix Programming" by Raymond for how to design programs that work well with shell. It has history and Ken Thompson gives some commentary (in the print version, at least). ---- I have a lot of thoughts on where shell is going in the future and I try to put most of them on https://www.oilshell.org/ https://www.oilshell.org/ ! Although it's easy to design something in your head but often hard to implement a solid version of it. For example I still think shell, awk, and make should be combined but that was over 4 years ago :) http://www.oilshell.org/blog/2016/11/13.html http://www.oilshell.org/blog/2016/11/13.html (I have made some progress on those fronts by deferring some tasks to Ninja, which is very Unix-y and nice. And some of the foundations for awk-like structured data are now in Oil)
macintux 5y ago
Not quite what you asked for, but the creator of oilshell is often here and has given quite a bit of thought to what a shell is and why create a new one: https://www.oilshell.org/ https://www.oilshell.org/
- chubot 5y ago
  Thanks! In addition to the other links I posted, here are high level thoughts on shell: http://www.oilshell.org/blog/2021/01/philosophy-design.html http://www.oilshell.org/blog/2021/01/philosophy-design.html http://www.oilshell.org/blog/2021/04/build-ci-comments.html http://www.oilshell.org/blog/2021/04/build-ci-comments.html (We need a language and abstraction for containers that composes, and that runs locally; containers relate to data dependencies, which relates to Ninja and make, which are shell-like) I try not to let the ideas get too far ahead of implementation, so those posts mostly link to comments that speculate on ideas. One big change in the last month, that I haven't blogged about, is a new headless mode which I just mentioned a few hours ago: https://news.ycombinator.com/item?id=27055926 https://news.ycombinator.com/item?id=27055926 The idea is basically to separate shell into 2 processes: UI and interpreter/runtime. Shell is a text- and language-oriented interface, but it doesn't have to be a terminal-oriented one! This has an initial implementation that I'm pretty excited about. If you want to write a shell UI but don't want to write a language, join #shell-gui on on the oilshell Zulip (link on home page). ---- Also it's great to see this work on shell parallelization, and looking forward to hearing about it at HotOS :)
  - akamaka 5y ago
    Sounds promising!
1vuio0pswjnm7 5y ago
https://www.in-ulm.de/~mascheck/ https://www.in-ulm.de/~mascheck/
- akamaka 5y ago
  This is great, thank you!
evmar 5y ago
I have been collecting new shell ideas here: http://evmar.github.io/smash/related.html http://evmar.github.io/smash/related.html

chubot 5y ago

There are Usenix papers on bash and ksh -- I archived the ksh one here: http://www.oilshell.org/archive/ksh-usenix.pdf http://www.oilshell.org/archive/ksh-usenix.pdf The surprising thing to me is that ksh was Lua-like in terms of being embeddable and Perl-/Python-like in terms of having GUI bindings(!) and being a rich language. Though I don't think it has garbage collection and thus was fundamentally more limited in its data structures than Perl/Python/JS/Ruby/PHP, etc. My viewpoint is that shell should be more like the dynamic languages that won -- which are basically those 5. I elaborated on that here: https://old.reddit.com/r/ProgrammingLanguages/comments/n1y2pd/lowlevel_and_sandboxed_scripting_all_in_a_single/gwkcl4l/ https://old.reddit.com/r/ProgrammingLanguages/comments/n1y2p... Probably one of the longest documents is a 182 page Master's thesis from the author of the Oh shell, linked here: https://github.com/oilshell/oil/wiki/Alternative-Shells https://github.com/oilshell/oil/wiki/Alternative-Shells There are many sections, but one of them goes into history, and other sections explore alternatives. ----- Ken Thompson's original paper on shell just popped up: https://news.ycombinator.com/item?id=25235450 https://news.ycombinator.com/item?id=25235450 (it was scanned and revived) There are also interviews with Stephen Bourne around. I would say that those 4 shells form the "dominant" strain: Thompson shell -> Bourne Shell -> ksh -> bash, although there are many offshoots. ---- If you want some fun reading that's not about the shell specifically but about Unix and language-oriented interfaces, check out "In the Beginning Was the Command Line" by Neal Stephenson (search HN archives, it has appeared many many times). Also I recommend "The Art of Unix Programming" by Raymond for how to design programs that work well with shell. It has history and Ken Thompson gives some commentary (in the print version, at least). ---- I have a lot of thoughts on where shell is going in the future and I try to put most of them on https://www.oilshell.org/ https://www.oilshell.org/ ! Although it's easy to design something in your head but often hard to implement a solid version of it. For example I still think shell, awk, and make should be combined but that was over 4 years ago :) http://www.oilshell.org/blog/2016/11/13.html http://www.oilshell.org/blog/2016/11/13.html (I have made some progress on those fronts by deferring some tasks to Ninja, which is very Unix-y and nice. And some of the foundations for awk-like structured data are now in Oil)

macintux 5y ago

Not quite what you asked for, but the creator of oilshell is often here and has given quite a bit of thought to what a shell is and why create a new one: https://www.oilshell.org/ https://www.oilshell.org/

chubot 5y ago
Thanks! In addition to the other links I posted, here are high level thoughts on shell: http://www.oilshell.org/blog/2021/01/philosophy-design.html http://www.oilshell.org/blog/2021/01/philosophy-design.html http://www.oilshell.org/blog/2021/04/build-ci-comments.html http://www.oilshell.org/blog/2021/04/build-ci-comments.html (We need a language and abstraction for containers that composes, and that runs locally; containers relate to data dependencies, which relates to Ninja and make, which are shell-like) I try not to let the ideas get too far ahead of implementation, so those posts mostly link to comments that speculate on ideas. One big change in the last month, that I haven't blogged about, is a new headless mode which I just mentioned a few hours ago: https://news.ycombinator.com/item?id=27055926 https://news.ycombinator.com/item?id=27055926 The idea is basically to separate shell into 2 processes: UI and interpreter/runtime. Shell is a text- and language-oriented interface, but it doesn't have to be a terminal-oriented one! This has an initial implementation that I'm pretty excited about. If you want to write a shell UI but don't want to write a language, join #shell-gui on on the oilshell Zulip (link on home page). ---- Also it's great to see this work on shell parallelization, and looking forward to hearing about it at HotOS :)
- akamaka 5y ago
  Sounds promising!

1vuio0pswjnm7 5y ago

https://www.in-ulm.de/~mascheck/ https://www.in-ulm.de/~mascheck/

akamaka 5y ago
This is great, thank you!

evmar 5y ago

I have been collecting new shell ideas here: http://evmar.github.io/smash/related.html http://evmar.github.io/smash/related.html