3 ms·

Hi chubot, I didn't realize that you are developing a shell as well. It's great to see competitions. Can you share some ideas that you learned from the authors

by xiaq 10y ago

Hi chubot, I didn't realize that you are developing a shell as well. It's great to see competitions. Can you share some ideas that you learned from the authors of said shells? I would love to hear those, designing shells is not an easy task. Structured data pipelines run in the same process as elvish, in goroutines. For interacting with external commands you need to serialize and deserialize, like this: ~> put [foo bar lorem ipsum] | to-json | python3 -c 'import sys, json;print(json.dumps([x.upper() for x in json.load(sys.stdin)]))' | from-json ▶ [FOO BAR LOREM IPSUM]

chubot 10y ago
OK cool, that makes sense. I had some good discussions with Michael MacInnis and recently Ilya Sher. I had actually read Michael's masters thesis in 2011 or 2012 and found it to be a very good historical overview of shells: https://scholar.google.com/scholar?cluster=9993440116444147516 https://scholar.google.com/scholar?cluster=99934401164441475... I agree with the goal of "taking shell seriously as a programming language", which I think all projects share. I disagreed on the need to make shell a variant of Lisp. I actually tried this a bit -- I hacked on femtolisp, which is used to bootstrap Julia, and I thought I could bootstrap a good shell with it. This didn't work out as nicely as I had hoped so I dropped the idea. Ilya was having some issues with the "two modes" of parsing -- command mode for shell-like constructs and expression mode for programming features (function calls, math, etc.). I thought this would be an issue too, but as mentioned in another comment, now that I've written a bash parser I think it's straightforward to solve, and Ilya agreed. The thing is that bash already has many different modes of parsing, which I implement using "lexer modes" or lexical state. This same technique can be applied to a new shell and then there should be a natural way of mixing commands and expressions (quoted and unquoted words, etc.): http://www.oilshell.org/blog/2016/10/19.html http://www.oilshell.org/blog/2016/10/19.html I'm also trying to make oil do what bash/dash/etc. do essentially syscall for syscall. I believe that is a little hard with Go because it has its own runtime. Although I think there is a way to pin a goroutine to a single thread which can mitigate those issues. I was planning to start a thread in maybe a month to review some of these issues and exchange feedback. Let me know if you are interested!

by xiaq 10y ago

chubot 10y ago

OK cool, that makes sense. I had some good discussions with Michael MacInnis and recently Ilya Sher. I had actually read Michael's masters thesis in 2011 or 2012 and found it to be a very good historical overview of shells: https://scholar.google.com/scholar?cluster=9993440116444147516 https://scholar.google.com/scholar?cluster=99934401164441475... I agree with the goal of "taking shell seriously as a programming language", which I think all projects share. I disagreed on the need to make shell a variant of Lisp. I actually tried this a bit -- I hacked on femtolisp, which is used to bootstrap Julia, and I thought I could bootstrap a good shell with it. This didn't work out as nicely as I had hoped so I dropped the idea. Ilya was having some issues with the "two modes" of parsing -- command mode for shell-like constructs and expression mode for programming features (function calls, math, etc.). I thought this would be an issue too, but as mentioned in another comment, now that I've written a bash parser I think it's straightforward to solve, and Ilya agreed. The thing is that bash already has many different modes of parsing, which I implement using "lexer modes" or lexical state. This same technique can be applied to a new shell and then there should be a natural way of mixing commands and expressions (quoted and unquoted words, etc.): http://www.oilshell.org/blog/2016/10/19.html http://www.oilshell.org/blog/2016/10/19.html I'm also trying to make oil do what bash/dash/etc. do essentially syscall for syscall. I believe that is a little hard with Go because it has its own runtime. Although I think there is a way to pin a goroutine to a single thread which can mitigate those issues. I was planning to start a thread in maybe a month to review some of these issues and exchange feedback. Let me know if you are interested!