7 ms·

Just wanted to say that I really like the ideas you've expressed on the blog. I've found myself thinking similar thoughts, such as about combining shell, awk, m

by jcrites 10y ago

Just wanted to say that I really like the ideas you've expressed on the blog. I've found myself thinking similar thoughts, such as about combining shell, awk, make into one language. Keep it up! I'll add that my ideal "command and control language" (as I've been thinking of it) would also be one that's strong at expressing data literals. If there are a number of data types like maps, sets, lists, etc., then it should be easy to initialize them with literals (not true in languages like Java). There are so many configuration file formats that I wonder if they could be subsumed into files with literals expressed in this language (perhaps with a "data mode" to prohibit executable code). I've been wanting to design a language along these lines as well. I'll start by learning about oil, and reach out if I'm interested and able to contribute!

Vendan 10y ago
as an interesting note, Lua actually started out as a data entry language (actually called DEL), that eventually morphed into Lua due to demand for more complex logic and such.
chubot 10y ago
Thanks! Yes I plan to have data literals -- it's basically going to be JSON, because that is sort of the least common denominator between JS, Python, Ruby, Perl, etc. Shell is a glue language, and JSON is pretty natural at this point. There is also going to be influence from R and CSV files (which goes with awk). I have thought about the config problem a lot -- and a configurable sandboxed "data mode" you are talking about is probably what I will go with. If you squint, the oil shell will look not unlike nginx configs, e.g. a bunch of words with nested {} for blocks. Maybe like https://github.com/vstakhov/libucl https://github.com/vstakhov/libucl or https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl (hm this seems to even have here docs!) This kind of thing was done a lot with Python at my last job, to various degrees of success (e.g. https://bazel.build/ https://bazel.build/ - the build language is derived from Python). Python sort of has some "data mode" sandboxing features, like being able to set __builtins__ when you exec/eval, but probably not enough. It didn't work well enough to prevent people from writing their own config file parser eventually. The syntax is close but not exactly what you want for some use cases.
- taeric 10y ago
  I'm curious how far from just dropping to sexps you are looking. While I will not claim that lisp is the be all of languages, when you have a literal syntax that is literally made for linked structures, it becomes a lot easier to represent whatever you are wanting to do. And it you can keep yourself from having tons of "pseudo languages" that are used for different structures.
  - chubot 10y ago
    I've mentioned this a few places so I should probably write a post about it, but I did experiment with Lisp and shell (using femtolisp which is used to bootstrap Julia). There is an obvious similarity in that they both have prefix rather than infix syntax. Tcl explored this design space pretty fruitfully as well. The short answer is that the experiment didn't work very well. If you want a shell based around Lisp, the new "oh shell" is closer to that. It has homoiconic syntax, which I don't agree with. I talked about that a bit here: http://www.oilshell.org/blog/2016/10/28.html http://www.oilshell.org/blog/2016/10/28.html The oh shell is influenced by the es shell, which was almost literally a Lisp. There is also EShell (ELisp) on my Github wiki, and earlier Scheme shell. So without going into too much detail, I think the idea has been tried and it has failed not by accident, but for fundamental reasons. That's not to say that Lisp isn't hugely influential. Julia and Elixir both have very Lisp-like metaprogramming which I hope to take some inspiration from. Without having tried it in detail yet, I think the lesson is that in 2017 you don't need homoiconity to have metaprogramming. And another lesson is that people like syntax (me included!) The shell especially needs syntax because it will often be typed in a terminal as opposed to in an editor with help.
    - taeric 10y ago
      I fully agree that lisp is not the answer for a shell. My question was only for your data literal syntax. Specifically, maps as association lists are very natural in lisp. So are many other structures. And JSON is easily seen as a crappy sexp. XML is easily a complicated sexp. Python's pickling is a complicated eval. And I get it. Some things have a more "natural" syntax that isn't postfix. Even in common lisp, the reader macro is often used for infix things. However, when the examples used are easily translated to sexps, and back, it makes me wonder why we don't think programmers could learn to write them.

by jcrites 10y ago

Vendan 10y ago

as an interesting note, Lua actually started out as a data entry language (actually called DEL), that eventually morphed into Lua due to demand for more complex logic and such.

chubot 10y ago

Thanks! Yes I plan to have data literals -- it's basically going to be JSON, because that is sort of the least common denominator between JS, Python, Ruby, Perl, etc. Shell is a glue language, and JSON is pretty natural at this point. There is also going to be influence from R and CSV files (which goes with awk). I have thought about the config problem a lot -- and a configurable sandboxed "data mode" you are talking about is probably what I will go with. If you squint, the oil shell will look not unlike nginx configs, e.g. a bunch of words with nested {} for blocks. Maybe like https://github.com/vstakhov/libucl https://github.com/vstakhov/libucl or https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl (hm this seems to even have here docs!) This kind of thing was done a lot with Python at my last job, to various degrees of success (e.g. https://bazel.build/ https://bazel.build/ - the build language is derived from Python). Python sort of has some "data mode" sandboxing features, like being able to set __builtins__ when you exec/eval, but probably not enough. It didn't work well enough to prevent people from writing their own config file parser eventually. The syntax is close but not exactly what you want for some use cases.

taeric 10y ago
I'm curious how far from just dropping to sexps you are looking. While I will not claim that lisp is the be all of languages, when you have a literal syntax that is literally made for linked structures, it becomes a lot easier to represent whatever you are wanting to do. And it you can keep yourself from having tons of "pseudo languages" that are used for different structures.
- chubot 10y ago
  I've mentioned this a few places so I should probably write a post about it, but I did experiment with Lisp and shell (using femtolisp which is used to bootstrap Julia). There is an obvious similarity in that they both have prefix rather than infix syntax. Tcl explored this design space pretty fruitfully as well. The short answer is that the experiment didn't work very well. If you want a shell based around Lisp, the new "oh shell" is closer to that. It has homoiconic syntax, which I don't agree with. I talked about that a bit here: http://www.oilshell.org/blog/2016/10/28.html http://www.oilshell.org/blog/2016/10/28.html The oh shell is influenced by the es shell, which was almost literally a Lisp. There is also EShell (ELisp) on my Github wiki, and earlier Scheme shell. So without going into too much detail, I think the idea has been tried and it has failed not by accident, but for fundamental reasons. That's not to say that Lisp isn't hugely influential. Julia and Elixir both have very Lisp-like metaprogramming which I hope to take some inspiration from. Without having tried it in detail yet, I think the lesson is that in 2017 you don't need homoiconity to have metaprogramming. And another lesson is that people like syntax (me included!) The shell especially needs syntax because it will often be typed in a terminal as opposed to in an editor with help.
  - taeric 10y ago
    I fully agree that lisp is not the answer for a shell. My question was only for your data literal syntax. Specifically, maps as association lists are very natural in lisp. So are many other structures. And JSON is easily seen as a crappy sexp. XML is easily a complicated sexp. Python's pickling is a complicated eval. And I get it. Some things have a more "natural" syntax that isn't postfix. Even in common lisp, the reader macro is often used for infix things. However, when the examples used are easily translated to sexps, and back, it makes me wonder why we don't think programmers could learn to write them.