7 ms·

Yes I agree... that's why I took the trouble to write a very complete bash parser, which taught me a few things, and improved the Oil language design. Some fea

by chubot 10y ago

Yes I agree... that's why I took the trouble to write a very complete bash parser, which taught me a few things, and improved the Oil language design. Some features are plain mistakes, but some of them are useful things that newer shells aren't taking into account (particularly with regard to file descriptors). I'm aiming for a superset of bash semantics, but with different syntax. Importantly, it will be backward compatible, because bash can be converted to oil automatically. (I'm working on that now, not released yet.) http://www.oilshell.org/blog/2016/11/09.html http://www.oilshell.org/blog/2016/11/09.html The rest of the blog has a bunch of shell corner cases, like: http://www.oilshell.org/blog/2016/11/06.html http://www.oilshell.org/blog/2016/11/06.html http://www.oilshell.org/blog/2016/10/28.html http://www.oilshell.org/blog/2016/10/28.html Although I probably only blogged about 20% of the weird stuff I found.

taeric 10y ago
Awesome posts, best of luck in this! Any suggestions/requests for users to help? And I realize I made an omission in my post. I meant to say not just learnings, but use. I would think the biggest hurdle for you will be building a user base.
- chubot 10y ago
  Thanks a lot! I'm hoping to set things up for contributions in the next couple months. I'm want to publish the test matrix and get some help because filling out the corners takes a lot of time. The general architecture is pretty sound though -- I hit all the "major areas" of shell, although I may need to make another pass at globbing. If you are adventurous you should be able to clone it, follow the instructions for ./pybuild.sh. Then ./spec.sh install-shells, and then run './spec.sh smoke'. That test should pass, but there are like 15-20 other categories in spec.sh in various states of disarray! Of course this is in Python, but to me it makes sense to fill out a sketch before porting it to C++ (only the runtime; I'm hoping to bootstrap the parsers in oil itself). Not sure if that will inhibit or help contribution.
jcrites 10y ago
Just wanted to say that I really like the ideas you've expressed on the blog. I've found myself thinking similar thoughts, such as about combining shell, awk, make into one language. Keep it up! I'll add that my ideal "command and control language" (as I've been thinking of it) would also be one that's strong at expressing data literals. If there are a number of data types like maps, sets, lists, etc., then it should be easy to initialize them with literals (not true in languages like Java). There are so many configuration file formats that I wonder if they could be subsumed into files with literals expressed in this language (perhaps with a "data mode" to prohibit executable code). I've been wanting to design a language along these lines as well. I'll start by learning about oil, and reach out if I'm interested and able to contribute!
- Vendan 10y ago
  as an interesting note, Lua actually started out as a data entry language (actually called DEL), that eventually morphed into Lua due to demand for more complex logic and such.
- chubot 10y ago
  Thanks! Yes I plan to have data literals -- it's basically going to be JSON, because that is sort of the least common denominator between JS, Python, Ruby, Perl, etc. Shell is a glue language, and JSON is pretty natural at this point. There is also going to be influence from R and CSV files (which goes with awk). I have thought about the config problem a lot -- and a configurable sandboxed "data mode" you are talking about is probably what I will go with. If you squint, the oil shell will look not unlike nginx configs, e.g. a bunch of words with nested {} for blocks. Maybe like https://github.com/vstakhov/libucl https://github.com/vstakhov/libucl or https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl (hm this seems to even have here docs!) This kind of thing was done a lot with Python at my last job, to various degrees of success (e.g. https://bazel.build/ https://bazel.build/ - the build language is derived from Python). Python sort of has some "data mode" sandboxing features, like being able to set __builtins__ when you exec/eval, but probably not enough. It didn't work well enough to prevent people from writing their own config file parser eventually. The syntax is close but not exactly what you want for some use cases.

by chubot 10y ago

taeric 10y ago

Awesome posts, best of luck in this! Any suggestions/requests for users to help? And I realize I made an omission in my post. I meant to say not just learnings, but use. I would think the biggest hurdle for you will be building a user base.

chubot 10y ago
Thanks a lot! I'm hoping to set things up for contributions in the next couple months. I'm want to publish the test matrix and get some help because filling out the corners takes a lot of time. The general architecture is pretty sound though -- I hit all the "major areas" of shell, although I may need to make another pass at globbing. If you are adventurous you should be able to clone it, follow the instructions for ./pybuild.sh. Then ./spec.sh install-shells, and then run './spec.sh smoke'. That test should pass, but there are like 15-20 other categories in spec.sh in various states of disarray! Of course this is in Python, but to me it makes sense to fill out a sketch before porting it to C++ (only the runtime; I'm hoping to bootstrap the parsers in oil itself). Not sure if that will inhibit or help contribution.

jcrites 10y ago

Just wanted to say that I really like the ideas you've expressed on the blog. I've found myself thinking similar thoughts, such as about combining shell, awk, make into one language. Keep it up! I'll add that my ideal "command and control language" (as I've been thinking of it) would also be one that's strong at expressing data literals. If there are a number of data types like maps, sets, lists, etc., then it should be easy to initialize them with literals (not true in languages like Java). There are so many configuration file formats that I wonder if they could be subsumed into files with literals expressed in this language (perhaps with a "data mode" to prohibit executable code). I've been wanting to design a language along these lines as well. I'll start by learning about oil, and reach out if I'm interested and able to contribute!

Vendan 10y ago
as an interesting note, Lua actually started out as a data entry language (actually called DEL), that eventually morphed into Lua due to demand for more complex logic and such.
chubot 10y ago
Thanks! Yes I plan to have data literals -- it's basically going to be JSON, because that is sort of the least common denominator between JS, Python, Ruby, Perl, etc. Shell is a glue language, and JSON is pretty natural at this point. There is also going to be influence from R and CSV files (which goes with awk). I have thought about the config problem a lot -- and a configurable sandboxed "data mode" you are talking about is probably what I will go with. If you squint, the oil shell will look not unlike nginx configs, e.g. a bunch of words with nested {} for blocks. Maybe like https://github.com/vstakhov/libucl https://github.com/vstakhov/libucl or https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl (hm this seems to even have here docs!) This kind of thing was done a lot with Python at my last job, to various degrees of success (e.g. https://bazel.build/ https://bazel.build/ - the build language is derived from Python). Python sort of has some "data mode" sandboxing features, like being able to set __builtins__ when you exec/eval, but probably not enough. It didn't work well enough to prevent people from writing their own config file parser eventually. The syntax is close but not exactly what you want for some use cases.