6 ms·

I'm interested in and excited about oil, but three things have stopped me every time I've tried to give it a go. 1. The complete and utter lack of useful docum

I'm interested in and excited about oil, but three things have stopped me every time I've tried to give it a go. 1. The complete and utter lack of useful documentation. There's not even a link to any docs from the main page of the shell's site. You have to get to the docs from the release page, [1] but I had to find this out by Googling. Given the number of blog-post-like entries in the docs, seems like it would have been less effort and more effective just to write standardized and complete documentation from the start. Just as an example: can anyone tell me what built-ins Oil has, how to use them, and where they are documented? I'll give you a hint. I found out from one of the blog posts that oil has "min" and "max" commands. The following is valid Oil code: const x = min(1, 2) But what other commands does Oil have, and how do I use them? I have no idea. Some of the doc pages 404 so maybe they're supposed to be there. I have found some pages listing built-ins which are mostly undocumented, but none of them list min or max, so I know they're not complete. 2. It's still pretty buggy and lots of features are missing. Development has seemingly moved slowly over the last several years. If it ever did get to the point where I could make it a daily driver then I'd be willing to track down and report bugs, of course. 3. I'm a little concerned by how much emphasis is put on POSIX / Bash compatibility. Unless there's going to be a way to fully disable what Oil considers "legacy" language support in a script, then that is going to leave a ton of footguns for people who want to write "modern" Oil code. It's not clear to me whether that's the plan or not (I haven't read every one of the blog posts), but it seems like a mistake to me to sacrifice possible syntax improvements on the altar of compatibility. To be clear, I realize this is basically someone's solo project, and I am impressed by how far it's been taken, just clarifying why I don't think it's ready for general use yet. [1] https://www.oilshell.org/release/latest/doc/ https://www.oilshell.org/release/latest/doc/

brundolf 5y ago
Is its goal to be "a shell language that's decent to program in"?
- chubot 5y ago
  Yes, the home page says that it's for Python and JavaScript users that avoid shell: https://www.oilshell.org/ https://www.oilshell.org/ Also see The Simplest Explanation of Oil https://www.oilshell.org/blog/2020/01/simplest-explanation.html https://www.oilshell.org/blog/2020/01/simplest-explanation.h... In addition to being "decent" / familiar / learnable, it's also more powerful, as bash is showing its age and lacking features many users would like (recursive data structures, JSON, etc.)
chubot 5y ago
(author here) I don't think anything you wrote should be a surprise to regular readers of the blog. Many of my blog posts in the last year say "the scope of the project is too large" and similar stuff. http://www.oilshell.org/why.html#why-not-use-it http://www.oilshell.org/why.html#why-not-use-it It might be a surprise if you're only reading what shows up on news aggregators. But to be clear, I basically write about what I'm thinking, what progress has been made, etc. And I do "documentation-driven development", which is IMO the best way to design a consistent and coherent language. The main purpose is to get people to test out the shell, and attract developers who want to push in the same direction. I like this way of working, as I've gotten a lot of valuable feedback and contributions on both OSH and Oil. And I can use more. Oil is NOT a "product" which you can download and use. (It doesn't even have a logo, partly for that reason.) Right now it's better for say people who have the desire and ability to make it through the quick start for contributing: https://github.com/oilshell/oil/wiki/Contributing https://github.com/oilshell/oil/wiki/Contributing (how to get a working bin/osh from git in 30 seconds on most Linux distros. You can then hack on it in pure Python and it will be fast) ---- The "summer blog backlog" has some project status updates that aren't out yet, but I'll give the tl;dr version here. There are roughly 5 equal sized parts of project, all large: 1. The compatible OSH language (mature, NOT buggy, but not fast yet) 2. The Oil language (this is 2 years old; there are bugs due to the way we reuse CPython) 3. The interactive shell (punting this to "headless shell") 4. Semi-automatic translation of the "executable spec" from Python to C++ 5. Documentation For context, last October I wrote a post saying that four features (that are DONE and polished) justify the OSH language and I can punt on the Oil language until the indefinite future. https://www.oilshell.org/blog/2020/10/osh-features.html https://www.oilshell.org/blog/2020/10/osh-features.html So that was one way of cutting scope. But basically I changed my mind based on prior successes. So I worked on the Oil language a lot this year, and I'm pleased with how its coming together. (The summer blog backlog has sketches of new features. Big things left: the "missing declarative part" of shell for configuration/not-YAML; functions/coprocesses as opposed to shell-like procs) I made a big list of alternative shells, and none of them is what I want, or what I think the open source ecosystem needs: https://github.com/oilshell/oil/wiki/Alternative-Shells https://github.com/oilshell/oil/wiki/Alternative-Shells This is a very long argument and has to do with the "Perlis-Thompson Principle" and "narrow waists", which I'm about 25% through explaining on the blog. ----- So the other way of cutting scope is punting the interactive shell into another process, with Oil being a "headless shell": See these two comments for details: https://lobste.rs/s/xestey/opinionated_guide_xargs#c_cn4r2i https://lobste.rs/s/xestey/opinionated_guide_xargs#c_cn4r2i https://lobste.rs/s/sdum3p/if_you_could_rewrite_anything_from#c_dmij7k https://lobste.rs/s/sdum3p/if_you_could_rewrite_anything_fro... (yes I need to make blog posts out of these) To clearly repeat what's in those posts: you won't get more than a basic interactive shell based on Oil unless someone else makes big effort. Join #shell-gui on Oil's Zulip, etc. ---- So the project is now focused on #1 #2 and #5: OSH and Oil and documentation. The translation to C++ passes about half the tests, but it has stalled. So I have to come up with some way of getting that done too, and it might involve soliciting donations to hire someone, some kind of two-sided market etc. But right now I'm just concentrating on the Oil language design and explaining it. So you might see many more compelling posts about things you won't be able to immediately use. That is on purpose! It all absolutely works -- download it now and try it -- but there is a gap to bridge to make it production quality, which involves documentation and translation to C++. (This post is from a year ago but may give some color on these issues: https://www.oilshell.org/blog/2020/08/risks.html https://www.oilshell.org/blog/2020/08/risks.html) In addition to direct collaboration, I also encourage parallel efforts. Someone started porting Oil to Nim, and another person wanted to rewrite Oil only (not OSH) in Rust, etc. I encourage all those efforts. The project is long, but the "hard part" OSH is quite mature (you should test it), and the "exciting part" Oil is less than 2 years old. And five years ago people said it was impossible to evolve bash into a good language. Now people seem to be taking it for granted that it should exist :)
smichael 5y ago
The linked Tour is part of the documentation, which at a glance looks extremely extensive. You mention some issues; are you saying these docs are too hard to discover and lacking a good-enough TOC or index ?
- bscphil 5y ago
  In addition to them being both of those things, they are also, so far as I can tell (which is incomplete because of the lack of an index), missing documentation for most functions. If you click through to some of the other documentation for parts of Oil, you'll see that they're mostly written in a "blog post" style, with the intent of giving a reader a brief tour of a couple of features, while leaving everything else completely undocumented. For an example of good documentation, look at the docs for the Python OS module. [1] You'll see that every single function in the OS module is listed, on order, with clarifications on its usage, what values it returns, what arguments it needs, and gives clarifying examples where needed. For a project that's much younger than Python (though by no means "new"), you don't expect quite as much. But to compare: the page documenting the command language 404s [2], the help file that I believe ships with osh documents a few commands but has huge missing gaps (and doesn't mention min and max at all) [3], and the page on strings has more of a theoretical discussion on strings than practical documentation and has major sections labeled "todo" [4]. [1] https://docs.python.org/3/library/os.html https://docs.python.org/3/library/os.html [2] https://www.oilshell.org/release/latest/doc/command-language.html https://www.oilshell.org/release/latest/doc/command-language... [3] https://www.oilshell.org/release/latest/doc/osh-help.html https://www.oilshell.org/release/latest/doc/osh-help.html [4] https://www.oilshell.org/release/latest/doc/strings.html https://www.oilshell.org/release/latest/doc/strings.html
  - chubot 5y ago
    Yeah the reference is supposed to be "OSH Help Topics" and "Oil Help Topics", which are linked from this page: https://www.oilshell.org/release/0.9.1/doc/ https://www.oilshell.org/release/0.9.1/doc/ They list everything concisely on one page. But the body of the docs is mostly not done. https://www.oilshell.org/release/0.9.1/doc/osh-help-topics.html https://www.oilshell.org/release/0.9.1/doc/osh-help-topics.h... https://www.oilshell.org/release/0.9.1/doc/oil-help-topics.html https://www.oilshell.org/release/0.9.1/doc/oil-help-topics.h... The release/$VERSION/ page lists the docs that are done, while /release/$VERSION/doc/ has all docs, including ones in progress. Anything that's in progress has a pink banner at the top. I got help filling out all the OSH builtins last year, and can use more help. ---- The 404 is an error which I noted here 2 days ago, and will be fixed in the next release: https://oilshell.zulipchat.com/#narrow/stream/121540-oil-discuss/topic/Oil.200.2E9.2E1.20Released https://oilshell.zulipchat.com/#narrow/stream/121540-oil-dis... If you see any errors like that in 0.9.2 or later, please file a bug. Also send feedback about things you would like documented. (But some things simply aren't done enough to be documented, like the list of builtin functions.)

brundolf 5y ago

Is its goal to be "a shell language that's decent to program in"?

chubot 5y ago
Yes, the home page says that it's for Python and JavaScript users that avoid shell: https://www.oilshell.org/ https://www.oilshell.org/ Also see The Simplest Explanation of Oil https://www.oilshell.org/blog/2020/01/simplest-explanation.html https://www.oilshell.org/blog/2020/01/simplest-explanation.h... In addition to being "decent" / familiar / learnable, it's also more powerful, as bash is showing its age and lacking features many users would like (recursive data structures, JSON, etc.)

chubot 5y ago

(author here) I don't think anything you wrote should be a surprise to regular readers of the blog. Many of my blog posts in the last year say "the scope of the project is too large" and similar stuff. http://www.oilshell.org/why.html#why-not-use-it http://www.oilshell.org/why.html#why-not-use-it It might be a surprise if you're only reading what shows up on news aggregators. But to be clear, I basically write about what I'm thinking, what progress has been made, etc. And I do "documentation-driven development", which is IMO the best way to design a consistent and coherent language. The main purpose is to get people to test out the shell, and attract developers who want to push in the same direction. I like this way of working, as I've gotten a lot of valuable feedback and contributions on both OSH and Oil. And I can use more. Oil is NOT a "product" which you can download and use. (It doesn't even have a logo, partly for that reason.) Right now it's better for say people who have the desire and ability to make it through the quick start for contributing: https://github.com/oilshell/oil/wiki/Contributing https://github.com/oilshell/oil/wiki/Contributing (how to get a working bin/osh from git in 30 seconds on most Linux distros. You can then hack on it in pure Python and it will be fast) ---- The "summer blog backlog" has some project status updates that aren't out yet, but I'll give the tl;dr version here. There are roughly 5 equal sized parts of project, all large: 1. The compatible OSH language (mature, NOT buggy, but not fast yet) 2. The Oil language (this is 2 years old; there are bugs due to the way we reuse CPython) 3. The interactive shell (punting this to "headless shell") 4. Semi-automatic translation of the "executable spec" from Python to C++ 5. Documentation For context, last October I wrote a post saying that four features (that are DONE and polished) justify the OSH language and I can punt on the Oil language until the indefinite future. https://www.oilshell.org/blog/2020/10/osh-features.html https://www.oilshell.org/blog/2020/10/osh-features.html So that was one way of cutting scope. But basically I changed my mind based on prior successes. So I worked on the Oil language a lot this year, and I'm pleased with how its coming together. (The summer blog backlog has sketches of new features. Big things left: the "missing declarative part" of shell for configuration/not-YAML; functions/coprocesses as opposed to shell-like procs) I made a big list of alternative shells, and none of them is what I want, or what I think the open source ecosystem needs: https://github.com/oilshell/oil/wiki/Alternative-Shells https://github.com/oilshell/oil/wiki/Alternative-Shells This is a very long argument and has to do with the "Perlis-Thompson Principle" and "narrow waists", which I'm about 25% through explaining on the blog. ----- So the other way of cutting scope is punting the interactive shell into another process, with Oil being a "headless shell": See these two comments for details: https://lobste.rs/s/xestey/opinionated_guide_xargs#c_cn4r2i https://lobste.rs/s/xestey/opinionated_guide_xargs#c_cn4r2i https://lobste.rs/s/sdum3p/if_you_could_rewrite_anything_from#c_dmij7k https://lobste.rs/s/sdum3p/if_you_could_rewrite_anything_fro... (yes I need to make blog posts out of these) To clearly repeat what's in those posts: you won't get more than a basic interactive shell based on Oil unless someone else makes big effort. Join #shell-gui on Oil's Zulip, etc. ---- So the project is now focused on #1 #2 and #5: OSH and Oil and documentation. The translation to C++ passes about half the tests, but it has stalled. So I have to come up with some way of getting that done too, and it might involve soliciting donations to hire someone, some kind of two-sided market etc. But right now I'm just concentrating on the Oil language design and explaining it. So you might see many more compelling posts about things you won't be able to immediately use. That is on purpose! It all absolutely works -- download it now and try it -- but there is a gap to bridge to make it production quality, which involves documentation and translation to C++. (This post is from a year ago but may give some color on these issues: https://www.oilshell.org/blog/2020/08/risks.html https://www.oilshell.org/blog/2020/08/risks.html) In addition to direct collaboration, I also encourage parallel efforts. Someone started porting Oil to Nim, and another person wanted to rewrite Oil only (not OSH) in Rust, etc. I encourage all those efforts. The project is long, but the "hard part" OSH is quite mature (you should test it), and the "exciting part" Oil is less than 2 years old. And five years ago people said it was impossible to evolve bash into a good language. Now people seem to be taking it for granted that it should exist :)

smichael 5y ago

The linked Tour is part of the documentation, which at a glance looks extremely extensive. You mention some issues; are you saying these docs are too hard to discover and lacking a good-enough TOC or index ?

bscphil 5y ago
In addition to them being both of those things, they are also, so far as I can tell (which is incomplete because of the lack of an index), missing documentation for most functions. If you click through to some of the other documentation for parts of Oil, you'll see that they're mostly written in a "blog post" style, with the intent of giving a reader a brief tour of a couple of features, while leaving everything else completely undocumented. For an example of good documentation, look at the docs for the Python OS module. [1] You'll see that every single function in the OS module is listed, on order, with clarifications on its usage, what values it returns, what arguments it needs, and gives clarifying examples where needed. For a project that's much younger than Python (though by no means "new"), you don't expect quite as much. But to compare: the page documenting the command language 404s [2], the help file that I believe ships with osh documents a few commands but has huge missing gaps (and doesn't mention min and max at all) [3], and the page on strings has more of a theoretical discussion on strings than practical documentation and has major sections labeled "todo" [4]. [1] https://docs.python.org/3/library/os.html https://docs.python.org/3/library/os.html [2] https://www.oilshell.org/release/latest/doc/command-language.html https://www.oilshell.org/release/latest/doc/command-language... [3] https://www.oilshell.org/release/latest/doc/osh-help.html https://www.oilshell.org/release/latest/doc/osh-help.html [4] https://www.oilshell.org/release/latest/doc/strings.html https://www.oilshell.org/release/latest/doc/strings.html
- chubot 5y ago
  Yeah the reference is supposed to be "OSH Help Topics" and "Oil Help Topics", which are linked from this page: https://www.oilshell.org/release/0.9.1/doc/ https://www.oilshell.org/release/0.9.1/doc/ They list everything concisely on one page. But the body of the docs is mostly not done. https://www.oilshell.org/release/0.9.1/doc/osh-help-topics.html https://www.oilshell.org/release/0.9.1/doc/osh-help-topics.h... https://www.oilshell.org/release/0.9.1/doc/oil-help-topics.html https://www.oilshell.org/release/0.9.1/doc/oil-help-topics.h... The release/$VERSION/ page lists the docs that are done, while /release/$VERSION/doc/ has all docs, including ones in progress. Anything that's in progress has a pink banner at the top. I got help filling out all the OSH builtins last year, and can use more help. ---- The 404 is an error which I noted here 2 days ago, and will be fixed in the next release: https://oilshell.zulipchat.com/#narrow/stream/121540-oil-discuss/topic/Oil.200.2E9.2E1.20Released https://oilshell.zulipchat.com/#narrow/stream/121540-oil-dis... If you see any errors like that in 0.9.2 or later, please file a bug. Also send feedback about things you would like documented. (But some things simply aren't done enough to be documented, like the list of builtin functions.)