3 ms·

The researchers main concerns (making shell programs perform better, especially by better leveraging the parallel computing resources of supercomputing environm

by pxc 5y ago

The researchers main concerns (making shell programs perform better, especially by better leveraging the parallel computing resources of supercomputing environments), but they do give a nod to stuff I care more about (usability and breaking out of the limits imposed by extant terminal emulator standards). I hope that this jump starts a bit more research into shell languages. Maybe the veneer of authority given by papers suggesting various improvements will help draw people to next-generation shells that take up those improvements and advertise that. If you were not aware, PowerShell has inspired a whole generation of next-generation shells built around structured pipelines, and they're very exciting. My personal favorite is Elvish, which also draws much inspiration from Fish, a lovely 'old-fashioned' (text-only pipeline) shell with a strong emphasis on interactivity and ease of use. One thing that makes Elvish special besides its ambition to be a pleasant and useful general-purpose programming language is its portability: it's typically distributed as a single static binary, and it even has native support for Windows. When I think of 'the future of the shell', I absolutely think of Elvish, namely with respect to: * better portability * high aims as a 'real' programming language * built-in static checks where possible (for now just syntax, I think, but a more complete type system is on the roadmap) * sophisticated and responsive interactive experience out of the box * structured pipelines, structured pipelines, structured pipelines To me, the performance issues the authors address are secondary to what I've enumerated above. But they're right that the fundamentals of the shell are great as well as overdue for some innovation.

effie 5y ago
I'm skeptical about academia being capable to help shell evolve. The paper is a good example why - they focus on justifying themselves and then on obscure academic ideas they like to pursue (POSIX formalization, highly parallel data processing) rather than usability for most users. Sometimes these align, but most of the time they don't. Your description of Elvish is much better in that respect. I'd love more capable shell with modern advancements like types and static checking. However, I don't get why structured data over pipelines is useful. You can already exchange any kind of data over pipeline as string of bytes. Sender and receiver have to be aware of the format in any case.
- pxc 5y ago
  > I don't get why structured data over pipelines is useful. You can already exchange any kind of data over pipeline as string of bytes. The Elvish docs have a pretty good answer to this here[0], but I'll give my own answer, too. There are a few things. Structured pipelines spare you from having to do escaping and transformations in order to preserve structure.[1] It's also a natural complement to a type system. Some other next-gen shells call their structured pipelines ‘typed pipelines’ instead.[2] Structured pipelines also provide a nice mechanism for separating the visual/textual representation of data from its structure, so you can ‘convert’ values between different display formats without doing any parsing, in a convenient, extensible way. PowerShell has this with Format-List[3] and Format-Table[4], for example. This kind of pretty-printing isn't implemented in Elvish yet, but Kurtis Rader helped me outline one possibility in an open GitHub issue.[5] On some level, structured pipelines just mean that commands ‘native’ to the shell language (builtins and functions written in the shell language) all understand the same types or data structures. This means you can ingest data from some external format (JSON, CSV, TOML, a SQL result, whatever) and then use common tools. Nushell's first example for working with pipelines[6] hints at one of these benefits: you can use a single command, like `inc` to increment the value associated with a certain key in a map, with data that comes from any format, and `inc` doesn't have to know anything about data formats or parsing. Another way to think about it is that it lets pipelines have the same kind of type checking as functions called with arguments. In a typed language, you get type checking for your arguments when you call a function like (some_command arg0 (another_command arg1 arg 2)) but in a way, pipelines are an alternative to subshells.[7] If you want passing values through the pipes to be as robust as passing them as arguments, you need structured pipelines. You're right that if you are using a lot of external commands in a ‘raw’ way, you don't get to take much advantage of the structured pipeline. Imo PowerShell proves that in practice, if you have a big enough library ecosystem, you can leverage wrappers and ‘pure’ tools written in your shell language to get a pretty nice programming experience. — 0: https://elv.sh/learn/unique-semantics.html#motivation https://elv.sh/learn/unique-semantics.html#motivation 1: https://elv.sh/learn/effective-elvish.html#returning-values-with-structured-output https://elv.sh/learn/effective-elvish.html#returning-values-... 2: https://murex.rocks/docs/user-guide/pipeline.html https://murex.rocks/docs/user-guide/pipeline.html 3: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/format-list?view=powershell-7.1 https://docs.microsoft.com/en-us/powershell/module/microsoft... 4: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/format-table?view=powershell-7.1 https://docs.microsoft.com/en-us/powershell/module/microsoft... 5: https://github.com/elves/elvish/issues/1149#issuecomment-705705345 https://github.com/elves/elvish/issues/1149#issuecomment-705... 6: https://www.nushell.sh/book/pipeline.html#basics https://www.nushell.sh/book/pipeline.html#basics 7: https://elv.sh/learn/effective-elvish.html#prefer-pipes-over-parentheses https://elv.sh/learn/effective-elvish.html#prefer-pipes-over...

by pxc 5y ago

effie 5y ago

I'm skeptical about academia being capable to help shell evolve. The paper is a good example why - they focus on justifying themselves and then on obscure academic ideas they like to pursue (POSIX formalization, highly parallel data processing) rather than usability for most users. Sometimes these align, but most of the time they don't. Your description of Elvish is much better in that respect. I'd love more capable shell with modern advancements like types and static checking. However, I don't get why structured data over pipelines is useful. You can already exchange any kind of data over pipeline as string of bytes. Sender and receiver have to be aware of the format in any case.

pxc 5y ago
> I don't get why structured data over pipelines is useful. You can already exchange any kind of data over pipeline as string of bytes. The Elvish docs have a pretty good answer to this here[0], but I'll give my own answer, too. There are a few things. Structured pipelines spare you from having to do escaping and transformations in order to preserve structure.[1] It's also a natural complement to a type system. Some other next-gen shells call their structured pipelines ‘typed pipelines’ instead.[2] Structured pipelines also provide a nice mechanism for separating the visual/textual representation of data from its structure, so you can ‘convert’ values between different display formats without doing any parsing, in a convenient, extensible way. PowerShell has this with Format-List[3] and Format-Table[4], for example. This kind of pretty-printing isn't implemented in Elvish yet, but Kurtis Rader helped me outline one possibility in an open GitHub issue.[5] On some level, structured pipelines just mean that commands ‘native’ to the shell language (builtins and functions written in the shell language) all understand the same types or data structures. This means you can ingest data from some external format (JSON, CSV, TOML, a SQL result, whatever) and then use common tools. Nushell's first example for working with pipelines[6] hints at one of these benefits: you can use a single command, like `inc` to increment the value associated with a certain key in a map, with data that comes from any format, and `inc` doesn't have to know anything about data formats or parsing. Another way to think about it is that it lets pipelines have the same kind of type checking as functions called with arguments. In a typed language, you get type checking for your arguments when you call a function like (some_command arg0 (another_command arg1 arg 2)) but in a way, pipelines are an alternative to subshells.[7] If you want passing values through the pipes to be as robust as passing them as arguments, you need structured pipelines. You're right that if you are using a lot of external commands in a ‘raw’ way, you don't get to take much advantage of the structured pipeline. Imo PowerShell proves that in practice, if you have a big enough library ecosystem, you can leverage wrappers and ‘pure’ tools written in your shell language to get a pretty nice programming experience. — 0: https://elv.sh/learn/unique-semantics.html#motivation https://elv.sh/learn/unique-semantics.html#motivation 1: https://elv.sh/learn/effective-elvish.html#returning-values-with-structured-output https://elv.sh/learn/effective-elvish.html#returning-values-... 2: https://murex.rocks/docs/user-guide/pipeline.html https://murex.rocks/docs/user-guide/pipeline.html 3: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/format-list?view=powershell-7.1 https://docs.microsoft.com/en-us/powershell/module/microsoft... 4: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/format-table?view=powershell-7.1 https://docs.microsoft.com/en-us/powershell/module/microsoft... 5: https://github.com/elves/elvish/issues/1149#issuecomment-705705345 https://github.com/elves/elvish/issues/1149#issuecomment-705... 6: https://www.nushell.sh/book/pipeline.html#basics https://www.nushell.sh/book/pipeline.html#basics 7: https://elv.sh/learn/effective-elvish.html#prefer-pipes-over-parentheses https://elv.sh/learn/effective-elvish.html#prefer-pipes-over...