3 ms·

> Projects like Nushell essentially need to handle every type of output from any command, and every type of input to any command, which is an absurd amount of w

by p-e-w 3y ago

> Projects like Nushell essentially need to handle every type of output from any command, and every type of input to any command, which is an absurd amount of work, and just not scalable. I think you're misunderstanding how Nushell works. They don't parse outputs, or generate inputs, from/for standard Unix commands. Instead, they implement their own commands with the same names as standard commands, and generate/consume structured data by default, using the same data structures everywhere. There is only a single implementation of those data structures. That's very easy to maintain. So running `ls` from Nushell does not shell out to the `ls` program on your system, and then try to make sense of its output. It runs a Nushell-internal command that is tailored to the kind of pipelines that Nushell is built around. They already have hundreds such commands implemented and working, and that approach absolutely does scale. Whatever issues may remain, it already works much more reliably than the default Unix tools. Saying that unstructured text streams are a universal interface is like saying that atoms are a universal construction kit – it's technically correct, but pretty useless in practice.

imiric 3y ago
You're right, I misunderstood the way it worked. But I'm not sure that approach is better. They either need to maintain full compatibility with existing tools, or users need to learn the idiosyncrasies of Nushell tools. And commands not reimplemented by Nushell wouldn't work, or they would need some kind of generic wrapper, which would have the drawbacks I mentioned. But, hey, this obviously has users who prefer it, so if this works for you, that's great. Personally, I'll stick to the standard GNU and POSIX tools. I do concede that this is partly due to the robustness of this ecosystem and my familiarity with it, which is hard to abandon. > Saying that unstructured text streams are a universal interface is like saying that atoms are a universal construction kit – it's technically correct, but pretty useless in practice. My point is that offloading the decision of how the tools are integrated beyond raw byte streams to the user, is the most flexible and future-proof approach, with the least overhead for individual tools. Doing anything more sophisticated, while potentially easier for the user, would require maintenance of the glue layer by each tool's developer, or a central maintainer ala Nushell. This loose coupling is a good thing.
- p-e-w 3y ago
  > They either need to maintain full compatibility with existing tools, or users need to learn the idiosyncrasies of Nushell tools. The existing tools aren't fully compatible with each other either. There are significant differences between GNU and BSD tools, for example, and yet more differences with BusyBox and others. The idea of "standard" tools is unfortunately an illusion, so not much is lost there. But more importantly, most of the many options the traditional tools offer are related to output selection and formatting. In Nushell, those problems are solved in a unified way by piping to builtin commands that work with structured data. So instead of learning twenty different cryptic flags for `ls`, you just learn three or four postprocessing commands, and use them for `ls` and everything else.
  - imiric 3y ago
    > The existing tools aren't fully compatible with each other either. Right, but those incompatibilities, as well as the way commands interoperate, are left to the user to resolve. No monolithic tool could realistically make that easier, unless they reimplement everything from scratch, as Nushell has done. But then you have to work with an entirely different and isolated ecosystem, and you depend on a single project to maintain all your workflows for you. Again, the ability for loosely coupled tools to work together is one of the strengths of Unix. We clearly have a difference of opinion here, so let's agree to disagree. :)
  - Brian_K_White 3y ago
    Both gnu and bsd grep input and output a stream of bytes. Without ever having tried it, I know that one random day when I for whatever reason have a wish to take the output from bsd grep and send it over tcp through netcat, to be collected by zsh's built-in tcp support and fed in to gnu's grep, it will work. No piece along the way made any jerk assumptions or required any any jerk tight coupling, and the bsd and gnu tools were completely compatible. That is more valuable and greater convenience than any other poorly concieved ideas to make it more convenient. This all 50 years after unix pipes were invented and in an environment the inventors did not even try to predict and handle. Instead they handled infinity by not trying to predict anything. They just made useful low level tools which you assemble however you may turn out to need to, and the tools make as few as possible assumptions which will all eventually break. The hammer doesn't only work with one kind of nail.

by p-e-w 3y ago

imiric 3y ago

You're right, I misunderstood the way it worked. But I'm not sure that approach is better. They either need to maintain full compatibility with existing tools, or users need to learn the idiosyncrasies of Nushell tools. And commands not reimplemented by Nushell wouldn't work, or they would need some kind of generic wrapper, which would have the drawbacks I mentioned. But, hey, this obviously has users who prefer it, so if this works for you, that's great. Personally, I'll stick to the standard GNU and POSIX tools. I do concede that this is partly due to the robustness of this ecosystem and my familiarity with it, which is hard to abandon. > Saying that unstructured text streams are a universal interface is like saying that atoms are a universal construction kit – it's technically correct, but pretty useless in practice. My point is that offloading the decision of how the tools are integrated beyond raw byte streams to the user, is the most flexible and future-proof approach, with the least overhead for individual tools. Doing anything more sophisticated, while potentially easier for the user, would require maintenance of the glue layer by each tool's developer, or a central maintainer ala Nushell. This loose coupling is a good thing.

p-e-w 3y ago
> They either need to maintain full compatibility with existing tools, or users need to learn the idiosyncrasies of Nushell tools. The existing tools aren't fully compatible with each other either. There are significant differences between GNU and BSD tools, for example, and yet more differences with BusyBox and others. The idea of "standard" tools is unfortunately an illusion, so not much is lost there. But more importantly, most of the many options the traditional tools offer are related to output selection and formatting. In Nushell, those problems are solved in a unified way by piping to builtin commands that work with structured data. So instead of learning twenty different cryptic flags for `ls`, you just learn three or four postprocessing commands, and use them for `ls` and everything else.
- imiric 3y ago
  > The existing tools aren't fully compatible with each other either. Right, but those incompatibilities, as well as the way commands interoperate, are left to the user to resolve. No monolithic tool could realistically make that easier, unless they reimplement everything from scratch, as Nushell has done. But then you have to work with an entirely different and isolated ecosystem, and you depend on a single project to maintain all your workflows for you. Again, the ability for loosely coupled tools to work together is one of the strengths of Unix. We clearly have a difference of opinion here, so let's agree to disagree. :)
- Brian_K_White 3y ago
  Both gnu and bsd grep input and output a stream of bytes. Without ever having tried it, I know that one random day when I for whatever reason have a wish to take the output from bsd grep and send it over tcp through netcat, to be collected by zsh's built-in tcp support and fed in to gnu's grep, it will work. No piece along the way made any jerk assumptions or required any any jerk tight coupling, and the bsd and gnu tools were completely compatible. That is more valuable and greater convenience than any other poorly concieved ideas to make it more convenient. This all 50 years after unix pipes were invented and in an environment the inventors did not even try to predict and handle. Instead they handled infinity by not trying to predict anything. They just made useful low level tools which you assemble however you may turn out to need to, and the tools make as few as possible assumptions which will all eventually break. The hammer doesn't only work with one kind of nail.