5 ms·
The Powershell idea to push structured data through the pipe, which this project fundamentally replicates with a relational structure, has its merits; but it wi
by toyg 7y ago
The Powershell idea to push structured data through the pipe, which this project fundamentally replicates with a relational structure, has its merits; but it will always displease someone, and carries the burden of having to rewrite millions of utilities in order to unlock the real potential.
What if, instead, we pushed the approach “upstream”, asking systems to have an additional stream that all shells can access? We have stdout and stderr, we could add “stddata” - a pipe that expects and produces structured data. Then it would become trivial to add support around the userland, without displeasing anyone - because it’s just an additional interface, not a replacement. The pipe should support two or three formats to start with (csv, json, and maybe xml, so you can handle both tabular and child-parent relations with a varying degree of precision) and shells could have a special character for piping stddata (I like ¥) so everything else about them would stay the same.
- toxik 7y agoThe problem with adding a new standard fd is that lots and lots of code relies on first three fds being stdin out err. It could of course have a high fd number. Would be a great thing!
- dragonsh 7y agoI am not sure how long "ls" will take with 1 million entries in directory, given this paradigm. This has been a problem so I use "ls -1" to list files in bash shell, otherwise it takes forever. Probably it might benefit some. Good to see competition in this area with sh, bash, zsh, fish, conch and many others.
- toyg 7y agoA csv output with one field is basically the same amount of data, but you would be guaranteed that the result can be ingested by a csv parser following the “stddata” dialect. It wouldn’t hurt your usecase, and it would make life easier for other ones.
- StreamBright 7y agoI am at the stage that I am more than happy to rewrite whatever it takes to get better experience. This includes the OS. I think we have sacrificed too much on the altar of backward compatibility. I guess I am just tired of breakage because a file has a space in its name.
- alxlaz 7y ago> I am at the stage that I am more than happy to rewrite whatever it takes to get better experience. This includes the OS Ever tried to rewrite one (and the software on top of it, since we're "including" the OS) :)?
- StreamBright 7y agoYeah, some places I have worked we had a research kernel because we could not measure HW performance accurately with Linux.
- alxlaz 7y agoHey, no cheating -- one that has a substantial baggage of applications on top of it. A research kernel is something you can do in one year working with a few dedicated undergrads. But then so is pretty much every single thing running on top of it. It looks easy until you figure out how much work gets into properly writing a PNG decoder or a PDF viewer. It's not fun, even in a memory-safe language. I admire your optimism and I, too, wish we'd leave this POSIX hellhole behind us (and I'm fond of this POSIX hellhole!) but I think at this stage in the story of computers, it's no longer realistic.
- StreamBright 7y agoTotally agree with you. Now the real question, do you need to implement PNG decoder and PDF viewer? Or you need to design alternatives that are far more safe to implement?
- 7y ago
- arendtio 7y agoWhy not just push all that data through stdout and interpret it at the other end accordingly? Another option would be to just use the 4th fd.
- tlamponi 7y agoUsing te 4th fd is exactly what OP proposed?
- arendtio 7y agoWell, not exactly. He merely suggested using another one. But I thought it would be good to talk something a little bit more specific. And I am not sure he knows there are already others available (for shell scripts at least fd[0-9]). After all, /dev/std{in,out,err} are more or less just convenience links (allowed extensions of POSIX). But using /dev/stderr might fail on some POSIX compliant systems.
- teknopaul 7y agoNot breaking stdin and stdout is key. Looking at the output from that shell, text parsing tools like awk are not going to integrate if you dont have a compatible tool yet. Everything in a base Linux install has to be rewritten to work. Personally i prefer the keep it simple approach. Id much rather /etc/hosts stayed the way it is rather than becoming xml, json or binary. The whole idea of "Power" shell is broken if you ask me. Shell should be simple. If it isnt, write some proper code. And do some code review, and testing first!
- toyg 7y agoYes, backward compatibility is important; that’s why I think adding a new named stream could be the way to go, since it wouldn’t break almost anything (I guess someone might have to reroute something, if it happens to be on the fd that is chosen for the link). I honestly don’t know how much work it would entail from OS devs, but once you have it in Linux and BSDs/Mac that’s basically it. Microsoft could take the chance to be an early adopter, considering they already have all the machinery to output structured data in PS; at that point, whether it’s pure posix or not would matter little in practice. I don’t particularly like PS for a number of reasons, but I do admit that its output facilities can be quite nice. Having something like that in Bash for the few times one really needs it, with standard interfaces and without having to learn a completely new paradigm, would be a nice win imho. This doesn’t mean we should touch /etc/hosts or anything, btw.
- virtualritz 7y agoWhy not add a new parameter, --nushell, to any too that supports the 2-dimensional format? These tools would just be stored in nushell's config file and the shell would silently add --nutshell to any command it ran that was registered to support it.