8 ms·
The compelling idea here is that they convert the output of common shell commands into tabular data that can be manipulated using common operators, so that you
by pixelmonkey 7y ago
The compelling idea here is that they convert the output of common shell commands into tabular data that can be manipulated using common operators, so that you don't have the remember sorting/filtering/grouping flags that may be different for every different shell command. So, imagine being able to use the same sorting/filtering logic on the output of `ls` as you might on the output of `ps`, and without relying on hacky solutions like pipelines to `grep`, `cut`, and `sort`.
It also means shell command output can be easily transposed into JSON and CSV. Actually pretty clever!
- entangledqubit 7y agoI've been delving into bash scripting a bit more than I'd like as of late and the lack of universally available consistent structured output for the CLI really got to me. Most of the script contents end up being these obfuscating and brittle "hacky solutions" that never should have been necessary. When I thought about pursuing fixing this I felt like the task was a bit overwhelming. I'm delighted that these developers are working on this!
- kbenson 7y agoI've generally found you can get most of what you need done with head, tail, grep, cut, paste, sed and tr (and maybe a couple others I'm missing). Oh, and test (both bash's and coreutil's). It can be a little hacky, but once you realize the crazy output you sometimes have to parse from third party programs, you realize that the ability to handle any free form text is pretty essential. That said, since I'm a Perl developer, I'll generally just go straight to perl's inlined implicit loop mode (-p or -n) if I need to use more than a couple of those utils, unless I want it to be portable with a minimum of requirements (as even though more binaries are used in the shell version, they're all essentially guaranteed to exist on a Linux/Unix).
- munchbunny 7y agoYou can also just use PowerShell. Its chief crime is its verbosity in naming, but it has the "universally available consistent structured output for the CLI" you're asking for.
- pitaj 7y agoNaw its chief crime is not having `&&`
- samstave 7y agoIm speaking from a position of naivete, so dont attack me; Would an ideal situation be “do + while” Whereas one may write While x is true do “blah blah” && if output is still true then end elif do &&? Or is that basically how it already works and we just have shitty interactions with the machines to get them too do the above (unless youre a more seniorbash person ((its one of my career regrets that i didnt spend a ton of time bash scripting)))
- pitaj 7y agoThere are ways around it but nothing as easy as `&&` in other shells
- nailer 7y agopwsh 7 is adding it
- travbrack 7y agoIs it actually useful outside Windows? I know the latest version is now portable because of .net core, but I haven't tried it on *nix.
- felixfbecker 7y agoYeah I'm using it as my primary shell on macOS. You can do everything you do with bash, but with objects, autocompletion etc
- vips7L 7y agoThe only thing I cant find in PS is fishlike predictability when using it as a shell.
- 7y ago
- bigtrakzapzap 7y agoThe key flaw of UNIX philosophy is destructuring deserialization and reserialization based on lines, necessitating all manner of argument escaping and field delimiters, when pipelines should be streams of typed messages that encapsulate data in typed fields. Logs especially (logging to files is a terrible idea, because it creates log rotation headaches and each program requires a log parser because of the loss of structured information) also. Line-oriented pipeline processing is fundamentally too simple. Settling on a common data simple/universal format robust enough for all purposes, including very large data sets, to exchange between programs and is just complicated enough to eliminate escapement and delimiter headaches without throwing away flexibility (by a new/refined set of processing tools/commands) is key.
- whatshisface 7y agoIt's not so much a flaw as a tradeoff. Because it's so easy to produce lines of text, virtually every program supports it, even programs written by hackers who aren't interested in following OS communication standards.
- bigtrakzapzap 7y agoNo, that's your opinion, not mine. It's a flaw because it creates escaping magic, delimiters that are in-band (spaces) and every program has to know how to parse the output of every other program, wasting human time and processing time. Structured communication, say like an open standard for a schema-less, self-describing, flexible de/serialization like protobufs/msgpack, would be far superior, more usable, more efficient and simple but no too simple to process streams of data with structure and programmability already there. Being able to dump structured information out of an exception directly into a log, and then from a log into a database, without any loss of information or extraneous log parsing, is a clear win. Or from a command as simple as listing files ("ls") into a database or into any other tool or program. Outputing only line-oriented strings is just throwing away type information, and creates more work for everyone else, even more so than continuing to stay with lines processing tools.
- 7y ago
- j88439h84 7y agoI agree. The idea of Mario is to use Python objects instead of text lines. This means it's much more structured and has a familiar syntax as well. https://github.com/python-mario/mario https://github.com/python-mario/mario
- z3t4 7y agoThats when you switch to a real programming language. Relying on output from other programs will always be brittle - as error messages etc will change every new dist release.
- yread 7y agoYeah just iterating through a directory with filenames with spaces..
- jayd16 7y agoSeems its only one level deep, no tables of tables are shown. Powershell has a lot of issues but one cool thing is you can access output as nested object data. Seems like the logical evolution.
- ecnahc515 7y agoSeems similar to the purpose of `osquery` which is SQL based.
- j88439h84 7y agoI agree objects are way better than text for manipulating structured data. Python is great at this, so I use Mario to do it in the shell. https://github.com/python-mario/mario https://github.com/python-mario/mario For example, to sort filenames by length $ ls | mario apply 'sorted(x, key=len)' chain doc src bench extra AUTHORS LICENSE
- sciurus 7y agoDo they actually convert the output of the existing commands? Or are they reimplementing then one by one? It looks like the latter. In their examples it looks like 'ls' is built-in to their shell instead of from e.g. coreutils.
- sha666sum 7y agols is a shell builtin. When he types ^ls he uses the one from coreutils.
- JohnBooty 7y agoI love the core idea of nushell, but reimplementing them seems rather insane, given the incalcuable number of programmer-hours that have gone into coreutils. That seems like a real mistake, rather than simply having nushell's "ls" be a wrapper for coreutils "ls -la" or some such. I understand the benefit of reimplementing everything from scratch, as that way you have a more consistent nushell experience everywhere, regardless of which version of "native" ls your system has. And allowing "^ls" is a useful and necessary hedge. But, wow reimplementing nearly everything seems like an enormous undertaking. (It is of course possible that I'm completely misunderstanding things! Perhaps the devs can comment?)
- oblio 7y agoCoreutils has been reimplemented N times: BSD, GNU, BusyBox, the Rust one, countless others I'm forgetting. It's doable.
- lozenge 7y agoCheck the ls man page and remove anything related to sorting, filtering, or formatting information (as under this system all this functionality is provided in a shared and generic way). There is not much left.
- devwastaken 7y agoThis is pretty much what I've idealized for a new shell. Where every program can just output a standard table of information for each action. No need to have the program itself be queried for it every time, or find all the right options. You just get all the data, and filter what you want programaticly.
- vthriller 7y ago> You just get all the data, and filter what you want programaticly. There are definitely cases when you're better off with producer-side filtering. For example, `find` allows you to not traverse certain directories with -prune, which might save a lot of resources.
- jodrellblank 7y agoPowerShell has a lot of "corrections" of this kind, to get producer-side filtering. e.g. Getting all Windows Eventlogs, then filtering client side turned out to be quite bad, so Get-EventLog became Get-WinEvent with a lot more filtering controls. Cmdlets which get data, like Get-AdUser or Get-MessageTrackingLog in Exchange, or Get-CimInstance often have a lot of common filtering parameters as well as a more general -Filter or -Query "build your own". Big source of annoyance where you can't do that, but might want to.
- felixfbecker 7y agoIt also generally applies to concept of wildcards/globs not just to file paths, but many parameters. So a cmdlet like Get-User with a -Name parameter to get a user with a certain name can also be called with John* to get all users with first name John. In my experience the filtering options are pretty consistent
- geophile 7y agoI think they took one step in the right direction, where two more were needed. The UNIX idea of piping text output from one command to the next was fine at the time, but it means that the downstream command has to parse text. With nushell, you are parsing slightly more structured text. The two steps they could have taken in addition: 1. Instead of generating rows of a table, why not lists of strings? E.g. ("abc", "def", 123) instead of "abc | def | 123". Much easier to deal with in the command receiving this input. 2. And instead of just lists, why not objects? E.g. a process object, a file object, or, if you'd like a list as above. I have implemented this idea: https://github.com/geophile/osh https://github.com/geophile/osh. For example, if you want to find processes in state 'S' with more than 3 descendents, and print the pid and command line of each, sorted by pid: osh ps ^ select 'p: p.state == "S" and len(p.descendents) > 3' ^ f 'p: (p.pid, p.commandline)' ^ sort $ ^ is used to denote piping. ps yields Process objects. Select examines those processes and keeps those with the right characteristics. Then f applies a function, mapping to the the pid and commandline in a list, and then sort sorts the lists, by pid (the first field of each list). $ does output.
- deleted 7y ago[deleted]
- xz0r 7y agoThis idea is already preset in xonsh
- svd4anything 7y agoHow do you think it compares to xonsh? I ask because I had been thinking recently about investing some effort into xonsh and now nushell has appeared. I’d be very curious what any serious xonsh users think about this newcomer.
- astrobe_ 7y ago> So, imagine being able to use the same sorting/filtering logic on the output of `ls` as you might on the output of `ps`, and without relying on hacky solutions like pipelines to `grep`, `cut`, and `sort` That would awksome.
- dima55 7y agoThere're already numerous ways to do this without rewriting the world. For instance https://github.com/dkogan/vnlog#powershell-style-filtering-of-common-shell-commands https://github.com/dkogan/vnlog#powershell-style-filtering-o...