46 ms·
Introducing nushell
- thih9 7y agoI like the concept a lot, especially the focus on outputting data in an organised way. What I feel could be improved is the presentation of the tabular data. Some ideas below: - The screenshots show two horizontal lines consisting of dashes, one at the start of the table, one at the end. To me they seem redundant and could be replaced with spaces (whitespace). I did a quick check and PostgreSQL's psql tables also don't have lines like this. - How about using Unicode lines instead of dashes and plus sign in the header? This way there would be fewer gaps there. I'm not sure though how this would affect compatibility.
- mkl 7y agoYes, I think Unicode lines would be much nicer. "+", "-", and "|" are used for many other purposes.
- nwah1 7y agoIdeally, presentation of the data would be handled by themes. Hopefully they build theming support.
- kzrdude 7y agoI'd immediately say it would be nice with configuration that could opt-out of the relative and fuzzy dates. Relative like "today" would be fine, if it includes the exact time today too.
- tux1968 7y agoYes, this is needed for sure. Some output didn't appear on my terminal because it is hard coded to appear as black. The author must have a light colored terminal that differs from mine.
- thesuperbigfrog 7y agoNu looks very promising. Are there any prebuilt binaries available?
- jntrnr1 7y agoNot yet, but we'll try to post some soon.
- guylepage3 7y agoLately I've been paying more attention to alternative CLI tooling, extensions, etc. Our open source community uses git-flow-avh for example and it has been extremely helpful. I can't wait to try this out.
- rabidrat 7y agoThis is great! It's like VisiData as a shell. I wonder how easy it would be to use the VisiData interface with the shell output?
- Legogris 7y agoInteresting! I am also looking forward to setting aside time to play around with https://elv.sh/ https://elv.sh/, which takes a completely different approach with a similar philosophy.
- xiaq 7y agoElvish author here, thanks for the plug :)
- felixfbecker 7y agoThis looks great, but what exactly are the benefits over PowerShell? The article mentions PowerShell, then says: > What if we could take the ideas of a structured shell and make it more functional? What if it worked on Windows, Linux, and macOS? What if it had great error messages? - Any shell with piping is already very functional, and PowerShell has script blocks (higher-order functions). Or does this mean "functional" in the literal sense? What functionality is missing? - PowerShell works on macOS, Linux and Windows - The examples of error messages further down look almost exactly like PowerShells error messages (underlining the command in the pipeline that failed) It is not clear to me what exactly the authors sought out to do different
- unixhero 7y agoFor one Powershell isn't libre as in freedom am I right.
- wvenable 7y agoI wanted to like PowerShell but I find the syntax choices absolutely exhausting. This looks far more comfortable to me even as someone who mostly uses Windows but also knows Unix shell.
- qwerty456127 7y ago
- lostmsu 7y agoI feel open parsing files is misguided. Downright wrong if it profilerates into scripts, because suddenly simple scripts can't work on JSON, because opening JSON returns something different from a stream of lines.
- joshstrange 7y agoWell I imagine it respect the shebang at the top of the script so it will be run in the right shell where it doesn't parse or do any of this fancy stuff.
- dragonwriter 7y ago> I feel open parsing files is misguided. Downright wrong if it profilerates into scripts, because suddenly simple scripts can't work on JSON, because opening JSON returns something different from a stream of lines. Doesn't “open --raw” address that? https://book.nushell.sh/en/loading_data#opening-in-raw-mode https://book.nushell.sh/en/loading_data#opening-in-raw-mode Also, I suspect most simple scripts that want to work on JSON Will want to parse it, so the default makes them more easily able to work with it, not less.
- lostmsu 7y agoI am having a problem with --raw not being default. Defaults must be secure and stable.
- mkl 7y agoCan you explain why parsing JSON can't be secure or stable?
- lostmsu 7y agoThe problem is not with JSON parsing. It is with the fact, that people will use open without --raw to process, for example, Python files, because it is easy and works now. And some day later shell will learn to parse them into AST with open, and today's scripts will no longer work. Or worse start doing something very wrong.
- dom96 7y agoWhoa. It has been a while since I've seen something that truly feels like it can revolutionise my workflow. Amazing job to everyone involved in this project! Really excited to try this, surprised that there is no pre-built binaries. Are these planned?
- jntrnr1 7y agoYes, we'll hopefully have those up soon.
- pixelmonkey 7y agoThe compelling idea here is that they convert the output of common shell commands into tabular data that can be manipulated using common operators, so that you don't have the remember sorting/filtering/grouping flags that may be different for every different shell command. So, imagine being able to use the same sorting/filtering logic on the output of `ls` as you might on the output of `ps`, and without relying on hacky solutions like pipelines to `grep`, `cut`, and `sort`. It also means shell command output can be easily transposed into JSON and CSV. Actually pretty clever!
- entangledqubit 7y agoI've been delving into bash scripting a bit more than I'd like as of late and the lack of universally available consistent structured output for the CLI really got to me. Most of the script contents end up being these obfuscating and brittle "hacky solutions" that never should have been necessary. When I thought about pursuing fixing this I felt like the task was a bit overwhelming. I'm delighted that these developers are working on this!
- kbenson 7y agoI've generally found you can get most of what you need done with head, tail, grep, cut, paste, sed and tr (and maybe a couple others I'm missing). Oh, and test (both bash's and coreutil's). It can be a little hacky, but once you realize the crazy output you sometimes have to parse from third party programs, you realize that the ability to handle any free form text is pretty essential. That said, since I'm a Perl developer, I'll generally just go straight to perl's inlined implicit loop mode (-p or -n) if I need to use more than a couple of those utils, unless I want it to be portable with a minimum of requirements (as even though more binaries are used in the shell version, they're all essentially guaranteed to exist on a Linux/Unix).
- munchbunny 7y agoYou can also just use PowerShell. Its chief crime is its verbosity in naming, but it has the "universally available consistent structured output for the CLI" you're asking for.
- nixpulvis 7y agoI too have been off and on writing a Rust shell, though I want it to support `chsh` POSIX compatibility for my Arch setup. It's no where near complete though. https://github.com/nixpulvis/oursh https://github.com/nixpulvis/oursh `nu` seems very cool (and I love the name ;), but isn't building atm.
- rvz 7y agoA brand new shell, especially in Rust is a great thing to see and also how it tackles the problems that exist in other shells. However, its interesting to see so many dependencies that are required to build it (73 crates last time I checked) and as shells have always been portable across many operating systems, I wonder how portable nushell would be since it wishes to replace my current shell.
- orf 7y agoThere are 464 dependencies in the complete tree when doing `cargo install`. That's a bit too many. But it's not often you compile your shell from scratch.
- jntrnr1 7y agoIt's definite a lot. We're looking into making more of the features optional, so you can streamline it if you want.
- orf 7y agohttps://github.com/nushell/nushell/pull/341 https://github.com/nushell/nushell/pull/341 now there are only 458 :)
- leetrout 7y agoThat is some impressive insight to the behavior of the lib. Have you ran into that before and knew it was an easy thing to get rid of? I’m always impressed when people show up with fixes like this (that I wouldn’t even know to look for).
- sams99 7y agoI did see quite a few that double up, like 2 versions of url and a few others. Just had a quick play ... on arch linux `yay nushell-git` already works!!! One area where I got a bit stuck was around help. `man where` gave me nothing, `help where` also. I tries stuff like `ls | where type = File` and got a type error. I think it would be amazing if this thing onboarded people a bit nicer, "where needs a condition, to learn about where type "help where" ... stuff like that Overall really enjoying the ideas here and I am absolutely going to be following this!
- rajangdavis 7y agoThis seems like it could potentially be an awesome tool for data cleaning/munging in the terminal. Really interested to see where this goes..
- orf 7y agoLooks amazing! Unfortunately a `cargo install` on stable fails due to a dependency "futures-async-stream v0.1.0-alpha.4" with the "rawkey,clipboard" features. The source is pretty interesting. Here's the PS command: https://github.com/nushell/nushell/blob/master/src/commands/ps.rs#L27 https://github.com/nushell/nushell/blob/master/src/commands/... And the sort-by: https://github.com/nushell/nushell/blob/master/src/commands/sort_by.rs#L29 https://github.com/nushell/nushell/blob/master/src/commands/...
- digitalinfinity 7y agoYup- the doc at https://book.nushell.sh/en/installation#installing-rust https://book.nushell.sh/en/installation#installing-rust does call out that it currently needs the rust nightly toolchain.
- tus88 7y agoJob management?
- jntrnr1 7y agoNot yet, though we're talking about a few possible approaches.
- OldHand2018 7y agoHow about a test framework? PowerShell has a unit test framework called Pester which is installed by default in Windows 10 and Server 2012+. Anything you can script in PowerShell can have unit tests and code coverage and it integrates into CI systems. I don't really like using PowerShell, but built-in unit testing is something I wish other shells offered. It is a huge plus.
- pvg 7y agoWonder if the temptation wasn't there to just call it 'nutshell' and if so, how it was resisted.
- maest 7y agoI love the idea of treating output as tabular data. I see you already have a where operator. What would fit this very well would be in-built sql-like syntax. Since tables are first class, you should have an in-built way of handling them. A languiage similar to q/kdb - it has a great SQL dsl, but also works as a regular programming language. It's also very concise and expressive, you can write oneliners that do mind bogglingly complex things.
- smashthepants 7y agoWhat's a dsl? I wasn't able to find an answer in my quick google search
- dustfinger 7y agoThis project has goals similar to GNU eshell [1]. Great ideas, thank you for sharing. [1] https://www.gnu.org/software/emacs/manual/html_mono/eshell.html#Introduction https://www.gnu.org/software/emacs/manual/html_mono/eshell.h...
- djsumdog 7y agoI really like this conceptually. I feel like with Powershell, you need a bunch of commands that actually pass objects around or act upon them; and something similar in Linux would require writing a lot of extensions first. I've been using fish shell for years and glad I started down that route. It's been incredibly useful. I really want to give this a shot too. I feel like too often I'll open up Python, read a file and then manipulate it when I need to something more complex than simple grep/sed/cut/wc etc... This feels like it really fills in that gap. It starts with something really simple: get your unstructured output in a structure, and then build from it. I could see people writing more custom extensions and objects later on if this gets popular.
- electricEmu 7y agoPowerShell works on Linux today. It's consistent and works fine for this Go-developer's needs. It's spectacular to have a shell with consistent verbiage and parameters. https://aka.ms/getps6-linux https://aka.ms/getps6-linux
- lukeschlather 7y agoThe big problem with Powershell, and it makes it borderline unusable on Linux, is its crazy error handling. The equivalent of bash's `set -e` is `$ErrorActionPreference="Stop"`. The problem is that it treats any output to stderr as an error and ignores actual posix errors (non-zero exit codes.) What's worse is in some contexts the error it throws on stderr is impossible to catch. I really love Powershell in a lot of ways but this makes a lot of really basic scripts in bash very cumbersome to write in Powershell.
- geekone 7y agoDo you have any examples of stderr errors being impossible to catch in PS, and/or a basic script that is very easy in Bash but cumbersome in PowerShell?
- lukeschlather 7y ago
- techntoke 7y agoIn the example you have, `ps` takes way too long.
- boobePhuu7iet7i 7y agoLooks stupid
- dang 7y agoWould you mind reviewing the site guidelines and sticking to the intended spirit when posting here? We'd be grateful. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- weyman 7y agoYes! I’ve always preferred PowerShell for scripting, Bash for an interactive shell. This seems like the best of both worlds, can’t wait to try it out.
- chrissoundz 7y agoI really enjoy seeing new ideas around interfaces. But I do wonder, rather than creating a new shell, couldn't we have a 'json/structured' mode for cli programs - and maintain compatibility with existing shells? For example if it was toggled with a SHELL_FORMAT environment variable? What issues could be faced with this approach?
- munchbunny 7y agoPowerShell has an IMO good answer to this: if you run a classic CLI program, its output is interpreted as an array of strings (lines), so if you pipe the output, you have a way to process it into something object-like. Having a toggle creates the problem that you have to write two implementations for your tools, or you end up having to memorize which commands can run in which modes. It's easier to have a consistent pattern and a way to up-convert existing CLI programs.
- joshspankit 7y agoOh lordy where to begin.
- joshspankit 7y agoAhh yes. Here’s where to begin: https://xkcd.com/927/ https://xkcd.com/927/
- j88439h84 7y agoMario uses Python objects in pipelines instead of plain text. https://github.com/python-mario/mario https://github.com/python-mario/mario
- qwerty456127 7y agoI'we always felt like there should be no file system and no shell but a relational database with a particularly suitable and extensible SQL dialect REPL.
- joshspankit 7y agoI’m digging this, and up you a graph database, But how would you handle “small-file” cruft? Still have optional “folder” rows?
- qwerty456127 7y agoI can't really tell how do we apply make this idea compatible with the common workflow/model of computing. In my imaginary world everything down to the level of a statement of what is a code file in the real world is a separate record in a relational database, there are no file names and no folders (only tags, commentaries and other metadata). In the real world folder hierarchy is probably inevitable so yes, it has to be stored in a way (in a separate table perhaps so we can have many-to-many relationships sort of what hardlinks do).
- joshspankit 7y agoI’m still with you on the complete reimagining of how a computer stores and accesses ‘code files’, but there’s still a challenge with ‘small files’, which is to say the hundreds or thousands of chunks of code that can have dozens of uses (Plex’s metadata cache, for example, or the hundreds of shared libraries that allow larger more complex programs to be built small by referencing them). One benefit of folder hierarchies (and I don’t think there are many) is that it’s easy to delete/move hundreds of thousands of tiny bits of associated data just by deleting/moving the parent.
- qwerty456127 7y agoI mean using tags instead of folders. You can make a tag that may mean anything and select all files with a particular tag.
- j88439h84 7y agoMario has a similar feel to this but in Python, and lets you use arbitrary code and construct your own functions in pure Python. https://github.com/python-mario/mario https://github.com/python-mario/mario $ mario read-csv-dicts map 'x["name"]' < hackers.csv Alice Bob Carol
- t0astbread 7y agoI like this a lot but why does this have to be a completely new shell? Couldn't this also just be a set of tools that I can use in an existing shell and get the best of both worlds?
- fucking_tragedy 7y agoThis was my take away, as well. Bourne and Bash are everywhere and zsh will be included in macOS updates. I definitely see value in a separate set of text processing tools.
- j88439h84 7y agoThe idea of Mario is to use Python objects in your native shell (bash/zsh/fish/etc) https://github.com/python-mario/mario https://github.com/python-mario/mario
- scrollaway 7y agoXonsh seems like a more popular, maintained and viable approach to that: https://github.com/xonsh/xonsh/ https://github.com/xonsh/xonsh/
- j88439h84 7y agoXonsh is a whole shell environment though, which isn't what parent wanted. As for popularity, yes, Mario is relatively new. :-)
- t0astbread 7y agoThe GitHub repo looks kinda weird on first glance (>1k commits but <100 stars, virtually no contributors or issues, empty LICENSE file) but yeah it is more of what I thought. Personally, I'm not very accustomed to Python though, so I find the syntax kinda weird. I'd prefer something that feels more like a standard Un*x tool (like nushell).
- Skunkleton 7y ago
- lemoncucumber 7y agoCould someone explain this to me in a nushell?
- gitgud 7y agoInteresting, there's definitely a niche for this. But I'm not sure it could replace a normal shell in my opinion. I'm confused at what other commands would do in this context? Will it tabularize; echo "Hello World"? There's an elegant simplicity of treating all piped output as a string rather than objects...
- 01100011 7y agoLooks cool, but when I see things like this I figure there's no point in trying to learn something that won't exist on 99% of the systems I need to work on. Bash sucks, but it's almost always there. Embedded systems, desktops, big iron... Like many of the established Unix tools, bash has market penetration and staying power.
- canadaduane 7y agoYou might be in the "late majority" phase of tech adoption, and that's ok. Some of us early adopters bang our heads on new stuff and pass the results of our tests on to slightly less foolhardy souls :) Eventually, established tools arise out of the mess. https://ondigitalmarketing.com/learn/odm/foundations/5-customer-segments-technology-adoption/ https://ondigitalmarketing.com/learn/odm/foundations/5-custo...
- amelius 7y agoDo shells exist which don't have a fixed maximum argument size, so you'd never need "xargs"? Or is the problem innate to Unix?
- codetrotter 7y ago> ARG_MAX, maximum length of arguments for a new process > or why do you get > command: arg list too long > [...] > It's only the exec() system call and its direct variants, which will yield this error. They return the corresponding error condition E2BIG (<sys/errno.h>). The shell is not to blame, it just delivers this error to you. https://www.in-ulm.de/~mascheck/various/argmax/ https://www.in-ulm.de/~mascheck/various/argmax/ Additionally, see also https://unix.stackexchange.com/a/120842 https://unix.stackexchange.com/a/120842 which speaks a bit about Linux specifics, and how Linux behaves a bit differently from a lot of the other operating systems in the Unix family with regards to limits on the length of arguments.
- felixfbecker 7y agoafaik PowerShell has no limit on the amount of arguments
- mstratman 7y agoAny time the first line of a project description reads "This is NewTool, written in <language>" it's an immediate red flag that makes it hard for me to continue reading. Obviously it doesn't necessarily mean the project has no merits, but language choice is one of the less relevant details and it signals that the author wanted to make a toy in a language new to them. Good for them, but most of the time the rest of the world doesn't care. If the project has merits, and this one might, it should talk about them in the intro.
- crispinb 7y ago> most of the time the rest of the world doesn't care Much of the time, HN readers however will. There's the odd software developer hanging about here, some of whom are interested in programming languages, or so I believe.
- perspective1 7y agoRust isn't new to Turner (the author), he was one the Rust Language team at Mozilla.
- SparkyMcUnicorn 7y agoAgree mostly, but there are certain languages that I give it a pass for given the context. A good shell written in Rust is intriguing. I don't think I want the fad to go away, because it can even help me know what to steer clear of. e.g. Introducing a new shell written in JS!).
- deleted 7y ago[deleted]
- zeveb 7y agoCompletely agreed. Also, this line in the article was another red flag: > Nu would not have been possible without Rust. That's nonsense! Nu would be possible in any Turing-complete language. Other languages support async/await, asynchronous streams & serialisation/deserialisation. Go leaps to mind, but — again — this sort of thing is possible in pretty much any programming language. Now, maybe it was easier or quicker in Rust than it would have been in another language. But it certainly would have been possible. Combine those two red flags together, and I get the sense that it's just an ethusiasts' project. Nothing wrong with that, but I don't think it's something I'll be converting my company's shell scripts right away …
- frankus 7y agoThis seems to address a lot of the shortcomings presented here: https://mkremins.github.io/blog/unix-not-acceptable-unix/ https://mkremins.github.io/blog/unix-not-acceptable-unix/
- nikisweeting 7y agoHmm I like this but there's no way it's replacing fish for me, I'd rather use it as standalone command that provides the same functionality. e.g. `ls | nu where size > 10` or `nu open Cargo.toml`
- IggleSniggle 7y agoI finally decided to go look at fish. Why didn’t I do this sooner??
- aerovistae 7y agoFeature suggestion: detect timing differences between program outputs to determine where an individual output ended and the next one began, so people can automatically scroll to the top of a stack trace for instance by double clicking it or something because the shell will recognize all those lines as a single message.
- ilaksh 7y agoThis looks great. Is the autocomplete as good as fish? If so I may switch from fish to nu. The autocomplete matters a lot. With fish I often repeat commands during development and I only have to type the first letter or two.
- felixfbecker 7y agoI find PowerShell autocompletion amazing. It has Ctrl+Space autocompletion similar to an IDE where you can select options with up and down arrow. Also ctrl+r for history search like fish (not always active like fish unfortunately)
- ulugh 7y agoCool project! I like the intuitive syntax and clean doc. Will give it a try.
- bigtrakzapzap 7y agoWow. Impressive usability OOTB. Great job!
- massung 7y agoPretty cool idea and project. Hope you take it further and it turns into something truly spectacular! My only comment is on the article. This line: > Nu would not have been possible without Rust. That's just patently false. Perhaps you wouldn't have chosen to code it in another language. Or perhaps Rust had one or two benefits that assisted in the development of Nu. But you don't word it that way or go into any details as to what made Rust special/better than X, Y or Z language (not that you need to, but if you're going to make a wild statement, you should be prepared to back it up). The only 3 points made following that were about features that have widely existed in other languages well before Rust came along: async/await, async streams, and serde. Rust is cool and is still exploring some really great ideas in making programs safer and more maintainable. But, it's not the greatest language ever (at least not yet), nor does it make the impossible possible. It can merely make some things easier at the cost of something else. I'm glad you were able to use it effectively to create a tool you - and others - find useful.
- moogly 7y agoYeah I agree. That was intellectually dishonest. I really like the idea and look of this project. To me it feels like PowerShell done better (and implemented by non-aliens).
- mongol 7y agoMaybe it should be read as "my child not be possible without me".
- lokedhs 7y agoI was about to post the same thing. I could put this together in Common Lisp. I know precisely what CL technologies I'd use to do it, and perhaps after finishing the work I'd say "this wouldn't have been possible without Lisp", which would be true in some respect, but just as dishonest as the original article. That doesn't mean I don't respect the work done. It's an interesting project and most definitely something that deserves attention. The language used to develop it is the least interesting aspect of this.
- nailer 7y agoAs a long time Unix person who also discovered pwsh, like Yehuda did in the article: Can I make nushell cmdlets (or equivalent) in something other than rust? pwsh limits me to .net languages, but I wanna use node.
- zellyn 7y agoHow do I do this in nushell? It's an alias I created recently to ripgrep through java and go files and tell me which have both of two words: function both { comm -1 -2 <(rg -i -l -t java -t go $1 | sort) <(rg -i -l -t java -t go $2 | sort) } Example usage: `both sharded jobqueue`
- tux1968 7y agoYour question brings to light some of the deep flexibility current shells allow with a very terse syntax. At least from Nu you can accept and process the output of your script. As an aside, your one liner is having to process every file twice, even ones that have already been excluded by the other ripgrep... something like this should be more performant: rg -il $2 $(rg -il -t java -t go $1)
- zellyn 7y agoNice. Perhaps with an xargs thrown in to be safe...
- tux1968 7y agoAfter playing with Nushell a bit, I think that you would do something like this: rg -0 -il -t java -t go sharded | split-row "\0" | rg -il jobqueue $it Instructing "rg" to use Nul as a separator and where "split-row" acts like xargs. (This only worked after I modified Nushell to recognize "\0" as the Nul separator. It's surprising that this case hasn't come up many times previously, so maybe i just missed the proper way to do it). Edit: Of course, not many filenames contain the newline character embedded in them, so usually this would be fine in an unmodified Nushell: rg -il -t java -t go sharded | lines | rg -il jobqueue $it
- perspective1 7y agoVery neat, love it. Few things-- 1) Recommend this install line (for those of us on mainline/standard who've never used a different toolchain) cargo +nightly install nu 2) Things like ls | where readonly don't work as expected-- readonly should be a bool, shouldn't it? 3) How do you paginate? ls | less doesn't work
- mamcx 7y agoThis is almost exactly what I have dreamed with a shell. I'm building a relational language (http://tablam.org http://tablam.org) with the plan of something alike. Some extra ideas: - Tables are amazing. But I think we need at least support for this "shapes" of data: List, Tables, Trees and maybe graphs - I think will be nice to have an equivalent of Request/responses alike http. And then a way to introspect what a command do, like auto-swagger. - Having widgets like table-browser, graphs, etc AS COMPONENTS. In FoxPro you could say "browse" and it display a DataTable grid with full editing capabilities on the current table (or it ask for a table to open). In this case: ls | ui_browse The key here is that the widgets work stand-alone, so is not necesary to fully build a terminal app just to see nicely data. Look like https://goaccess.io https://goaccess.io and more like this idea: https://github.com/sqshq/sampler https://github.com/sqshq/sampler This mean we could get a jupyter-like experience here! - Some way to do curryng and maybe environment alias: You could setup an environment for the system, the user, this directory. In the environment is a .toml with vars and stuff. The other thing where I think shells fail... is that are still old-style terminal. Having text output is cool, but having more rich widget system will unlock better the potential
- felixfbecker 7y agoLike PowerShell's different output formats (table, list, wide, ...)?
- mamcx 7y agoNo output. Declared the input/output is in the shape table, list, tree. Like when a http response say is json, text, csv, etc. This information unlock certain capabilities and will allow to provide efficient execution of queries. If psql (postgresql terminal cli) declare their data is in the "tables" shape then it could take the |where name = "" query itself and execute it instead of output all the rows...
- Per_Bothner 7y ago"The other thing where I think shells fail... is that are still old-style terminal. Having text output is cool, but having more rich widget system will unlock better the potential" You might find DomTerm (http://domterm.org http://domterm.org) interesting. It combines solid traditional-terminal support (including solid xterm compatibility) with a data model based on HTML/DOM. So you can "print" images, HTML tables, and more. DomTerm "understands" shell/repl commands as consisting of prompts, inputs, and outputs. You can add markers to group the output into logical groups, so it is automatically re-formatted on window re-size. And more.
- djabatt 7y agoI just like seeing the fresh hacking in the Unix Shell space. I miss my old friends sed, awk, and xargs. This is compelling and fun too.
- entire-name 7y agoAn object pipeline is certainly interesting, and I do use PowerShell from time to time. However, there is a reason extremely simple concepts such as "everything is a file" or "input/output/error are all text" survived for this long. I will quote an answer provided in a vaguely related question [0]: > There is no spoon^H^H^Hstructured data. There is only text. [0]: https://unix.stackexchange.com/a/400 https://unix.stackexchange.com/a/400
- hestefisk 7y agoI like this. Very neat with tables; easier to memorise than PowerShells very verbose command names. It feels like a pseudo SQL interface to the OS. Only area where I think it oversteps the Unix philosophy is the integrated terminal multiplexer. I think the shell should be terminal agnostic.
- cwmoore 7y agoWithin the boundaries of the demo, it feels like all my time reading man pages is a landline network, and this is cellular.
- m3at 7y agoThis is great, we have a tendency to be content with existing tools we're familiar with but be blind to their limitations. I'm looking forward to how this project evolve! However an important point for practical usage is the capability to run existing scripts, for example as oilshell [1] does. I skimmed through the README and doc but couldn't find the current status nor future plans. [1] http://www.oilshell.org/blog/2019/06/17.html http://www.oilshell.org/blog/2019/06/17.html
- fouc 7y agoThis nicely ties into the recent HN thread about a dream OS that would provide databases at the OS level instead of forcing the re-implementation of databases in many programs. https://news.ycombinator.com/item?id=20754592 https://news.ycombinator.com/item?id=20754592
- jonahss 7y agoLove it! Thanks. I've always thought powershell was a great concept, badly executed (those long names with capital letters, confusing errors, and insanely long time to return when a command is missing). Is anyone interested in a shell with a GUI? As in, data still passing as text, but having a shell which can render textual data into widgets when displayed in the shell?
- felixfbecker 7y agoPowerShell has this with out-gridview, check out: https://devblogs.microsoft.com/powershell/out-gridview-returns/ https://devblogs.microsoft.com/powershell/out-gridview-retur... My favourite feature is that you can select objects in the GUI to pass on to the next pipeline step
- sigmonsays 7y agoi feel like structured data in a shell will phase out. The posix pipes allow streaming any data. So if programs want to stream structured data, just build programs who know how to communicate with each other using structured data. The older shells withstand the test of time with generic interfaces. For instance, because of the influx of JSON the past few years I have learned to use jq very well. It does everything I need and I didn't need to upgrade my shell to use it. What if everything becomes protobuf down the road?
- JohnBooty 7y agoi feel like structured data in a shell will phase out. The posix pipes allow streaming any data. There's a huge mismatch in coreutils. Nearly everything is built around a line-based philosophy. However, nearly any coreutil you care to name also likes to cram multiple bits of information on a single line. That means you often have to do a bunch of ugly manipulation with awk, sed, cut, whatever to anything programmatic with the output of each command. For instance, because of the influx of JSON the past few years I have learned to use jq very well. It does everything I need Yeah, jq is awesome and covers a lot of needs. But coreutils don't really emit json, so... yeah. There's a lot that it can't work with, right? What if everything becomes protobuf down the road? I know you're being facetious, I think protobuf is clearly overkill for very many needs. Nushell seems to build upon what we've learned in the last 30-40 years. Pipes and simple interchange formats work well and are very useful for a lot of use cases. However, experience also shows that very few programs output lines of text that are atomic in nature. Each line represents multiple things and we frequently need to parse them out. That seems like a very common use case that our shell can handle for us.
- gigatexal 7y agoThis looks really cool but isn’t Powershell cross platform now? It looks to shadow much of the features of Powershell which opens scripting to the whole of the .Net library
- juststeve 7y agoThe assumption with powershell is that your admin tasks should be coded in .net, and their dependencies should be imported from .net. Not required, but preferred.
- gigatexal 7y agoYeah and since .Net isn’t my cup of tea this shell could be pretty cool
- juststeve 7y agoyes hopefully this project can implement a double pipe operator or something, so one pipe means pipe structured data (like powershell) and the other is pipe stdin/stdout output via memory so it is super fast.
- abcdefg123456 7y agoGo ahead and downvote me, but use logic instead of emotions. This is the difference between junior and senior engineer. "Working hard" can actually work against you. Anyone notice how poor design/interface creates jobs? Once you learn the poor design, then you become one of them. Lawyers, accountants, mechanics, linux engineers, most professions are scams. Once you learn it, you're better off making money in the profession than actually simplifying it and letting the public know the profession is a sham. It's a racket, except no one can be blamed because someone else made the terrible rules/system. "Popular" technologies (on Indeed.com, resume driven development) are not popular because they are good, they are popular because they are bad. If you want to make money, beware of making things too simple... using awful and complex technologies (Kubernetes - this is way too complex, Jenkins - a GUI wrapper around the linux cli?, Docker - linux admins discover a way to stay employed, AWS - we converted some simple apps to a GUI and charge millions, Chef - creating an immutable image is too easy, Salesforce - let's lock you in and use our own language, Splunk - elasticsearch is too cheap let's charge millions, etc.) is a good business model. It's like any industry... actually solving people's problems is a bad business model. Strippers make more money than prostitutes.
- Gehinnn 7y agoIs the processed data typed? Can you offer auto completion of data fields based on these types? I also thought quite a couple of times about implementing a data centric, typed shell language. My primary goal was to have a shell where one could discover every feature by oneself through the magic of types.
- burlesona 7y agoI read this, and read all about it, and the whole time I thought the name was “nutshell,” until I saw they abbreviated it “nu.” Now I’m really disappointed that it’s not called “nutshell.” But this is still very cool.
- MeteorMarc 7y agoNushell in a nutshell would have been the better title!
- doodpants 7y agoI sincerely hope that nushell becomes popular enough for O'Reilly to publish a book with that title. :-)
- bradknowles 7y agoIt looks to me like they took the concept of “everything as sql” to the extreme. Do you want to live your entire life and everything you do in sql? I don’t. I think I will pass.
- savolai 7y agoThe idea is inspiring! I'm finding it hard to get over them using pipes and dashes for table borders. Is this the 70's? It's genuinely hard to read these tables. See: https://www.interaction-design.org/literature/topics/gestalt-principles https://www.interaction-design.org/literature/topics/gestalt...
- SneakyPit 7y agoIn case you are interested in more sophisticated ls or find, try fselect ( https://github.com/jhspetersson/fselect https://github.com/jhspetersson/fselect ). That tool supports JSON as output as well. Fselect has few really cool features, esp. when you got to deal with complex queries. Although not unix way at all.
- kerng 7y agoThis is very cool! The giant drawback of Poweshell is that it's so focused on Windows - this projects seems to provide some of that power cross platform. Very neat idea!
- felixfbecker 7y agoPowerShell has been focused on cross platform since 2016 (3 years now)
- grandinj 7y agohow long before it starts implementing SQL over the input data? They are already tables after all :-)
- 21stio 7y agobrings osquery.io to my mind
- chubot 7y agoYeah for http://oilshell.org/ http://oilshell.org/ I am planning to use the model of "data frames", which I compare to SQL here: What Is a Data Frame? (In Python, R, and SQL) http://www.oilshell.org/blog/2018/11/30.html http://www.oilshell.org/blog/2018/11/30.html I think starting with either SQL or data frames would be a good choice. They are similar except data frames also focus more on mathematical modeling, and data frames usually don't have indices. Otherwise they both have selection/projection, sorting, various kinds of joining, etc. Data frames also have vectorized arithmetic. I am trying to reuse the designs that have proven to work over time (R, SQL, bash) rather than inventing new variants with potentially needless differences.
- farx 7y agoWhat a missed opportunity not naming the post "nushell in a nutshell"
- quickthrower2 7y agoThe next step this begs for is for the shell to be like lisp. The results are s expressions and you can write macros to slice dice and pipeline the data to your hearts content.
- Sorrop 7y agoYou may find closh interesting then https://github.com/dundalek/closh https://github.com/dundalek/closh Previous HN entry about it: https://news.ycombinator.com/item?id=15600928 https://news.ycombinator.com/item?id=15600928
- maest 7y agoPeople are really excited about this. Here's a question: Would you pay for this? My prior is that no, people would not pay for a new shell - they'd prefer sticking to bash because A. it's "standard" B. it's free. I have to wonder how developing new tolls like nushell can be sustained over the long run.
- roryrjb 7y agoThere's a lot of comments here already so I don't know if this has been addressed already, but I don't get this shell. It seems to be fixing a problem that doesn't exist, and a lot of comments are echoing it. Current shells are not deficient, as Doug McIlroy originally stated, the universal interface is text and tools should take input and generate output based on this. You obviously can then pipe things together with these tools not necessarily knowing in advance how they will be used in a pipeline. You can get very far with sed, awk, etc, you just have to spend a little bit of time learning them, but they are not complicated. You can of course then introduce a command like jq for specialised purposes. But hte point is the shell isn't providing this, they are separate tools that you orchestrate using the shell. There is not a flaw with how it works at the moment, I truly believe and this isn't a dig at this project or anyone in particular, that people do not spend enough time learning these tools that have been there for decades. The re-inventions that appear on GitHub are not a product of the deficiency of the existing tools, I don't know what they are but it's not that. Unless I am missing something.
- oblio 7y agoYou are missing something. Current shells are brittle and they're just bad tech in 2019. We've made them half decent by brute forcing tools and hacks for them. Everyone has to reinvent the same basic tool output parsing because ls (or top or ps or...) can't spit out real structured data.
- roryrjb 7y agoI disagree. I'm not saying that everyone should think the way I do, I guess realistically all I mean is that when I'm confronted with a parsing issue in this context I just write a shell script. Ok, this can be taken as reinventing, which I'm advocating against, but I see this as utilising the UNIX philosophy and by sticking to POSIX shell specifically (and using sed, awk, tr, et al) it is still a built-in and portable way to deal with this problem. Also I am not saying this because of some irrational submission to an old philosophy, I advocate this style of working because I find it easy, convenient and fast, and it works for me in the real world. But of course people have different ways of working and I respect that. This is just what works for me and how I see things. Definitely not meant as an attack against this project or different ways of working.
- toyg 7y agoThe Powershell idea to push structured data through the pipe, which this project fundamentally replicates with a relational structure, has its merits; but it will always displease someone, and carries the burden of having to rewrite millions of utilities in order to unlock the real potential. What if, instead, we pushed the approach “upstream”, asking systems to have an additional stream that all shells can access? We have stdout and stderr, we could add “stddata” - a pipe that expects and produces structured data. Then it would become trivial to add support around the userland, without displeasing anyone - because it’s just an additional interface, not a replacement. The pipe should support two or three formats to start with (csv, json, and maybe xml, so you can handle both tabular and child-parent relations with a varying degree of precision) and shells could have a special character for piping stddata (I like ¥) so everything else about them would stay the same.
- toxik 7y agoThe problem with adding a new standard fd is that lots and lots of code relies on first three fds being stdin out err. It could of course have a high fd number. Would be a great thing!
- dragonsh 7y agoI am not sure how long "ls" will take with 1 million entries in directory, given this paradigm. This has been a problem so I use "ls -1" to list files in bash shell, otherwise it takes forever. Probably it might benefit some. Good to see competition in this area with sh, bash, zsh, fish, conch and many others.
- toyg 7y agoA csv output with one field is basically the same amount of data, but you would be guaranteed that the result can be ingested by a csv parser following the “stddata” dialect. It wouldn’t hurt your usecase, and it would make life easier for other ones.
- StreamBright 7y agoI am at the stage that I am more than happy to rewrite whatever it takes to get better experience. This includes the OS. I think we have sacrificed too much on the altar of backward compatibility. I guess I am just tired of breakage because a file has a space in its name.
- GordonS 7y agoI like how powerful Powershell is, but I absolutely loathe it's verbose syntax and Pascal-Kebab-Case naming (at least for idiomatic Powershell). This seems to have a lot of the power of Powershell, but with a terse, familiar syntax - I really like the look of this! Being cross-platfotm is a huge seller for me too - I mainly work on Windows, but also on Linux and MacOS.
- dgudkov 7y ago> Everything is data We use a similar approach in our data prep tool [1] for non-technical users. Lists of files, lists of emails, contents of text files, spreadsheets or database tables - piping it all together allows mixing ETL and process automation in a similar fashion. [1] https://easymorph.com https://easymorph.com Disclaimer: I'm the founder.
- arendtio 7y agoI pretty much like the idea, but my biggest problem with current day shells is error handling. Errexit (aka `set -e`) is a great idea badly executed. There are many things which can be improved (e.g. printing a stack trace by default), but at its core, there are two things which break it. 1. Inconsistent behavior: https://www.in-ulm.de/~mascheck/various/set-e/ https://www.in-ulm.de/~mascheck/various/set-e/ 2. You can't rely on it. Inconsistent behavior should be no problem if a new cross-platform implementation is coming, but not being able to rely on it sucks. To give a quick demo: #!/bin/bash foo() { set -e false echo "Sucks" } printf 'Normal: %s\n' "$(foo)" printf 'Condition: %s\n' "$(foo || true)" Output: Normal: Condition: Sucks So if someone is about to bring a new shell, please fix the error handling along the way. But hey, Yehuda Katz is part of the project. So I am looking forward to what is coming next :-)
- sigotirandolas 7y agoAfter running into this and many other issues in sh/bash scripts (such as the pitfalls of handling newlines and spaces correctly with the splitting rules, the awkwardness of basic features such as lists or arithmetic, and the generally inconsistent and obscure syntax), I've moved all my shell scripts to Python3 and I have no regrets about it. Calling external programs is still very easy with the subprocess module. And it's installed pretty much everywhere you have sh/bash on. To be fair, I am not even sure if creating a new shell scripting language is something worthwhile in this day and age. There's some value in the convenience of instantly turning a sequence of commands you just manually typed into a script, but there's a conflict in that the shell has to provide you a way to hack a sequence of actions together with as few characters as possible, while a script has to be comprehensible, maintainable and extensible. And the inertia you would have to overcome to successfully introduce a new programming language in addition to a new shell makes it intractable.
- chubot 7y agoI have fixed this in https://oilshell.org/ https://oilshell.org/ : https://www.oilshell.org/release/0.7.pre3/doc/osh-manual.html#toc_3 https://www.oilshell.org/release/0.7.pre3/doc/osh-manual.htm... Well actually there's one more thing I want to fix: https://github.com/oilshell/oil/issues/179 https://github.com/oilshell/oil/issues/179 That will fix your case, because foo is a function, and the disabling of errexit in || is indeed problematic there. Please subscribe to the bug, or feel free to e-mail at andy@oilshell.org if you want to know when it's fixed. I have another person "on the line" (author of envdir I believe) who also wants this fixed. It would be great to have you both try this out and verify that errexit in Oil has no more pitfalls :)
- magoon 7y agoBash is ubiquitous - the best selling point for knowing it and sticking to it. I desire a fancy new shell but don’t want it to become a crutch, limiting my usefulness and familiarity when I’m on a system lacking it.
- Klasiaster 7y agoI find it good in general but don't like the idea of reusing the names like `ls` and so on which is confusing with the option to invoke the real `ls`. Since they have single-character names already, a simple `l` would be one way to solve this. I'm interested to see a solution to error handling (within multiple pipes!) and subprocess management as well but couldn't find much in the docs.
- tarruda 7y agoThe idea of programs that work on structured data is quite old. Anyone ever read "structural regular expressions" by Rob Pike? Highly recommended: http://doc.cat-v.org/bell_labs/structural_regexps/ http://doc.cat-v.org/bell_labs/structural_regexps/ Also, using structural regexps for efficient interactive text editing: https://github.com/martanne/vis https://github.com/martanne/vis
- antpls 7y agoInteresting PoC, but does it solve a real world problem? If I had to reinvent the shell by breaking backward compatibility, I would do a statistical study using all the shell scripts publicly available on GitHub, a survey about the critics of the current shells in the literature, and then I would try to optimize a metric such as : words typed to complete a task, reproductivity, time-to-market, communication, mathematical soundness, etc. The result of the study might be that the current shell is enough and the cost of switching to another is too big (I don't know, it's an hypothesis) People shared many thoughts and opinions on this thread, so that's already a win by itself
- ab5tract 7y agoWhat real word problem does bash solve over sh? Or tcsh over csh? Or ksh? Fish is the newest shell I’ve been using and it’s leaps and bounds better than dealing with posix syntax for the sake of unnecessary compatibility (if I need posix, I can run a posix shell without trouble). That said, fish is explicit about how the features it includes feel more like they should have evolved in th 90s, not the 2010s. Asking forgiveness is better than asking permission. Innovation rarely looks like a sure bet prior to it unfolding.
- coldtea 7y ago>Interesting PoC, but does it solve a real world problem? Yes, the real world problem of making it combining shell commands easier and more interactive. A solution doesn't need to solve a whole new problem. It's enough that it solves an existing, already solved problem, better. The same way the travel luggage with 4 wheels doesn't solve anything the travel luggage with 2 wheels or no wheels solves, just makes so much more easy and awesome.
- tgtweak 7y agoIt also opens up a compelling use case for having a nice web interface for working with the shell (where you can button sort and export to Json on output) since the structure is common.
- sriram_malhar 7y agoThis looks very interesting. Unfortunately, I'm having trouble building it on Mac OS X 10.14.5 and with the latest rust update installed. cargo:warning=libgit2/src/streams/stransport.c:12:10: fatal error: 'CoreFoundation/CoreFoundation.h' file not found cargo:warning=#include <CoreFoundation/CoreFoundation.h> Can anyone clue me in? An Xcode update perhaps? (I'm doing it, but it is taking forever). I have a CoreFoundation.h on my system under MacOSX10.14.sdk, but it is not under a CoreFoundation folder.
- coldtea 7y ago^ls -la | lines | skip 1 | split-column " " perms files group user size month day time name This is neat, but does nushell have the ability (should be trivial to add) to save such a pipeline and reuse it (similar to an "alias")? e.g. myls = ^ls -la | lines | skip 1 | split-column " " perms files group user size month day time name and then: myls | where size > 1000 Also perhaps the built-in commands could be namespaced? E.g. nu.ls or nu-ls vs ls. Then you wouldn't need escaping, and the names wouldn't shadow traditional commands (but could allude to them).
- okaram 7y agoAbout namespacing, they are doing something equivalent, but making theirs the default. If they make theirs nu.ls, many (most) people would type ls ;)
- coldtea 7y ago>* If they make theirs nu.ls, many (most) people would type ls ;)* Sure, and they'll get "ls" (the standard one, from POSIX), which should be the least surprise. From then on, they'll know to call nu.ls explicitly when they need it. Sounds like a better option.
- chubot 7y agoFWIW, one of the reasons I made https://oilshell.org https://oilshell.org sh-compatible is because it already has a very elegant answer to this problem: Pipelines Support Vectorized, Point-Free, and Imperative Style http://www.oilshell.org/blog/2017/01/15.html http://www.oilshell.org/blog/2017/01/15.html You just do myls() { ls -la | lines ... ; } my ls | awk '$1 > 1000' That is, a function can be TRANSPARENTLY put in a pipeline. Shell functions have stdin and stdout that compose. The awk part isn't ideal, but I have some plans to fix that: http://www.oilshell.org/blog/tags.html?tag=awk#awk http://www.oilshell.org/blog/tags.html?tag=awk#awk https://github.com/oilshell/oil/wiki/Structured-Data-in-Oil https://github.com/oilshell/oil/wiki/Structured-Data-in-Oil
- naasking 7y agoNot sure if they're aware of Shcaml, which seems even more powerful since it supports anonymous records in lines: https://www.ccs.neu.edu/home/tov/code/shcaml/doc/ https://www.ccs.neu.edu/home/tov/code/shcaml/doc/ Edit: features look similar in fact, since you can name columns and refer to them by name. Neat!
- KhoomeiK 7y agoAm I the only one who first read the title as "nutshell"?
- skywhopper 7y agoThis is really cool. I've always liked the ideas behind Powershell's approach, and so I'm glad to see someone is working on Linux-friendly shell that has the same sort of features. I have to say that I don't care for the snark about "having to remember" arguments to various pipeline-munging commands. I get that this is a big challenge for new folks working with typical unix shells, but I'm not sure I see how this actually solves it. In order to avoid all those flags, a system like this requires more verbs overall, and those verbs have their own argument structure, eg `split-column " | " firstname lastname job`, so now instead of being able to label the arguments and order them how I want, I now have to remember the order they come in, and what they mean, and that `split-column " "` apparently eats extra whitespace (does `split-column " | "`?). Yes, the standard unix tools have these issues, as well. That's my point. So yes, I love this idea, please keep pursuing it, and I look forward to trying it out. But focus on the benefits of working with structured data rather than pushing lines that sound good but aren't actually true.
- deleted 7y ago[deleted]
- simonblack 7y agoWhy did I keep getting the impression on reading the article that I was seeing mysql 'select' readouts?
- nihilarian 7y agoA nushell for the numan