5 ms·

I feel like these tools very much go against the Unix philosophy of "Write programs that do one thing and do it well". They try to do the pretty user interface

I feel like these tools very much go against the Unix philosophy of "Write programs that do one thing and do it well". They try to do the pretty user interface and the underlying operation in a single tool. I prefer PowerShell in this respect where the output of each command is not text streams (as in the Unix world) but objects which can be operated on in a more object oriented way. You spend less time thinking about text parsing and more time thinking about the data you're working with. I think the command line is great as a way of manipulating data streams but it is incredibly lacking as a user interface. There is very little consistency of the interface between commands and new commands and options aren't easily discoverable.

burntsushi 8y ago
ripgrep definitely goes against the Unix philosophy. This is intentional. The Unix philosophy is a means to an end, and not an end unto itself. The key way that ripgrep violates the Unix philosophy is that it couples the filtering of what to search with the act of searching. You hit the nail on the head with styling, because the output of ripgrep is itself the thing that prevents composability. However, from my own observations (and my own use), folks seem to appreciate having some amount of coupling and integration here. I've often seen folks claim that a legitimate use case (for them) for ripgrep is to "replace crazy 'find ./ ... | xargs' concoctions." The degree to which the aforementioned is "crazy" or not varies based on the individual, but there's a not insubstantial number of people who appreciate the succinctness that coupling brings. As the maintainer, the coupling is annoying, because it means ripgrep needs to implement more stuff. For example, ripgrep provides a `--sort-files` option, which is something that standard grep doesn't need because you can fairly easy compose its output with 'sort'. You could do the same with ripgrep (ripgrep supports the standard composable grep output format), but then you lose the "pretty styling" that users appreciate. So you have no choice. In terms of giving more structure to output, I am mostly unconvinced by your argument, but I've been happily using text streams as my user interface for a very long time, and I like its balance. With that said, the next release of ripgrep will come with a --json flag, which enables structured output. :-)
- bradknowles 8y ago
  Which I can then pipe to "gron" so that I can make something sensible and greppable out of that json output, right? ;)
  - dflock 8y ago
    Well, you could pipe it to "jq" - https://stedolan.github.io/jq/ https://stedolan.github.io/jq/
    - needlepont 8y ago
      Which is a pretty abominable tool TBH. It breaks on just about every little edge and isn't very useful unless you like to see pretty colors and nested outputs + learn it's terrible output formatting spec. When using json I still use jansson for quick one offs.
- jgtrosh 8y ago
  I wonder if there is or could be a good standard way of providing decoupled tools and their coupled interface. Last year I wrote a friendly script to search content on YouTube, which prints human-readable content if plugged to a terminal and URLs otherwise, and another one which applies a pattern to turn such YouTube URLs to the underlying video or audio streams using youtube-dl. (I don't think this plays along nicely with YouTube's ToS but whatever I'm the only user.) The obvious use case is to look at the output of the first command, and then pipe it to the second after choosing what to watch, and find a way to feed that to mplayer. So you'd want to provide a shortcut script, maybe make it interactive… but when does it go from an alias to a new program altogether? This interaction is very intuitive in the browser and I find trying to reproduce it with a CLI tool is quite challenging. Btw I love ripgrep, it made me look very hip in front of colleagues a couple of times.
- needlepont 8y ago
  I've never understood complaints like this. If you are proficient in the unix environment any sort of gymnastics can be handled via generation of some 'object' from plaintext and command generation on the fly. find $PATH -name '.c' -exec grep -l socket {} \; | awk ' {printf "mv %s %s\n",$0,sprintf("%s.old",$0)}' find $PATH -name '.c' -exec grep -l socket {} \; | awk ' BEGIN {n=0} {printf "{\"items\": \%s",sprintf("[\"%d\",\"%s\"]}\n",n++,$0)}' If you aren't proficient or have an aesthetic or religious aversion to unix userland and traditional tools you'll play some other game I guess. Reinventing the wheel without understanding the model and power is a next-gen game. I don't have time for it.
demosito666 8y ago
This is correct. The thing is, powershell is not for humans. It is purposed more towards configuration and system scipting, and thus should be compared to something like ansible. Using it as a shell is counter-productive unless you have very specific mindset. Unix shell, on the other hand, is a trade-off: it offers you options to process and automate and reasonable convenience when working interactively. None of those aspects is perfect, of course, but it allows gradual learning curve (i.e. being able to do quick and dirty things from the beginning), is much more versatile and ubiquitous.
- deleted 8y ago
  [deleted]
- h1d 8y ago
  Who is going to write "configuration and system scipting" if it isn't for humans? Ansible playbook reads very easily to humans.
  - demosito666 8y ago
    Try to run it interactively :)
- krylon 8y ago
  At work, I always keep a PowerShell instance open. I like the fact that many common commands have Unix-like aliases, plus a lot of server software offers PowerShell interfaces for administration (e.g. Exchange, VSphere, SharePoint, ...). On Windows, I strongly prefer PowerShell for interactive work, although I cringe a little each time I see how much memory it uses.
l0b0 8y ago
"One thing" has never been very well defined - basically every common Linux shell tool could be said to do more than one thing. GNU `grep` supports four different pattern styles (fixed strings, basic regex, extended regex and Perl regex), has lots of options for output formatting (counting, line numbers, whether to display file names, etc.) and has a bunch of rarely-used (but useful!) options for various corner cases, such as line buffering. Even GNU `cat`, the canonical "do one thing" tool, has several formatting options. If `grep` returned match objects rather than a semantically void binary stream it would be really easy to hand over formatting to another tool, but that just isn't how *nix tools work yet.
- bradknowles 8y ago
  The thing I would like to see with grep is to have it optionally give me a 0 return code if it doesn't find anything. I know why the authors chose to use a non-zero return code as the default, but when I'm using grep deep in a pipeline and doing my own checking of the results to see if nothing was found, I don't need grep bombing out the whole pipeline with a non-zero return code. The alternative of being forced to use "(grep pattern||true)" instead of a plain "grep -q" is kinda painful.
  - vitorbaptistaa 8y ago
    I'm not in my computer now, but doesn't "grep -v" solves your issue? If I remember correctly, it inverts the query, so it would return 0 if there wasn't a match.
    - Sean1708 8y ago
      No, -v will return 0 unless every line matches: $ cat > foo here's some random text $ grep -v some foo; echo $? here's random text 0 $ grep -v bar foo; echo $? here's some random text 0 $ grep -vE "(here's|some|random|text)" foo; echo $? 1
  - mangamadaiyan 8y ago
    Would grep -c work for your use-case?
Nelson69 8y ago
How should ls coloring work in your view? ls just does the file system reading and then some other tool parses the ls output matches some expressions and adds color? Or something like that? I suppose it's just a generic colorizer tool that takes a configuration file of expressions and arbitrarily colors data feed through it?
jniedrauer 8y ago
I disagree about objects being preferable to text in a shell. Text is very easy to reason about, and you can quickly determine what transformations you need to make based on visual feedback. Working with objects means spending a lot more time in the documentation learning what properties you have to work with. In a programming language, it's obviously no context. But in a shell, I want to spend more time doing and less time reading.
- TeMPOraL 8y ago
  > Text is very easy to reason about, and you can quickly determine what transformations you need to make based on visual feedback. Working with objects means spending a lot more time in the documentation learning what properties you have to work with. Or not. Take `ps`. You'll need to spend time in documentation anyway, figuring out what process properties can be shown with what flag, and then you'll be bitten later by things like the difference between `ps ux` and `ps aux` including process names in square brackets, etc. Contrast with PS equivalent, `Get-Process`. Type `Get-Process | Get-Member` to list properties of the objects returned by `Get-Process`, and you can quickly see both properties you can inspect (with descriptive names, not "VSZ" or "RSS") and what methods you can call directly (instead of extracting properties and piping to other programs). This is IMO much cleaner, easier to work with interactively (properties instead of constant parsing and unparsing of text), better for interoperability (you're limited by what actual objects expose, not by what pieces of them a CLI program wishes to print, and if it so happens that objects somehow print more than they expose in properties, you can still call ToString() on them and get that data "the UNIX way"), and correctly separates presentation from content. The only real drawback I've seen of Powershell is the lack of quality-of-life scripts and executables in the system. Like the md5 example elsewhere in this thread.
  - jniedrauer 8y ago
    A shell is supposed to be an interactive user interface and a very thin glue layer for some simple tasks. Once you move beyond a certain level of complexity, a shell is just the wrong tool for the job. Powershell is too complex to be a good shell, and there are too many bizarre idiosyncrasies about it to hold its own as a programming language. It just doesn't really have a place.

burntsushi 8y ago

ripgrep definitely goes against the Unix philosophy. This is intentional. The Unix philosophy is a means to an end, and not an end unto itself. The key way that ripgrep violates the Unix philosophy is that it couples the filtering of what to search with the act of searching. You hit the nail on the head with styling, because the output of ripgrep is itself the thing that prevents composability. However, from my own observations (and my own use), folks seem to appreciate having some amount of coupling and integration here. I've often seen folks claim that a legitimate use case (for them) for ripgrep is to "replace crazy 'find ./ ... | xargs' concoctions." The degree to which the aforementioned is "crazy" or not varies based on the individual, but there's a not insubstantial number of people who appreciate the succinctness that coupling brings. As the maintainer, the coupling is annoying, because it means ripgrep needs to implement more stuff. For example, ripgrep provides a `--sort-files` option, which is something that standard grep doesn't need because you can fairly easy compose its output with 'sort'. You could do the same with ripgrep (ripgrep supports the standard composable grep output format), but then you lose the "pretty styling" that users appreciate. So you have no choice. In terms of giving more structure to output, I am mostly unconvinced by your argument, but I've been happily using text streams as my user interface for a very long time, and I like its balance. With that said, the next release of ripgrep will come with a --json flag, which enables structured output. :-)

bradknowles 8y ago
Which I can then pipe to "gron" so that I can make something sensible and greppable out of that json output, right? ;)
- dflock 8y ago
  Well, you could pipe it to "jq" - https://stedolan.github.io/jq/ https://stedolan.github.io/jq/
  - needlepont 8y ago
    Which is a pretty abominable tool TBH. It breaks on just about every little edge and isn't very useful unless you like to see pretty colors and nested outputs + learn it's terrible output formatting spec. When using json I still use jansson for quick one offs.
jgtrosh 8y ago
I wonder if there is or could be a good standard way of providing decoupled tools and their coupled interface. Last year I wrote a friendly script to search content on YouTube, which prints human-readable content if plugged to a terminal and URLs otherwise, and another one which applies a pattern to turn such YouTube URLs to the underlying video or audio streams using youtube-dl. (I don't think this plays along nicely with YouTube's ToS but whatever I'm the only user.) The obvious use case is to look at the output of the first command, and then pipe it to the second after choosing what to watch, and find a way to feed that to mplayer. So you'd want to provide a shortcut script, maybe make it interactive… but when does it go from an alias to a new program altogether? This interaction is very intuitive in the browser and I find trying to reproduce it with a CLI tool is quite challenging. Btw I love ripgrep, it made me look very hip in front of colleagues a couple of times.
needlepont 8y ago
I've never understood complaints like this. If you are proficient in the unix environment any sort of gymnastics can be handled via generation of some 'object' from plaintext and command generation on the fly. find $PATH -name '.c' -exec grep -l socket {} \; | awk ' {printf "mv %s %s\n",$0,sprintf("%s.old",$0)}' find $PATH -name '.c' -exec grep -l socket {} \; | awk ' BEGIN {n=0} {printf "{\"items\": \%s",sprintf("[\"%d\",\"%s\"]}\n",n++,$0)}' If you aren't proficient or have an aesthetic or religious aversion to unix userland and traditional tools you'll play some other game I guess. Reinventing the wheel without understanding the model and power is a next-gen game. I don't have time for it.

demosito666 8y ago

This is correct. The thing is, powershell is not for humans. It is purposed more towards configuration and system scipting, and thus should be compared to something like ansible. Using it as a shell is counter-productive unless you have very specific mindset. Unix shell, on the other hand, is a trade-off: it offers you options to process and automate and reasonable convenience when working interactively. None of those aspects is perfect, of course, but it allows gradual learning curve (i.e. being able to do quick and dirty things from the beginning), is much more versatile and ubiquitous.

deleted 8y ago
[deleted]
h1d 8y ago
Who is going to write "configuration and system scipting" if it isn't for humans? Ansible playbook reads very easily to humans.
- demosito666 8y ago
  Try to run it interactively :)
krylon 8y ago
At work, I always keep a PowerShell instance open. I like the fact that many common commands have Unix-like aliases, plus a lot of server software offers PowerShell interfaces for administration (e.g. Exchange, VSphere, SharePoint, ...). On Windows, I strongly prefer PowerShell for interactive work, although I cringe a little each time I see how much memory it uses.

l0b0 8y ago

"One thing" has never been very well defined - basically every common Linux shell tool could be said to do more than one thing. GNU `grep` supports four different pattern styles (fixed strings, basic regex, extended regex and Perl regex), has lots of options for output formatting (counting, line numbers, whether to display file names, etc.) and has a bunch of rarely-used (but useful!) options for various corner cases, such as line buffering. Even GNU `cat`, the canonical "do one thing" tool, has several formatting options. If `grep` returned match objects rather than a semantically void binary stream it would be really easy to hand over formatting to another tool, but that just isn't how *nix tools work yet.

bradknowles 8y ago
The thing I would like to see with grep is to have it optionally give me a 0 return code if it doesn't find anything. I know why the authors chose to use a non-zero return code as the default, but when I'm using grep deep in a pipeline and doing my own checking of the results to see if nothing was found, I don't need grep bombing out the whole pipeline with a non-zero return code. The alternative of being forced to use "(grep pattern||true)" instead of a plain "grep -q" is kinda painful.
- vitorbaptistaa 8y ago
  I'm not in my computer now, but doesn't "grep -v" solves your issue? If I remember correctly, it inverts the query, so it would return 0 if there wasn't a match.
  - Sean1708 8y ago
    No, -v will return 0 unless every line matches: $ cat > foo here's some random text $ grep -v some foo; echo $? here's random text 0 $ grep -v bar foo; echo $? here's some random text 0 $ grep -vE "(here's|some|random|text)" foo; echo $? 1
- mangamadaiyan 8y ago
  Would grep -c work for your use-case?

Nelson69 8y ago

How should ls coloring work in your view? ls just does the file system reading and then some other tool parses the ls output matches some expressions and adds color? Or something like that? I suppose it's just a generic colorizer tool that takes a configuration file of expressions and arbitrarily colors data feed through it?

jniedrauer 8y ago

I disagree about objects being preferable to text in a shell. Text is very easy to reason about, and you can quickly determine what transformations you need to make based on visual feedback. Working with objects means spending a lot more time in the documentation learning what properties you have to work with. In a programming language, it's obviously no context. But in a shell, I want to spend more time doing and less time reading.

TeMPOraL 8y ago
> Text is very easy to reason about, and you can quickly determine what transformations you need to make based on visual feedback. Working with objects means spending a lot more time in the documentation learning what properties you have to work with. Or not. Take `ps`. You'll need to spend time in documentation anyway, figuring out what process properties can be shown with what flag, and then you'll be bitten later by things like the difference between `ps ux` and `ps aux` including process names in square brackets, etc. Contrast with PS equivalent, `Get-Process`. Type `Get-Process | Get-Member` to list properties of the objects returned by `Get-Process`, and you can quickly see both properties you can inspect (with descriptive names, not "VSZ" or "RSS") and what methods you can call directly (instead of extracting properties and piping to other programs). This is IMO much cleaner, easier to work with interactively (properties instead of constant parsing and unparsing of text), better for interoperability (you're limited by what actual objects expose, not by what pieces of them a CLI program wishes to print, and if it so happens that objects somehow print more than they expose in properties, you can still call ToString() on them and get that data "the UNIX way"), and correctly separates presentation from content. The only real drawback I've seen of Powershell is the lack of quality-of-life scripts and executables in the system. Like the md5 example elsewhere in this thread.
- jniedrauer 8y ago
  A shell is supposed to be an interactive user interface and a very thin glue layer for some simple tasks. Once you move beyond a certain level of complexity, a shell is just the wrong tool for the job. Powershell is too complex to be a good shell, and there are too many bizarre idiosyncrasies about it to hold its own as a programming language. It just doesn't really have a place.