4 ms·

ripgrep[1] is functionally incredibly similar to grep and ag, but is significantly faster[2] and supports a wider range of character encodings. In its short lif

ripgrep[1] is functionally incredibly similar to grep and ag, but is significantly faster[2] and supports a wider range of character encodings. In its short lifetime it has already become the default search tool for VSCode. I've switched to using it as my daily driver for text search and am incredibly happy with it. [1]: https://github.com/BurntSushi/ripgrep https://github.com/BurntSushi/ripgrep [2]: http://blog.burntsushi.net/ripgrep/ http://blog.burntsushi.net/ripgrep/ Edit: I confused awk with ag originally leading to this comment. Using ripgrep as a pre-filter to awk is still a ridiculous amount faster, especially on large trees, so while the OP's suggestion is cool I can't see myself reaching for it often.

burntsushi 9y ago
> and supports a wider range of character encodings Since this is an uncommon feature, I'd like to emphasize this. :-) In particular, ripgrep will automatically search UTF-16 encoded files via BOM sniffing. That means you can run ripgrep over a directory on Windows and be confident that it will correctly search both UTF-8 and UTF-16 encoded files automatically without having to think about it. More generally, it supports all encodings found in the Encoding Standard[1]. However, only UTF-16 is automatically detected (where UTF-8 is the presumed default), so you'll need to explicitly specify `-E sjis` (for example) if you want to search Shift_JIS encoded files. Also, I didn't really need to do much of anything to get this working. This is all thanks to @hsivonen's encoding_rs[2] crate, which is now (I think) in Firefox. [1] - https://encoding.spec.whatwg.org/#concept-encoding-get https://encoding.spec.whatwg.org/#concept-encoding-get [2] - https://github.com/hsivonen/encoding_rs https://github.com/hsivonen/encoding_rs
cyphar 9y ago
awk is a full programming language, and most of the times that I'm doing awk scripting I have to use things like associative arrays and arithmetic. Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's all that $n does). I was under the impression that ripgrep is a grep implementation that was incredibly optimised thanks to BurntSushi being a complete madman.
- burntsushi 9y ago
  Yeah, ripgrep is "a grep," not an awk. I'm not sure how they wound up being conflated here. ripgrep does have a `-r/--replace` flag which is somewhat of a generalization of grep's `-o/--only-matching` flag (and also part of ack, I believe) by permitting sub-capture expansion that probably does replace awk for the "pulling out a field from a line" use case you mentioned. But it's pretty awkward for simple cases. e.g., I'd much rather use `blah | awk '{print $2}'` to get the second field delimited by arbitrary whitespace.
  - coldtea 9y ago
    >Yeah, ripgrep is "a grep," not an awk. I'm not sure how they wound up being conflated here. Simple: a lot of people use awk just as a grep.
- fiddlerwoaroof 9y ago
  The main reason I use awk 90% of the time is that its field parsing algorithm does "the right thing" in most cases (i.e. divide fields by 1 or more whitespace characters) without a lot of boilerplate, so it's really easy to throw into a pipeline.
  - cyphar 9y ago
    That's what I was referencing when I said > Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's all that $n does). There's nothing magical about awk's default FS. It's literally just /\s+/. If cut's -d was slightly more clever you wouldn't need to use awk.
    - bewuethr 9y ago
      The default FS throws away leading blanks, though, which doesn't happen if you set it explicitly to \s+, so a tiny little bit of magic does go on after all.
deleted 9y ago
[deleted]
usernam 9y ago
ripgrep doesn't yet support compressed files (z/gz), while grep/ack/ag do. On *nix it's extremely common to have sparsely compressed directories, from logfiles to non-changing documentation. I really wished ripgrep would just stream to a fast coprocess and support any stream compressor transparently instead of trying to use a built-in rust library.
- burntsushi 9y ago
  Suggestions are most welcome on the issue tracker. I don't think your idea has been suggested yet?
  - usernam 9y ago
    I have commented on #225 about this.
dingo_bat 9y ago
I tried a lot but could not install ripgrep on ubuntu :( Apparently the code is in rust, which I know zero of. So, I didn't try building it either.
- burntsushi 9y ago
  It's simple to install ripgrep on pretty much any Linux because I distribute statically compiled binaries: $ curl -LO 'https://github.com/BurntSushi/ripgrep/releases/download/0.5.2/ripgrep-0.5.2-x86_64-unknown-linux-musl.tar.gz' $ tar xf ripgrep-0.5.2-*.tar.gz $ cp ripgrep-0.5.2-*/rg $HOME/bin/rg If you're not a fan of this approach (downloading random binaries and slapping them into your $HOME/bin), then your other choice is to build from source. I don't use Ubuntu, but install Rust[1] and then building ripgrep is easy: $ git clone git://github.com/BurntSushi/ripgrep $ cd ripgrep $ cargo build --release $ ./target/release/rg -V ripgrep 0.5.2 And yes, it would be great to get ripgrep packaged into Ubuntu.[2] There seems to be an up-to-date PPA here.[3] [1] - https://www.rust-lang.org/en-US/install.html https://www.rust-lang.org/en-US/install.html [2] - https://github.com/BurntSushi/ripgrep/issues/10 https://github.com/BurntSushi/ripgrep/issues/10 [3] - https://launchpad.net/~x4121/+archive/ubuntu/ripgrep https://launchpad.net/~x4121/+archive/ubuntu/ripgrep

burntsushi 9y ago

> and supports a wider range of character encodings Since this is an uncommon feature, I'd like to emphasize this. :-) In particular, ripgrep will automatically search UTF-16 encoded files via BOM sniffing. That means you can run ripgrep over a directory on Windows and be confident that it will correctly search both UTF-8 and UTF-16 encoded files automatically without having to think about it. More generally, it supports all encodings found in the Encoding Standard[1]. However, only UTF-16 is automatically detected (where UTF-8 is the presumed default), so you'll need to explicitly specify `-E sjis` (for example) if you want to search Shift_JIS encoded files. Also, I didn't really need to do much of anything to get this working. This is all thanks to @hsivonen's encoding_rs[2] crate, which is now (I think) in Firefox. [1] - https://encoding.spec.whatwg.org/#concept-encoding-get https://encoding.spec.whatwg.org/#concept-encoding-get [2] - https://github.com/hsivonen/encoding_rs https://github.com/hsivonen/encoding_rs

cyphar 9y ago

awk is a full programming language, and most of the times that I'm doing awk scripting I have to use things like associative arrays and arithmetic. Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's all that $n does). I was under the impression that ripgrep is a grep implementation that was incredibly optimised thanks to BurntSushi being a complete madman.

burntsushi 9y ago
Yeah, ripgrep is "a grep," not an awk. I'm not sure how they wound up being conflated here. ripgrep does have a `-r/--replace` flag which is somewhat of a generalization of grep's `-o/--only-matching` flag (and also part of ack, I believe) by permitting sub-capture expansion that probably does replace awk for the "pulling out a field from a line" use case you mentioned. But it's pretty awkward for simple cases. e.g., I'd much rather use `blah | awk '{print $2}'` to get the second field delimited by arbitrary whitespace.
- coldtea 9y ago
  >Yeah, ripgrep is "a grep," not an awk. I'm not sure how they wound up being conflated here. Simple: a lot of people use awk just as a grep.
fiddlerwoaroof 9y ago
The main reason I use awk 90% of the time is that its field parsing algorithm does "the right thing" in most cases (i.e. divide fields by 1 or more whitespace characters) without a lot of boilerplate, so it's really easy to throw into a pipeline.
- cyphar 9y ago
  That's what I was referencing when I said > Pulling out a field from a line is what most people use awk for, but it's honestly the least interesting part of awk. In fact, if cut supported regular expressions for specifying the field and record separators people wouldn't be using awk for that purpose (because that's all that $n does). There's nothing magical about awk's default FS. It's literally just /\s+/. If cut's -d was slightly more clever you wouldn't need to use awk.
  - bewuethr 9y ago
    The default FS throws away leading blanks, though, which doesn't happen if you set it explicitly to \s+, so a tiny little bit of magic does go on after all.

deleted 9y ago

[deleted]

usernam 9y ago

ripgrep doesn't yet support compressed files (z/gz), while grep/ack/ag do. On *nix it's extremely common to have sparsely compressed directories, from logfiles to non-changing documentation. I really wished ripgrep would just stream to a fast coprocess and support any stream compressor transparently instead of trying to use a built-in rust library.

burntsushi 9y ago
Suggestions are most welcome on the issue tracker. I don't think your idea has been suggested yet?
- usernam 9y ago
  I have commented on #225 about this.

dingo_bat 9y ago

I tried a lot but could not install ripgrep on ubuntu :( Apparently the code is in rust, which I know zero of. So, I didn't try building it either.

burntsushi 9y ago
It's simple to install ripgrep on pretty much any Linux because I distribute statically compiled binaries: $ curl -LO 'https://github.com/BurntSushi/ripgrep/releases/download/0.5.2/ripgrep-0.5.2-x86_64-unknown-linux-musl.tar.gz' $ tar xf ripgrep-0.5.2-*.tar.gz $ cp ripgrep-0.5.2-*/rg $HOME/bin/rg If you're not a fan of this approach (downloading random binaries and slapping them into your $HOME/bin), then your other choice is to build from source. I don't use Ubuntu, but install Rust[1] and then building ripgrep is easy: $ git clone git://github.com/BurntSushi/ripgrep $ cd ripgrep $ cargo build --release $ ./target/release/rg -V ripgrep 0.5.2 And yes, it would be great to get ripgrep packaged into Ubuntu.[2] There seems to be an up-to-date PPA here.[3] [1] - https://www.rust-lang.org/en-US/install.html https://www.rust-lang.org/en-US/install.html [2] - https://github.com/BurntSushi/ripgrep/issues/10 https://github.com/BurntSushi/ripgrep/issues/10 [3] - https://launchpad.net/~x4121/+archive/ubuntu/ripgrep https://launchpad.net/~x4121/+archive/ubuntu/ripgrep