5 ms·

Thanks for the response! Some notes: 1. In my benchmarks, I do control for line numbers by either explicitly making it a variable (i.e., when you see `(lines)`

Thanks for the response! Some notes: 1. In my benchmarks, I do control for line numbers by either explicitly making it a variable (i.e., when you see `(lines)`) or by making all tools count lines to make the comparison fair. For the most part, this only tends to matter in the single-file benchmarks. 2. For memory maps, you might get very different results depending on your environment. For example, I enabled memory maps on Windows where they seem to do a bit better. (I think my blog post gives enough details that you could reproduce the benchmark environment precisely if you were so inclined. This was important to me, so I spent a lot of time documenting it.) 3. The set of features supported by rg should be very very close to what is supported by ag. Reviewing `ag`'s man page again, probably the only things missing from rg are --ackmate, --depth, some of the color configurability flags (but rg does do coloring), --passthrough, --smart-case and --stats maybe? I might be missing some others. And Mercurial support (but ag's is incomplete). In exchange, rg gives you much better single file performance, better large-repo performance and real Unicode support that doesn't slow way down. I'd say those are pretty decent expectations. :-) Thanks for ag by the way. It and ack have definitely triggered a new kind of searching. I have some further information retrievalish ideas on evolving the concept, but those will have to wait!

kzrdude 10y ago
`ag --pager` is the most important one (I use alias ag='ag --pager "less -FRX"')
ggreer 10y ago
In terms of core features, ripgrep is totally there. It searches fast. It ignores files pretty accurately. It outputs results in a pleasant and useful format. If a new user tries rg, they'll be very happy. My warning about the feature differences was meant to temper ag users' expectations. There are lots of little things that ag users are accustomed to that are either different or missing in ripgrep. Off the top of my head: Ag reads the user's global gitignore. (This is harder than most people think.) It detects stdout redirects such as "ag blah > output.txt" and ignores output.txt. It can search gz and xz files. It defaults to smart-case searching. It can limit a search to one hardware device (--one-device), avoiding slow reads on network mounts. And as a commenter already pointed out, it supports the --pager option. Taken together, all those small differences are likely to cause an average ag user some grief. I wanted to manage expectations so that users wouldn't create annoying "issues" (really, feature requests) on your GitHub repo. Sorry if that came off the wrong way. On a completely unrelated note: I see ripgrep supports .rgignore files, similar to how ag supports .agignore. It'd be nice if we could combine forces and choose a single filename for this purpose. That way when the next search tool comes along, it can use the same thing instead of .zzignore or whatever. It would also make it easier for users to switch between our tools. I'd suggest a generic name like ".ignore" or ".ignores", but I'm sure some tool creates such files or directories already. Edit: Actually, it looks like .ignore can work. The only examples I've found of .ignore files are actual files containing ignore patterns.
- mvermaat 10y ago
  I'd say that .ignore is too generic a name. What is to be ignored by what? But I like the idea of standardizing this. Perhaps the cache directory tagging standard gives some inspiration. http://www.brynosaurus.com/cachedir/spec.html http://www.brynosaurus.com/cachedir/spec.html
- samatman 10y ago
  Could I suggest .ignorerc? I would immediately intuit that such a file is a (r)untime (c)onfiguration for ignoring something. .ignore isn't... bad, it just looks like something I can safely delete, like a file styled `such-and-such~`
  - gwern 10y ago
    If you're taking the time to create a .ignore file and set up personally-relevant regexps, I doubt you'll forget. (You'll probably also have it versioned.) What I like about '.ignore' is that it's not tied to grep (which will never use it) but expresses that the concept is agnostic. You can imagine lots of programs supporting it.
- kaushalmodi 10y ago
  May be .searchignore?
  - rurban 10y ago
    .grepignore
- burntsushi 10y ago
  You raise good points, thank you. I hope to support some of those features, since they seem like nice conveniences. In principle I'd be fine standardizing on a common ignore file. We'd need to come up with a format (I think I'd vote for "do what gitignore does", since I think that's what we're both doing now anyway). Adding files to this list is kind of a bummer though. I could probably get away with replacing `.rgignore` proper, but I suspect you'd need to add it without replacing `.agignore`, or else those angry users you were talking about might show themselves. :-) I do kind of like `.grepignore` since `grep` has kind of elevated itself to "search tool" as a term, but I can see how that would be confusing. `.searchignore` feels too long. `.ignore` and `.ignorerc` feel a bit too generic, but either seems like the frontrunner at the moment.
coldtea 10y ago
I notice that subsequent runs in the same (non changing) directory get different results. These runs are all within 20 seconds, what gives? $ rg each | md5sum 670b544e15f9430d9934334a11a87b7e - $ rg each | md5sum 4d13be6b4531ad52b1b476314fe98fb7 - rg each | md5sum 88e15dbb943665ea54482cb499741938 - rg each | md5sum eec6d6d5c9a592cec25aa8b0c19aae15 - rg each | md5sum ad74b78ef8f0d21450f8f87415555af0 - And: $ date Sat Sep 24 01:42:27 EEST 2016 $ rg each > foo1 $ rg each > foo2 $ rg each > foo3 $ rg each > foo4 $ ls -la foo* -rw-r--r-- 1 coldtea staff 1429646 Sep 24 01:42 foo1 -rw-r--r-- 1 coldtea staff 2250868 Sep 24 01:42 foo2 -rw-r--r-- 1 coldtea staff 4536031 Sep 24 01:42 foo3 -rw-r--r-- 1 coldtea staff 9140652 Sep 24 01:42 foo4 $ date Sat Sep 24 01:42:44 EEST 2016 OS X 10.12, installed with brew.
- ggreer 10y ago
  In the first case, it's searching in parallel, so I bet the order of results is different each time. In the second case, rg each > foo2 found results in foo1 and put them in foo2. Then rg each > foo3 found results in foo1 and foo2, and put them in foo3. Etc. That's why the file size increases so quickly.
  - coldtea 10y ago
    >In the first case, it's searching in parallel, so I bet the order of results is different. Aha. Thought that needed the -j flag (it says: default threads: 0 in the cli help). Could it do anything to put them out in order of "depth" (and directory/file sorting order)? >In the second case, rg each > foo2 found results in foo1 and put them in foo2. Then rg each > foo3 found results in foo1 and foo2, and put them in foo3. Etc. That's why the file size increases so quickly. LOL, facepalm -- yes.
    - ggreer 10y ago
      Forcing it to use one worker (-j1, I think) should give it deterministic output.
- mbrubeck 10y ago
rocho 10y ago
I just made a cursory pass at `man rg`, but it seems to me that the `-g` option from ag is also missing. I use it with vim-ctrlp to search file names. Thanks for rg and the very informative blog post!
- burntsushi 10y ago
  It's there. You need to pass --files.

kzrdude 10y ago

`ag --pager` is the most important one (I use alias ag='ag --pager "less -FRX"')

ggreer 10y ago

In terms of core features, ripgrep is totally there. It searches fast. It ignores files pretty accurately. It outputs results in a pleasant and useful format. If a new user tries rg, they'll be very happy. My warning about the feature differences was meant to temper ag users' expectations. There are lots of little things that ag users are accustomed to that are either different or missing in ripgrep. Off the top of my head: Ag reads the user's global gitignore. (This is harder than most people think.) It detects stdout redirects such as "ag blah > output.txt" and ignores output.txt. It can search gz and xz files. It defaults to smart-case searching. It can limit a search to one hardware device (--one-device), avoiding slow reads on network mounts. And as a commenter already pointed out, it supports the --pager option. Taken together, all those small differences are likely to cause an average ag user some grief. I wanted to manage expectations so that users wouldn't create annoying "issues" (really, feature requests) on your GitHub repo. Sorry if that came off the wrong way. On a completely unrelated note: I see ripgrep supports .rgignore files, similar to how ag supports .agignore. It'd be nice if we could combine forces and choose a single filename for this purpose. That way when the next search tool comes along, it can use the same thing instead of .zzignore or whatever. It would also make it easier for users to switch between our tools. I'd suggest a generic name like ".ignore" or ".ignores", but I'm sure some tool creates such files or directories already. Edit: Actually, it looks like .ignore can work. The only examples I've found of .ignore files are actual files containing ignore patterns.

mvermaat 10y ago
I'd say that .ignore is too generic a name. What is to be ignored by what? But I like the idea of standardizing this. Perhaps the cache directory tagging standard gives some inspiration. http://www.brynosaurus.com/cachedir/spec.html http://www.brynosaurus.com/cachedir/spec.html
samatman 10y ago
Could I suggest .ignorerc? I would immediately intuit that such a file is a (r)untime (c)onfiguration for ignoring something. .ignore isn't... bad, it just looks like something I can safely delete, like a file styled `such-and-such~`
- gwern 10y ago
  If you're taking the time to create a .ignore file and set up personally-relevant regexps, I doubt you'll forget. (You'll probably also have it versioned.) What I like about '.ignore' is that it's not tied to grep (which will never use it) but expresses that the concept is agnostic. You can imagine lots of programs supporting it.
kaushalmodi 10y ago
May be .searchignore?
- rurban 10y ago
  .grepignore
burntsushi 10y ago
You raise good points, thank you. I hope to support some of those features, since they seem like nice conveniences. In principle I'd be fine standardizing on a common ignore file. We'd need to come up with a format (I think I'd vote for "do what gitignore does", since I think that's what we're both doing now anyway). Adding files to this list is kind of a bummer though. I could probably get away with replacing `.rgignore` proper, but I suspect you'd need to add it without replacing `.agignore`, or else those angry users you were talking about might show themselves. :-) I do kind of like `.grepignore` since `grep` has kind of elevated itself to "search tool" as a term, but I can see how that would be confusing. `.searchignore` feels too long. `.ignore` and `.ignorerc` feel a bit too generic, but either seems like the frontrunner at the moment.

coldtea 10y ago

I notice that subsequent runs in the same (non changing) directory get different results. These runs are all within 20 seconds, what gives? $ rg each | md5sum 670b544e15f9430d9934334a11a87b7e - $ rg each | md5sum 4d13be6b4531ad52b1b476314fe98fb7 - rg each | md5sum 88e15dbb943665ea54482cb499741938 - rg each | md5sum eec6d6d5c9a592cec25aa8b0c19aae15 - rg each | md5sum ad74b78ef8f0d21450f8f87415555af0 - And: $ date Sat Sep 24 01:42:27 EEST 2016 $ rg each > foo1 $ rg each > foo2 $ rg each > foo3 $ rg each > foo4 $ ls -la foo* -rw-r--r-- 1 coldtea staff 1429646 Sep 24 01:42 foo1 -rw-r--r-- 1 coldtea staff 2250868 Sep 24 01:42 foo2 -rw-r--r-- 1 coldtea staff 4536031 Sep 24 01:42 foo3 -rw-r--r-- 1 coldtea staff 9140652 Sep 24 01:42 foo4 $ date Sat Sep 24 01:42:44 EEST 2016 OS X 10.12, installed with brew.

ggreer 10y ago
In the first case, it's searching in parallel, so I bet the order of results is different each time. In the second case, rg each > foo2 found results in foo1 and put them in foo2. Then rg each > foo3 found results in foo1 and foo2, and put them in foo3. Etc. That's why the file size increases so quickly.
- coldtea 10y ago
  >In the first case, it's searching in parallel, so I bet the order of results is different. Aha. Thought that needed the -j flag (it says: default threads: 0 in the cli help). Could it do anything to put them out in order of "depth" (and directory/file sorting order)? >In the second case, rg each > foo2 found results in foo1 and put them in foo2. Then rg each > foo3 found results in foo1 and foo2, and put them in foo3. Etc. That's why the file size increases so quickly. LOL, facepalm -- yes.
  - ggreer 10y ago
    Forcing it to use one worker (-j1, I think) should give it deterministic output.
mbrubeck 10y ago

rocho 10y ago

I just made a cursory pass at `man rg`, but it seems to me that the `-g` option from ag is also missing. I use it with vim-ctrlp to search file names. Thanks for rg and the very informative blog post!

burntsushi 10y ago
It's there. You need to pass --files.