5 ms·
The Silver Searcher: An attempt to make something better than ack
- StavrosK 15y agoWait, how is anything faster than grep?
- steve-howard 15y agoI believe it's because ack and friends skip files you ordinarily don't want by default. So there's no .svn-base duplicates, no cache files generated by tools, etc.
- troels 15y agoSo `find ... | xargs grep ...` is still faster, right?
- petdance 15y agoDepends on if you include time to type the command. If your find command looks like find . -name '*.pl' -o -name '*.pm' | xargs grep foo and it takes 1 second to finish, and your ack command is ack foo --perl and it takes 1.5 seconds to finish, you can say the grep is faster. But I just timed the time it takes to type those, and they took me 9.2 vs. 2.3 seconds. So which is faster: 9.2+1.0 or 2.3+1.5?
- pixelbeat 15y agoI prefer the much simpler approach of reusing the existing utils with a simple shell script, which is much faster than ack: http://www.pixelbeat.org/scripts/findrepo http://www.pixelbeat.org/scripts/findrepo A quick test on a moderately big repo: $ time findrepo test '*' | wc -l 158819 real 0m0.532s $ time ack -a test | wc -l 76526 real 0m8.762s
- petdance 15y agoI'm more than happy to include findrepo on the betterthengrep.com/more-tools page. If you'll make a page for it, or at least have something for newbies to read, I'll add a link. My concern is that someone who's new to all of this isn't going to understand what to do if I just link to http://www.pixelbeat.org/scripts/findrepo http://www.pixelbeat.org/scripts/findrepo
- colomon 15y agoMuch as I've loved ack, I'd be all in favor of a replacement that was really significantly better. That said, the readme there lists five reasons why Silver Searcher is better than ack. Two of them are nonsense (who cares what language it's written in, or same-order-of-magnitude differences in how big the executables are?), two sound like they would be pretty trivial changes to ack, and the last is a significant speed improvement. But then you read further down the readme, and it says the current development state is somewhere between "Runs" and "Behaves correctly". Isn't it kind of premature to be bragging about how fast you are before your code actually behaves correctly? Also, it makes me wonder how much of the speed increase is based on the easy changes filtering out more files...
- Garoof 15y ago(who cares what language it's written in, or same-order-of-magnitude differences in how big the executables are?) That last reason there is about the length of the filename, not about the size of the file. I don't think anyone's supposed to really care, no.
- colomon 15y agoOh! Well, then that's another for the trivial to make ack do it list... :)
- AngryParsley 15y agoThe list is somewhat tongue-in-cheek. Also I haven't updated the README in a while. While ag certainly isn't as stable or feature-rich as ack, I have fixed bugs in ag that are still in ack. https://github.com/petdance/ack/issues/220 https://github.com/petdance/ack/issues/220 for example.
- petdance 15y agoHey, if you've got a solution for it, I'd love to hear.
- 15y ago
- kibwen 15y agoI haven't used ack since I discovered git's built-in `git grep` command, which, as I understand it, doesn't even need to sift through your files; it just examines your git index, so it's waaaay faster than either grep or ack. It's also recursive by default, doesn't require a file-matching parameter, and colorizes its output, which together were the reasons that I used ack over grep in the first place. Pretty brilliant, all told.
- colomon 15y agoOooo. I've still got to search lots of things that aren't grep, so it's not a complete ack replacement for me, but wow, git grep is like two orders of magnitude faster in my quick tests!
- js2 15y agoThe index is just a list of the files and associated metadata, it is not their contents. So git grep still needs to read the files from disk, though it doesn't need to walk the filesystem to locate them[@]. [@] Technically git grep has five modes of operation: 1. Search the contents of the tracked files as they currently are on disk. This is the default. 2. With --cached, search the contents of the tracked files as they are in the index (i.e. ignore any un-added changes). 3. With --no-index, search all files recursively from the current directory down. This allows you to use "git grep" as a "grep -R" replacement even when your CWD is not inside a repo. 4. With --untracked, search all files recursively from the current directory down in addition to files in the index. (The difference between this and --no-index when used inside a repo is that --untracked honors the .gitignore mechanism by default, i.e., --untracked is a synonym for --no-index --exclude-standard when inside a repo.) 5. With a tree'ish (commit, tag, branch name, etc), search all files in the tree.
- lobster_johnson 15y ago"git grep" only searches tracked files by default, so to get correct behaviour, you will need to create an alias (I don't think there's a way to change the default). It also only works on Git repos; having to think before choosing between "git grep" and "ack" just adds unnecessary mental context switching. Personally, I prefer ack's output, which puts the file name on a separate line, and includes line numbers by default. Together with -C you get a much more readable output. Mostly I end up just using Sublime Text's built-in file search, which is like ack, but has a generous -C setting enabled by default, and supports replacing.
- AngryParsley 15y agoI'm the author. People are asking how this thing is faster than ack or grep. Here's how: - Literal matches use Boyer-Moore-Horspool strstr.[1] - Files are mmap()ed instead of read into a buffer. - If you're building with PCRE 8.21 or greater, regex searches use the JIT compiler.[2] Also I call pcre_study() before executing the regex on a jillion files. - Ag reads your .gitignore and .hgignore files to ignore code you don't care about. - Instead of calling fnmatch() on every pattern in your ignore files, non-regex patterns are loaded into an array and binary searched. I wrote a couple of blog posts about profiling The Silver Searcher and improving performance. http://geoff.greer.fm/2012/01/23/making-programs-faster-profiling/ http://geoff.greer.fm/2012/01/23/making-programs-faster-prof... is the most informative one, IMO. 1. http://en.wikipedia.org/wiki/Boyer%E2%80%93Moore%E2%80%93Horspool_algorithm http://en.wikipedia.org/wiki/Boyer%E2%80%93Moore%E2%80%93Hor... 2. http://sljit.sourceforge.net/pcre.html http://sljit.sourceforge.net/pcre.html
- js2 15y agoFWIW, git grep also uses threads in some circumstances to get better performance. Also, obligatory link whenever Boyer-Moore is mentioned: http://ridiculousfish.com/blog/posts/old-age-and-treachery.html http://ridiculousfish.com/blog/posts/old-age-and-treachery.h...
- AngryParsley 15y agoThat's a cool blog post. I've tried not to look at the grep source code until I've written my own solutions, so I didn't know grep made that tradeoff.
- petdance 15y agoOooh, I forgot about your project when I did the betterthangrep.com website overhaul. I just added it to http://betterthangrep.com/more-tools/ http://betterthangrep.com/more-tools/