3 ms·

I could really use a better workflow to refine grep matches. Has anyone made a tool that combines grep (regex search) with fzf (multiple positive/negative patte

I could really use a better workflow to refine grep matches. Has anyone made a tool that combines grep (regex search) with fzf (multiple positive/negative patterns)? What I really want is something like: grep pattern1 **/* | grep pattern2 | grep -v exclude_these | grep -v also_exclude The problem is this loses filenames and context lines in the output. I want to apply several positive and negative regexes, and only at the very end annotate with filenames and context. Anyone have a good workflow for this?

totetsu 5y ago
Lnav with filters? or can you stack -e expression arguments onto one grep command?
codemac 5y ago
Are you ok with the initial list existing in memory? Then I highly recommend: emacs / vim buffers
henshao 5y ago
Just posting this without trying it out...grep -nH gives you line numbers and full filepath context, and as another commenter said, -ve allows you to string multiple excludes, so grep -inH <search> -ve not_this -ve nor_this -ve nor_that.
- fiddlerwoaroof 5y ago
  You don’t have to stack -v like that: -v inverts all the -e expressions, so this should be equivalent: grep -inHv -e foo -e bar -e baz
  - fiddlerwoaroof 5y ago
    So, awk sort of works as a fancy grep: ps axjw | awk '(/zsh/ || /login/) && !(/awk/ || /direnv/)' And you can easily add features like "print the first line unconditionally: ps axjw | awk 'NR == 1 {print} (/zsh/ || /login/) && !(/awk/ || /direnv/)' IMO, awk is in a pretty uniquely sweet spot between grep and perl/ruby/python.
    - henshao 5y ago
      Thanks for the grep tip! Unfortunately I've not put in the effort to learn awk past printing the n-th column of ls (my typical use) - the extra syntax required to properly 'quote' and {} things puts me off.
eminence32 5y ago
I glue together ripgrep with fzf and bat like this: rg --line-number --no-heading --color=always --smart-case "$@" | fzf -d ':' -n 2.. --ansi --no-sort --preview-window 'down:20%:+{2}' --preview 'bat --style=numbers --color=always --highlight-line {2} {1}' I get a small preview of the file in the bottom 20% of my terminal, and I can use fzf's matching to further filter, or to exclude things I capture the output and then `cut` it up to get filename and line number, which then get used to open the file/line in vim
- anotherhue 5y ago
  Thank you for this
montroser 5y ago
If your grep has -P, then you could do all in one go with lookahead assertions... grep -P '(?=.*pattern1)(?=.*pattern2)(?!.*exclude_these)' **/* Maybe not the most beautiful or easy to have at the fingertips, but could be worse.
rektide 5y ago
There's a ripgrep ticket for multiple patterns. No one seems to have come up with a good specification for exactly how it ought to work, so no development has started. https://github.com/BurntSushi/ripgrep/issues/875 https://github.com/BurntSushi/ripgrep/issues/875
LanternLight83 5y ago
https://paweldu.dev/posts/fzf-live-repl/ https://paweldu.dev/posts/fzf-live-repl/ Previous discussion here: https://news.ycombinator.com/item?id=20455857 https://news.ycombinator.com/item?id=20455857 Also featured in that thread: https://github.com/akavel/up https://github.com/akavel/up For example: echo '' | fzf --print-query --preview 'grep -R {q} . 2> /dev/null'
HellsMaddy 5y ago
Not exactly what you’re asking for but similar: I wrote a script called aag, based on ag, that lets you find files that contain multiple matches anywhere in the file. It’s a per-file logical AND of multiple ag searches. https://gist.github.com/b0o/f0759c84af6ef773a37a499fdb1c05af https://gist.github.com/b0o/f0759c84af6ef773a37a499fdb1c05af Example: you’re trying to find a file that contains the strings “MIT License” (case insensitive), “npm install”, and a match for “react-.*” somewhere in your home directory: aag ~ ---- -i "mit license" --- "npm install" --- "react-.*" The excessive dashes are necessary so that it’s possible to pass different options to each separate invocation of ag. In case it’s not clear why this is useful, normally ag (or grep) searches are linewise. It’s not so easy if you are looking for things that occur on different lines, and possibly in different orders.
tyingq 5y ago
Perl would be one option: perl -ne '( /foo/ and /bar/ and !/baz/ and !/splat/ ) and print "$ARGV: $_"' * I didn't include the context part, since that would be more than a one-liner, but it's not hard to do in a short script.
omaranto 5y ago
I do things like that inside Emacs with the consult-grep command from the Consult package, combined with the Orderless matching style, Embark to collect the results in a buffer. This has several advantages over the command line: - Interactivity: the results are updated live as you type, so you catch typos sooner and can tweak the search terms as you go. - The buffer of search results you collect with Embark is not dead text like it would be in the terminal, instead, each line is a link to the corresponding file taking you to the line that matched. - Wgrep lets you edit all the matching lines in place! Packages referenced: 1. Consult: https://github.com/minad/consult/ https://github.com/minad/consult/ 2. Orderless: https://github.com/oantolin/orderless https://github.com/oantolin/orderless 3. Embark: https://github.com/oantolin/embark https://github.com/oantolin/embark 4. Wgrep: http://github.com/mhayashi1120/Emacs-wgrep/raw/master/wgrep.el http://github.com/mhayashi1120/Emacs-wgrep/raw/master/wgrep....
dystroy 5y ago
You can combine patterns on file names and/or file content with broot. Here's my usual workflow: https://dystroy.org/blog/broot-c-search/ https://dystroy.org/blog/broot-c-search/
philsnow 5y ago
I use skim / `sk` for this kind of task: https://github.com/lotabout/skim https://github.com/lotabout/skim . For example, for iterating on jq incantations, I have in my shell rc file: function jqsk { sk --tac --ansi --regex --query . --multi --interactive --cmd '{}' \ --bind 'enter:select-all+accept,ctrl-y:select-all+execute-silent(for line in {+}; do echo $line; done | pbcopy)+deselect-all' \ --cmd-history=${HOME}/.sk_history --cmd-history-size=100000 \ --no-clear-if-empty \ --cmd-query "cat $1 | jq --raw-output --color-output --exit-status '.'" } I should look into whether fzf or any of the other tools in this thread can be coerced into doing the same thing, the above is pretty verbose and arcane, and it doesn't always behave the way I expect.
hnlmorg 5y ago
If the first grep has "-HRn" flags you then pass the file and line number information down the pipeline. -H File name -R Recursive (directories) -n Line number
nmz 5y ago
you probably want awk or perl. awk '/pattern1/ && /pattern2/ && !/exclude1|exclude2/ {print FILENAME, $0}' */*

totetsu 5y ago

Lnav with filters? or can you stack -e expression arguments onto one grep command?

codemac 5y ago

Are you ok with the initial list existing in memory? Then I highly recommend: emacs / vim buffers

henshao 5y ago

Just posting this without trying it out...grep -nH gives you line numbers and full filepath context, and as another commenter said, -ve allows you to string multiple excludes, so grep -inH <search> -ve not_this -ve nor_this -ve nor_that.

fiddlerwoaroof 5y ago
You don’t have to stack -v like that: -v inverts all the -e expressions, so this should be equivalent: grep -inHv -e foo -e bar -e baz
- fiddlerwoaroof 5y ago
  So, awk sort of works as a fancy grep: ps axjw | awk '(/zsh/ || /login/) && !(/awk/ || /direnv/)' And you can easily add features like "print the first line unconditionally: ps axjw | awk 'NR == 1 {print} (/zsh/ || /login/) && !(/awk/ || /direnv/)' IMO, awk is in a pretty uniquely sweet spot between grep and perl/ruby/python.
  - henshao 5y ago
    Thanks for the grep tip! Unfortunately I've not put in the effort to learn awk past printing the n-th column of ls (my typical use) - the extra syntax required to properly 'quote' and {} things puts me off.

eminence32 5y ago

I glue together ripgrep with fzf and bat like this: rg --line-number --no-heading --color=always --smart-case "$@" | fzf -d ':' -n 2.. --ansi --no-sort --preview-window 'down:20%:+{2}' --preview 'bat --style=numbers --color=always --highlight-line {2} {1}' I get a small preview of the file in the bottom 20% of my terminal, and I can use fzf's matching to further filter, or to exclude things I capture the output and then `cut` it up to get filename and line number, which then get used to open the file/line in vim

anotherhue 5y ago
Thank you for this

montroser 5y ago

If your grep has -P, then you could do all in one go with lookahead assertions... grep -P '(?=.*pattern1)(?=.*pattern2)(?!.*exclude_these)' **/* Maybe not the most beautiful or easy to have at the fingertips, but could be worse.

rektide 5y ago

There's a ripgrep ticket for multiple patterns. No one seems to have come up with a good specification for exactly how it ought to work, so no development has started. https://github.com/BurntSushi/ripgrep/issues/875 https://github.com/BurntSushi/ripgrep/issues/875

LanternLight83 5y ago

https://paweldu.dev/posts/fzf-live-repl/ https://paweldu.dev/posts/fzf-live-repl/ Previous discussion here: https://news.ycombinator.com/item?id=20455857 https://news.ycombinator.com/item?id=20455857 Also featured in that thread: https://github.com/akavel/up https://github.com/akavel/up For example: echo '' | fzf --print-query --preview 'grep -R {q} . 2> /dev/null'

HellsMaddy 5y ago

Not exactly what you’re asking for but similar: I wrote a script called aag, based on ag, that lets you find files that contain multiple matches anywhere in the file. It’s a per-file logical AND of multiple ag searches. https://gist.github.com/b0o/f0759c84af6ef773a37a499fdb1c05af https://gist.github.com/b0o/f0759c84af6ef773a37a499fdb1c05af Example: you’re trying to find a file that contains the strings “MIT License” (case insensitive), “npm install”, and a match for “react-.*” somewhere in your home directory: aag ~ ---- -i "mit license" --- "npm install" --- "react-.*" The excessive dashes are necessary so that it’s possible to pass different options to each separate invocation of ag. In case it’s not clear why this is useful, normally ag (or grep) searches are linewise. It’s not so easy if you are looking for things that occur on different lines, and possibly in different orders.

tyingq 5y ago

Perl would be one option: perl -ne '( /foo/ and /bar/ and !/baz/ and !/splat/ ) and print "$ARGV: $_"' * I didn't include the context part, since that would be more than a one-liner, but it's not hard to do in a short script.

omaranto 5y ago

I do things like that inside Emacs with the consult-grep command from the Consult package, combined with the Orderless matching style, Embark to collect the results in a buffer. This has several advantages over the command line: - Interactivity: the results are updated live as you type, so you catch typos sooner and can tweak the search terms as you go. - The buffer of search results you collect with Embark is not dead text like it would be in the terminal, instead, each line is a link to the corresponding file taking you to the line that matched. - Wgrep lets you edit all the matching lines in place! Packages referenced: 1. Consult: https://github.com/minad/consult/ https://github.com/minad/consult/ 2. Orderless: https://github.com/oantolin/orderless https://github.com/oantolin/orderless 3. Embark: https://github.com/oantolin/embark https://github.com/oantolin/embark 4. Wgrep: http://github.com/mhayashi1120/Emacs-wgrep/raw/master/wgrep.el http://github.com/mhayashi1120/Emacs-wgrep/raw/master/wgrep....

dystroy 5y ago

You can combine patterns on file names and/or file content with broot. Here's my usual workflow: https://dystroy.org/blog/broot-c-search/ https://dystroy.org/blog/broot-c-search/

philsnow 5y ago

I use skim / `sk` for this kind of task: https://github.com/lotabout/skim https://github.com/lotabout/skim . For example, for iterating on jq incantations, I have in my shell rc file: function jqsk { sk --tac --ansi --regex --query . --multi --interactive --cmd '{}' \ --bind 'enter:select-all+accept,ctrl-y:select-all+execute-silent(for line in {+}; do echo $line; done | pbcopy)+deselect-all' \ --cmd-history=${HOME}/.sk_history --cmd-history-size=100000 \ --no-clear-if-empty \ --cmd-query "cat $1 | jq --raw-output --color-output --exit-status '.'" } I should look into whether fzf or any of the other tools in this thread can be coerced into doing the same thing, the above is pretty verbose and arcane, and it doesn't always behave the way I expect.

hnlmorg 5y ago

If the first grep has "-HRn" flags you then pass the file and line number information down the pipeline. -H File name -R Recursive (directories) -n Line number

nmz 5y ago

you probably want awk or perl. awk '/pattern1/ && /pattern2/ && !/exclude1|exclude2/ {print FILENAME, $0}' */*