5 ms·

Grep is often better suited for extracting matched portion, especially if you also have PCRE option: grep -oP -m1 'name:\s*"\K[^"]+'

by asicsp 4y ago

olsgaarddk 4y ago
I thought I was pretty good at regex, but I could never have written this one and had to consult both `man grep` and regex101.com. Explanations for the beginner and intermediate regex and grep user: `-o`: Only return the match, instead of the entire line `-P`: use Perl compatible regex `-m` max-count, Stop reading a file after NUM matching lines. And now for the regex: `name:`: find the exact match `\s*"`: Zero or more spaces leading up to and including an double quote `\K`: This was the kicker for me. "resets the starting point of the reported match. Any previously consumed characters are no longer included in the final match" - basically tells the regex engine that the characters _before_ `\K` needs to be there in order to form a match, but it should only return the characters _after_ `\K` as the match. This is super handy! Is there a "reverse \K"? `[^"]+`: One or more characters that are not a double quote. This basically means "Find the line that has a key called "name" and return all the characters after the first double quote and until the last double quote"
- asicsp 4y ago
  If you'd like to learn more about such grep powers, check out my free ebook [0] What do you mean by "reverse \K"? Are you aware of lookarounds? Perhaps you meant positive lookahead? # match digits only if there is a semicolon afterwards $ echo '12; 42,31;100' | grep -oP '\d+(?=;)' 12 31 [0] https://learnbyexample.github.io/learn_gnugrep_ripgrep/introduction.html https://learnbyexample.github.io/learn_gnugrep_ripgrep/intro...
  - mxmkm 4y ago
    In vim there is `\zs` and `\ze` where `\zs` is the `\K` equivalent in grep. Basically :%s/hello \zsworld\ze out there/planet/g would find all `hello world out there` and replace `world` to `planet`.
  - Joker_vD 4y ago
    Consider this: grep -P 'start: (\d+) end' "How do I make it print only the captured group with the number, not the whole line?" is a pretty common Stack Overflow question. The "\K" thing gets rid of the "start: " part, but what about " end"? That's were "reverse \K" would come in handy.
    - asicsp 4y ago
      That's where lookarounds help: grep -oP 'start: \K\d+(?= end)' `\K` is kinda similar to lookbehind (but not exactly same as it is not zero-width), and particularly helpful for variable length patterns. If you need to process further, you can make use of `-r` option in `ripgrep` or move to other tools like sed, awk, perl, etc.
    - burntsushi 4y ago
      That's where ripgrep's -r/--replace flag comes in handy: $ echo 'foobar start: 123 end quuxbar' | rg 'start: ([0-9]+) end' foobar start: 123 end quuxbar $ echo 'foobar start: 123 end quuxbar' | rg 'start: ([0-9]+) end' -r '$1' foobar 123 quuxbar $ echo 'foobar start: 123 end quuxbar' | rg 'start: ([0-9]+) end' -or '$1' 123
- Someone 4y ago
  > Is there a "reverse \K"? Look up lookahead and lookbehind (https://www.regular-expressions.info/lookaround.html https://www.regular-expressions.info/lookaround.html, https://perldoc.perl.org/perlre#Extended-Patterns https://perldoc.perl.org/perlre#Extended-Patterns)
- darrenf 4y ago
  > all the characters after the first double quote and until the last double quote Until the next double quote, not necessarily the last one.
1vuio0pswjnm7 4y ago
Why doesn't POSIX include the "-o" option. The word "often" in the parent comment implies there are times when GNU grep is not better suited than sed with ERE. (I often use flex instead of sed.) For example, https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44754 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44754 Or from the 3.8 manual: 6.1 Known Bugs Large repetition counts in the `{n,m}' construct may cause grep to use lots of memory. In addition, certain other obscure regular expressions require exponential time and space, and may cause grep to run out of memory. Back-references can greatly slow down matching, as they can generate exponentially many matching possibilities that can consume both time and memory to explore. Also, the POSIX specification for back-references is at times unclear. Furthermore, many regular expression implementations have back-reference bugs that can cause programs to return incorrect answers or even crash, and fixing these bugs has often been low-priority: for example, as of 2021 the GNU C library bug database contained back-reference bugs 52, 10844, 11053, 24269 and 25322, with little sign of forthcoming fixes. Luckily, back-references are rarely useful and it should be little trouble to avoid them in practical applications.
- asicsp 4y ago
  >there are times when GNU grep is not better suited than sed with ERE Because, among other things, sed has filtering features like based on line numbers, address range, etc that aren't present in grep. Regarding known bugs, that probably extends to GNU sed/awk as well. An issue I filed (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26864 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26864) led to the manual update about back-reference bugs.
  - deleted 4y ago
    [deleted]

Grep is often better suited for extracting matched portion, especially if you also have PCRE option: grep -oP -m1 'name:\s*"\K[^"]+'

by asicsp 4y ago

Grep is often better suited for extracting matched portion, especially if you also have PCRE option: grep -oP -m1 'name:\s*"\K[^"]+'

olsgaarddk 4y ago

I thought I was pretty good at regex, but I could never have written this one and had to consult both `man grep` and regex101.com. Explanations for the beginner and intermediate regex and grep user: `-o`: Only return the match, instead of the entire line `-P`: use Perl compatible regex `-m` max-count, Stop reading a file after NUM matching lines. And now for the regex: `name:`: find the exact match `\s*"`: Zero or more spaces leading up to and including an double quote `\K`: This was the kicker for me. "resets the starting point of the reported match. Any previously consumed characters are no longer included in the final match" - basically tells the regex engine that the characters _before_ `\K` needs to be there in order to form a match, but it should only return the characters _after_ `\K` as the match. This is super handy! Is there a "reverse \K"? `[^"]+`: One or more characters that are not a double quote. This basically means "Find the line that has a key called "name" and return all the characters after the first double quote and until the last double quote"

asicsp 4y ago
If you'd like to learn more about such grep powers, check out my free ebook [0] What do you mean by "reverse \K"? Are you aware of lookarounds? Perhaps you meant positive lookahead? # match digits only if there is a semicolon afterwards $ echo '12; 42,31;100' | grep -oP '\d+(?=;)' 12 31 [0] https://learnbyexample.github.io/learn_gnugrep_ripgrep/introduction.html https://learnbyexample.github.io/learn_gnugrep_ripgrep/intro...
- mxmkm 4y ago
  In vim there is `\zs` and `\ze` where `\zs` is the `\K` equivalent in grep. Basically :%s/hello \zsworld\ze out there/planet/g would find all `hello world out there` and replace `world` to `planet`.
- Joker_vD 4y ago
  Consider this: grep -P 'start: (\d+) end' "How do I make it print only the captured group with the number, not the whole line?" is a pretty common Stack Overflow question. The "\K" thing gets rid of the "start: " part, but what about " end"? That's were "reverse \K" would come in handy.
  - asicsp 4y ago
    That's where lookarounds help: grep -oP 'start: \K\d+(?= end)' `\K` is kinda similar to lookbehind (but not exactly same as it is not zero-width), and particularly helpful for variable length patterns. If you need to process further, you can make use of `-r` option in `ripgrep` or move to other tools like sed, awk, perl, etc.
  - burntsushi 4y ago
    That's where ripgrep's -r/--replace flag comes in handy: $ echo 'foobar start: 123 end quuxbar' | rg 'start: ([0-9]+) end' foobar start: 123 end quuxbar $ echo 'foobar start: 123 end quuxbar' | rg 'start: ([0-9]+) end' -r '$1' foobar 123 quuxbar $ echo 'foobar start: 123 end quuxbar' | rg 'start: ([0-9]+) end' -or '$1' 123
Someone 4y ago
> Is there a "reverse \K"? Look up lookahead and lookbehind (https://www.regular-expressions.info/lookaround.html https://www.regular-expressions.info/lookaround.html, https://perldoc.perl.org/perlre#Extended-Patterns https://perldoc.perl.org/perlre#Extended-Patterns)
darrenf 4y ago
> all the characters after the first double quote and until the last double quote Until the next double quote, not necessarily the last one.

1vuio0pswjnm7 4y ago

Why doesn't POSIX include the "-o" option. The word "often" in the parent comment implies there are times when GNU grep is not better suited than sed with ERE. (I often use flex instead of sed.) For example, https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44754 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44754 Or from the 3.8 manual: 6.1 Known Bugs Large repetition counts in the `{n,m}' construct may cause grep to use lots of memory. In addition, certain other obscure regular expressions require exponential time and space, and may cause grep to run out of memory. Back-references can greatly slow down matching, as they can generate exponentially many matching possibilities that can consume both time and memory to explore. Also, the POSIX specification for back-references is at times unclear. Furthermore, many regular expression implementations have back-reference bugs that can cause programs to return incorrect answers or even crash, and fixing these bugs has often been low-priority: for example, as of 2021 the GNU C library bug database contained back-reference bugs 52, 10844, 11053, 24269 and 25322, with little sign of forthcoming fixes. Luckily, back-references are rarely useful and it should be little trouble to avoid them in practical applications.

asicsp 4y ago
>there are times when GNU grep is not better suited than sed with ERE Because, among other things, sed has filtering features like based on line numbers, address range, etc that aren't present in grep. Regarding known bugs, that probably extends to GNU sed/awk as well. An issue I filed (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26864 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26864) led to the manual update about back-reference bugs.
- deleted 4y ago
  [deleted]