4 ms·

I use this command very frequently to check how often an event occurs in a log file over time (specifically in 10-minute buckets), assuming the file is formatti

I use this command very frequently to check how often an event occurs in a log file over time (specifically in 10-minute buckets), assuming the file is formatting like "INFO - [2014-08-27 16:16:29,578] Something something something" cat /path/to/logfile | grep PATTERN | sed 's/.*\(2014-..-..\) \(..\):\(.\).*/\1 \2:\3x/' | uniq -c results in: 273 2014-08-27 14:5x 222 2014-08-27 15:0x 201 2014-08-27 15:1x 171 2014-08-27 15:2x 349 2014-08-27 15:3x 230 2014-08-27 15:4x 236 2014-08-27 15:5x 339 2014-08-27 16:0x 330 2014-08-27 16:1x This can subsequently be visualized with a tool like gnuplot or Excel.

mprovost 12y ago
Useless use of cat?
- bch 12y ago
  "Don't pipe a cat" is how I'm used to describing what you're talking about -- it may have been a performance issue in days past, but these days I think it's simply a matter of style. Not that style is not important.
  - mprovost 12y ago
    This was drilled into me back in the usenet days. If you see a cat command with a single argument it's almost always replaceable by a shell redirection, or in this case just by passing the filename as an argument to grep. If you're processing lots of data like in the article there's no point in passing it through a separate command and pipe first.
    - smorrow 12y ago
      I think people like reading cat thefile | grepsedawk -opts 'prog' from left to right, and that they think the only alternative is grepsedawk -opts 'prog' thefile. But there's grep <thefile -opts 're'. I like that one best; it reads the same way you'd tend to think it.
    - clarry 12y ago
      But when you're interactively creating your pipeline, having the cat at the very beginning can save you some shuffling around. For example, while your second command is be grep and you're passing your file directly to grep, you might then realize you need another command before grep. So you'll have to move that argument to the other command. With a useless cat in front, you just insert the new command between cat and grep. It doesn't really cause harm.
smorrow 12y ago
sed -E for awk or grep -E regexes.

mprovost 12y ago

Useless use of cat?

bch 12y ago
"Don't pipe a cat" is how I'm used to describing what you're talking about -- it may have been a performance issue in days past, but these days I think it's simply a matter of style. Not that style is not important.
- mprovost 12y ago
  This was drilled into me back in the usenet days. If you see a cat command with a single argument it's almost always replaceable by a shell redirection, or in this case just by passing the filename as an argument to grep. If you're processing lots of data like in the article there's no point in passing it through a separate command and pipe first.
  - smorrow 12y ago
    I think people like reading cat thefile | grepsedawk -opts 'prog' from left to right, and that they think the only alternative is grepsedawk -opts 'prog' thefile. But there's grep <thefile -opts 're'. I like that one best; it reads the same way you'd tend to think it.
  - clarry 12y ago
    But when you're interactively creating your pipeline, having the cat at the very beginning can save you some shuffling around. For example, while your second command is be grep and you're passing your file directly to grep, you might then realize you need another command before grep. So you'll have to move that argument to the other command. With a useless cat in front, you just insert the new command between cat and grep. It doesn't really cause harm.

smorrow 12y ago

sed -E for awk or grep -E regexes.