3 ms·

Useful one-line scripts for sed (2005)

52 points by kawera 9y ago

baldfat 9y ago
I don't know what wins for most under used tool, sed or awk? Both of these do wonders for clearing up messy data and for a wealth of other things.
gwaefawfewaef 9y ago
This guy is very interesting. pement.org http://pement.org/perl/biblink126_pl.txt http://pement.org/perl/biblink126_pl.txt
- bpchaps 9y ago
  Definitely. I heard about him (10 years ago) from one of his old bosses who mentioned this page. This sed one liner page took about a month to go through from basically zero Unix experience. Probably the most important learning piece I've ever gone through. Good stuff.
zurn 9y ago
Widely useful pick of the bunch: # print section of file between two regular expressions (inclusive) sed -n '/Iowa/,/Montana/p' # case sensitive
- predakanga 9y ago
  I've found that pattern to be very useful when altering/sanitising MySQL backups. # Set the engine of all tables in schema `foo` to TokuDB sed -re '/^USE `foo`/,/^USE / s/^\) ENGINE=\S+/) ENGINE=TokuDB/'
- majewsky 9y ago
  I use stuff like that all the time with config files where sections span multiple lines. For example, if ~/.ssh/config has a section like Host example HostName example.com IdentityFile ~/.ssh/id_example Port 1234 Host next ... Then the following selects the identity file: sed -n '/^Host example/,/^Host\>/{/^\s*IdentityFile/{s/^\s*IdentityFile\s*//;p}}' ~/.ssh/config Although I would probably use awk after sed because it handled space-separated fields nicer: sed -n '/^Host example/,/^Host\>/p' ~/.ssh/config | awk '$1=="IdentityFile"{print$2}'
801699 9y ago
Here is something to use as a replacement for "grep -q". In the past I found grep -q was not portable. There are times when I do not have grep. But I always have sed. cat grepq test $# -ge 1|| exec echo usage: $0 PATTERN \[FILE\] # count lines until PATTERN __=$(exec sed -n '/'"$1"'/!d;=;q' $2); # no of lines exec test ${#__} -gt 0; For example, grepq '93.184.216.34 example.com' /etc/hosts|| echo 93.184.216.34 example.com >> /etc/hosts One of the very early OReilly books has a chapter or two about sed that tries to describe it using an analogy to a scrivener in a monastary. Quite amusing. This is not the "sed and awk" book. It was an earlier book on text editing with UNIX. It may have been co-authored by OReilly himself; I cannot remember.

52 points by kawera 9y ago

I don't know what wins for most under used tool, sed or awk? Both of these do wonders for clearing up messy data and for a wealth of other things.

gwaefawfewaef 9y ago

This guy is very interesting. pement.org http://pement.org/perl/biblink126_pl.txt http://pement.org/perl/biblink126_pl.txt

bpchaps 9y ago
Definitely. I heard about him (10 years ago) from one of his old bosses who mentioned this page. This sed one liner page took about a month to go through from basically zero Unix experience. Probably the most important learning piece I've ever gone through. Good stuff.

zurn 9y ago

Widely useful pick of the bunch: # print section of file between two regular expressions (inclusive) sed -n '/Iowa/,/Montana/p' # case sensitive

predakanga 9y ago
I've found that pattern to be very useful when altering/sanitising MySQL backups. # Set the engine of all tables in schema `foo` to TokuDB sed -re '/^USE `foo`/,/^USE / s/^\) ENGINE=\S+/) ENGINE=TokuDB/'
majewsky 9y ago
I use stuff like that all the time with config files where sections span multiple lines. For example, if ~/.ssh/config has a section like Host example HostName example.com IdentityFile ~/.ssh/id_example Port 1234 Host next ... Then the following selects the identity file: sed -n '/^Host example/,/^Host\>/{/^\s*IdentityFile/{s/^\s*IdentityFile\s*//;p}}' ~/.ssh/config Although I would probably use awk after sed because it handled space-separated fields nicer: sed -n '/^Host example/,/^Host\>/p' ~/.ssh/config | awk '$1=="IdentityFile"{print$2}'

801699 9y ago

Here is something to use as a replacement for "grep -q". In the past I found grep -q was not portable. There are times when I do not have grep. But I always have sed. cat grepq test $# -ge 1|| exec echo usage: $0 PATTERN \[FILE\] # count lines until PATTERN __=$(exec sed -n '/'"$1"'/!d;=;q' $2); # no of lines exec test ${#__} -gt 0; For example, grepq '93.184.216.34 example.com' /etc/hosts|| echo 93.184.216.34 example.com >> /etc/hosts One of the very early OReilly books has a chapter or two about sed that tries to describe it using an analogy to a scrivener in a monastary. Quite amusing. This is not the "sed and awk" book. It was an earlier book on text editing with UNIX. It may have been co-authored by OReilly himself; I cannot remember.