23 ms·

Using different delimiters in sed (2010)

59 points by navigaid 8y ago

giobox 8y ago
I’ve seen so many hilariously bad workarounds for this in old shell scripts from lazy engineers who apparently can’t read a man page and don’t realise the delimiter character is substitutable with more or less any character you like. This is one of this extremely useful features that a surprisingly large number of users don’t seem to know about.
- michaelcampbell 8y ago
  Indeed. `/` covers 99% of my cases, `|` covers 99% of the remaining. I think i might have had to use something "exotic" once or twice in 30+ years now.
SubiculumCode 8y ago
For one offs, I like to change up my delimiter so that I a) remind myslef I can and b)think the slashes are ugly dont like escaping. I prefer pipes.
keyle 8y ago
Well, I have some bash scripts to improve... great post, another useful fact on HN.
teddyh 8y ago
The common alternate character, for those cases where normal slashes are not suitable, is a comma character; ‘,’. With Sed code, readability is always an issue, and it can be improved by not using just any character and limiting yourself to as few alternative modes of syntax as possible. With a bit of exposure, commands like s/foo/bar/ and s,foo,bar, can be read with equal ease.
- cyphar 8y ago
  > The common alternate character, for those cases where normal slashes are not suitable, is a comma character; ‘,’. I've always used "|" since it's so visually similar, and you always want to quote regular expressions anyway.
  - teddyh 8y ago
    Yes, I’ve seen the vertical bar character used for this too. But in my experience the comma character is more commonly used, and my point was that we should all try to use as commonly used syntax as possible. Therefore I encourage everybody to use the comma character as their usual alternative character in order for the world of Sed code to be slightly less confusing.
    - ajross 8y ago
      My experience, colored more by perl styles than sed per se, is that the vertical bar is the overwhelmingly typical usage, actually. It's what Wall picked for the examples in the camel book, for example.
    - laumars 8y ago
      My experience has been that there isn’t a de facto standard outside of /. Arguing that people must use some standard you’ve adopted is a little like saying people should use tabs instead of spaces.
  - yiyus 8y ago
    Using "|" can be confusing when the sed command is part of a pipeline. I agree with the other user that the comma character is more used. It is slightly easier to type too.
  - Someone 8y ago
    Doesn’t that confuse you because it is the alternation symbol (https://regular-expressions.mobi/alternation.html https://regular-expressions.mobi/alternation.html) in many, if not all, regular expression syntaxes?
- falsedan 8y ago
  I prefer @ because it is so weird and unexpected people are unlikely to misread the command and think it does something with multiple commands tho perl’s qr (quote regex) operator is miles ahead, it even supports natural pairs of opening/closing delimiters
  - laumars 8y ago
    I really like doing the following in Perl: s{/u/r/i}{do something} Sadly doesn’t support that (or at least none of the sed’s I’ve used). So I fallback to using hash (#) instead (for no particular reason aside that’s what I’ve always used).
- agumonkey 8y ago
  s,,,g is my favorite because it avoid /\//\\/ epilespy
- jolmg 8y ago
  I tend to use : as the alternate delimiters.
chx 8y ago
The address one is really useful I wish I knew it this very week. Thanks!
devy 8y ago
On a related note, I found Bruce Barnett's sed tutorial is one of the best Sed guide out there: http://www.grymoire.com/Unix/Sed.html http://www.grymoire.com/Unix/Sed.html
- mdaniel 8y ago
  It was really good, thank you! I learned a ton of new tricks from it Separately, I feel like I could have a full time job submitting PRs to fix the GNU-ism of using `sed -i"" ...` versus the BSD/macOS syntax of `sed -i "" ...`. I don't know which of those two camps broke from the other, but holy hell it _broke_ for sure.
  - thisacctforreal 8y ago
    How does the GNU-ism work at a shell level? In your first example $0 is "sed", $1 is "-i", $2 is "...", no? What signifigance does the -i"" have in bash apart from 'concat -i to ""'? The BSD syntax is plain, $1 is the option "-i", $2 is the value for that option: "", $3 is "...". Edit: off by one
    - mdaniel 8y ago
      I don't believe it's a bash-ism, I would guess it's GNU getopt defining the `-i` as only optionally requiring an argument, versus the built-in getopt that BSD is using: https://git.savannah.gnu.org/cgit/sed.git/tree/sed/sed.c?h=v4.7#n196 https://git.savannah.gnu.org/cgit/sed.git/tree/sed/sed.c?h=v... https://github.com/freebsd/freebsd/blob/release/7.0.0/usr.bin/sed/main.c#L133 https://github.com/freebsd/freebsd/blob/release/7.0.0/usr.bi... So, while your interpretation of the shell splitting is correct, it's the different getopt declarations that cause this pain
    - mmt 8y ago
      Indeed, appending an empty string to -i doesn't modify it in any way, AFAIK. It's that the GNU version doesn't require an argument, but, if an argument is to be provided, it must be done as part of the flag, not as a separate element of argv. The MacOS version allows either way of providing an argument, so -i.orig tends to be portable (assuming the -i flag is supported in the first place). Differences in how "traditional" (be they sysv or bsd) versus gnu utilities handle flags and arguments [1] is very well rooted in history, and is hardly unique to sed. I suspect the main reason this has been forgotten is that Linux, which ships with gnu utilities, has been so dominant for so long, though, even before that, it was difficult to find a then-current unix on which gnu utilities couldn't be installed. [1] As the sibling comment points out, the source of difference is getopt, of which there were more than just two versions.. including not even using a library
finnh 8y ago
tilda has always been my go-to; in fact i tend to use it in preference to / even if there is no literal / in my statement. s~foo~bar~ I still use / when pointing out typos in PRs, tho :) s/calulate/calculate/

59 points by navigaid 8y ago

I’ve seen so many hilariously bad workarounds for this in old shell scripts from lazy engineers who apparently can’t read a man page and don’t realise the delimiter character is substitutable with more or less any character you like. This is one of this extremely useful features that a surprisingly large number of users don’t seem to know about.

michaelcampbell 8y ago
Indeed. `/` covers 99% of my cases, `|` covers 99% of the remaining. I think i might have had to use something "exotic" once or twice in 30+ years now.

SubiculumCode 8y ago

For one offs, I like to change up my delimiter so that I a) remind myslef I can and b)think the slashes are ugly dont like escaping. I prefer pipes.

keyle 8y ago

Well, I have some bash scripts to improve... great post, another useful fact on HN.

teddyh 8y ago

The common alternate character, for those cases where normal slashes are not suitable, is a comma character; ‘,’. With Sed code, readability is always an issue, and it can be improved by not using just any character and limiting yourself to as few alternative modes of syntax as possible. With a bit of exposure, commands like s/foo/bar/ and s,foo,bar, can be read with equal ease.

cyphar 8y ago
> The common alternate character, for those cases where normal slashes are not suitable, is a comma character; ‘,’. I've always used "|" since it's so visually similar, and you always want to quote regular expressions anyway.
- teddyh 8y ago
  Yes, I’ve seen the vertical bar character used for this too. But in my experience the comma character is more commonly used, and my point was that we should all try to use as commonly used syntax as possible. Therefore I encourage everybody to use the comma character as their usual alternative character in order for the world of Sed code to be slightly less confusing.
  - ajross 8y ago
    My experience, colored more by perl styles than sed per se, is that the vertical bar is the overwhelmingly typical usage, actually. It's what Wall picked for the examples in the camel book, for example.
  - laumars 8y ago
    My experience has been that there isn’t a de facto standard outside of /. Arguing that people must use some standard you’ve adopted is a little like saying people should use tabs instead of spaces.
- yiyus 8y ago
  Using "|" can be confusing when the sed command is part of a pipeline. I agree with the other user that the comma character is more used. It is slightly easier to type too.
- Someone 8y ago
  Doesn’t that confuse you because it is the alternation symbol (https://regular-expressions.mobi/alternation.html https://regular-expressions.mobi/alternation.html) in many, if not all, regular expression syntaxes?
falsedan 8y ago
I prefer @ because it is so weird and unexpected people are unlikely to misread the command and think it does something with multiple commands tho perl’s qr (quote regex) operator is miles ahead, it even supports natural pairs of opening/closing delimiters
- laumars 8y ago
  I really like doing the following in Perl: s{/u/r/i}{do something} Sadly doesn’t support that (or at least none of the sed’s I’ve used). So I fallback to using hash (#) instead (for no particular reason aside that’s what I’ve always used).
agumonkey 8y ago
s,,,g is my favorite because it avoid /\//\\/ epilespy
jolmg 8y ago
I tend to use : as the alternate delimiters.

chx 8y ago

The address one is really useful I wish I knew it this very week. Thanks!

devy 8y ago

On a related note, I found Bruce Barnett's sed tutorial is one of the best Sed guide out there: http://www.grymoire.com/Unix/Sed.html http://www.grymoire.com/Unix/Sed.html

mdaniel 8y ago
It was really good, thank you! I learned a ton of new tricks from it Separately, I feel like I could have a full time job submitting PRs to fix the GNU-ism of using `sed -i"" ...` versus the BSD/macOS syntax of `sed -i "" ...`. I don't know which of those two camps broke from the other, but holy hell it _broke_ for sure.
- thisacctforreal 8y ago
  How does the GNU-ism work at a shell level? In your first example $0 is "sed", $1 is "-i", $2 is "...", no? What signifigance does the -i"" have in bash apart from 'concat -i to ""'? The BSD syntax is plain, $1 is the option "-i", $2 is the value for that option: "", $3 is "...". Edit: off by one
  - mdaniel 8y ago
    I don't believe it's a bash-ism, I would guess it's GNU getopt defining the `-i` as only optionally requiring an argument, versus the built-in getopt that BSD is using: https://git.savannah.gnu.org/cgit/sed.git/tree/sed/sed.c?h=v4.7#n196 https://git.savannah.gnu.org/cgit/sed.git/tree/sed/sed.c?h=v... https://github.com/freebsd/freebsd/blob/release/7.0.0/usr.bin/sed/main.c#L133 https://github.com/freebsd/freebsd/blob/release/7.0.0/usr.bi... So, while your interpretation of the shell splitting is correct, it's the different getopt declarations that cause this pain
  - mmt 8y ago
    Indeed, appending an empty string to -i doesn't modify it in any way, AFAIK. It's that the GNU version doesn't require an argument, but, if an argument is to be provided, it must be done as part of the flag, not as a separate element of argv. The MacOS version allows either way of providing an argument, so -i.orig tends to be portable (assuming the -i flag is supported in the first place). Differences in how "traditional" (be they sysv or bsd) versus gnu utilities handle flags and arguments [1] is very well rooted in history, and is hardly unique to sed. I suspect the main reason this has been forgotten is that Linux, which ships with gnu utilities, has been so dominant for so long, though, even before that, it was difficult to find a then-current unix on which gnu utilities couldn't be installed. [1] As the sibling comment points out, the source of difference is getopt, of which there were more than just two versions.. including not even using a library

finnh 8y ago

tilda has always been my go-to; in fact i tend to use it in preference to / even if there is no literal / in my statement. s~foo~bar~ I still use / when pointing out typos in PRs, tho :) s/calulate/calculate/