25 ms·
Escaping strings in Bash using !:q
- ctur 6y agoEven better -- use this script, paste anything into stdin. It will come back out quotable and pasteable into a shell. #!/bin/bash printf '%q' "$(cat)" echo
- Tepix 6y agoEven better, put this in your ~/.bashrc function bashquote () { printf '%q' "$(cat)" echo }
- asicsp 6y agoGolfed one-liner bashquote() { printf '%q\n' "$(cat)" ; }
- KMnO4 6y agoYou should contact your administrator if you are still paying per-line for your .bashrc
- dotancohen 6y agoI think he's just saving on keystokes because those Model M's are getting hard to find.
- Tepix 6y agoCan't we get rid of cat somehow?
- kps 6y agoYes, bashquote() { printf '%q\n' "$(</dev/stdin)"; } Bash treats the string "/dev/stdin" as magic, so that works even if /dev/stdin doesn't exist. However, bash (unlike ksh) spawns a subshell for $(</dev/stdin), so using cat is actually lighter. (Also, it's not clear to me why $(<&0) doesn't work in bash.)
- klysm 6y agoIs the echo there just for the newline? Why not slap a \n in there and make it a one liner?
- twic 6y ago$ echo "text with spaces" | bash -c 'printf "%q" "$(cat)"'; echo text\ with\ spaces For me, this escapes spaces with backslashes; the example in the article escapes them by quoting the whole string. Is this a difference in bash versions, or do these uses differ somehow? (I replaced the single quotes with double so i could pass this to bash -c, but i get the same result if i use your code verbatim as a script)
- asicsp 6y agotext\ with\ spaces is same as 'text with spaces' See https://mywiki.wooledge.org/Quotes https://mywiki.wooledge.org/Quotes
- twic 6y agoIt's plainly not the same, you can see that by looking at it. It encodes the same text, but it encodes it a different way. And the difference is not negligible. With quotes, i can type or paste more into the middle, and the string is still valid. With escapes, i have to be careful to escape the added text. It's significantly more ergonomic to use quoted text.
- eMSF 6y agoIf you intend to type or paste quotes, you still need some additional escaping. And on the other hand, most substrings of (non-quoted) escaped text are themselves properly escaped.
- kps 6y agoksh's %q prefers quotes; for your example it prints 'text with spaces', and for the article's example, $'This string \'has single\' "and double" quotes and a $' vs bash's %q, This\ string\ \'has\ single\'\ \"and\ double\"\ quotes\ and\ a\ \$
- zeroimpl 6y agoYes the example in the article and parent comments ought to be referred to as escaping, not quoting. Most of the time, quoting with single quotes leads to something far easier on the eyes. Here's an example function for quoting from stdin: function bashquotesingle() { printf "'"; sed "s/'/'\\\\''/g"; printf "'"; } Though it does have a bug, at least on macOS, where sed adds a trailing newline. (See https://stackoverflow.com/questions/13325138/why-does-sed-add-a-new-line-in-osx https://stackoverflow.com/questions/13325138/why-does-sed-ad...)
- birdyrooster 6y agoOr on Mac, just run this script and it will modify your clipboard #!/bin/bash printf '%q' "$(pbpaste)" | pbcopy
- anvandare 6y agoCool trick! Adding an additional :p modifier prevents the complaint about command not found: $ # This string 'has single' "and double" quotes and a $ $ !:q '# This string '\''has single'\'' "and double" quotes and a $' # This string 'has single' "and double" quotes and a $: command not found $ $ # This string 'has single' "and double" quotes and a $ $ !:q:p '# This string '\''has single'\'' "and double" quotes and a $' $
- mkl 6y ago!:q and then pressing M-^ (probably Alt-Shift-6) to do the history expansion is probably more useful; then you can immediately edit the line.
- ivanhoe 6y agoOr one can use `bind space:magic-space` in .bashrc to activate the auto-expansion of the history on space
- andrewaylett 6y agoOr `echo !:q`
- adrianmonk 6y agoI'm not at a computer right now, but this also seems like it should work: : !:q The appeal (to me) would be less clutter in the output.
- emmelaich 6y agoNote that you can precede the !:q with `echo` to see the result as well as the invocation without an error. $ echo !:q echo '# This string '\''has single'\'' "and double" quotes and a $' # This string 'has single' "and double" quotes and a $
- eMSF 6y agoSeeing the result of a modified history expansion is (or was) kind of a desired feature, so it's built-in just like the ability to escape stuff with :q $ # here be quote ' " chars (use :p to print but not execute) $ :!q:p '# here be quote '\'' " chars' (use :s to modify – no :q here because we modify the already quoted text !!, unquoted is now !-2) $ !:s/be/are/:p '# here are quote '\'' " chars'
- slobiwan 6y agoThis is very useful for saving off a one-liner .sh file once you've finally gotten the one that you want to work. I will have to remember this! echo !:q > doagain.sh
- tus88 6y agoTry doing it with :#s///
- samtheprogram 6y agoCould someone explain why this was downvoted? Is this dangerous, or does it not perform the same operation?
- mkl 6y agoIt is not dangerous, and does not perform the same operation (it is actually incorrect and incomplete: ":" and "#" swapped, and missing substitution words). I guess it was downvoted because it didn't explain? The bash manual does, though, under "HISTORY EXPANSION". Try this: Type a command, but don't press enter, e.g. echo 'hello'; Type !#:s/hello/world/ Press enter. !# means the whole current command so far, and the s/a/b/ modifier replaces the first instance of "a" with "b". Press up to get the command that ran: echo 'hello'; echo 'world'; To see such history expansion things before they are run, press M-^ (probably Alt-Shift-6).
- zeepzeep 6y agoOh I thought like, end a string without a quote...
- kbenson 6y agoIf only more languages supported quote operators like Perl. They really do make certain things like this so much easier. Non interpolating quotes: q/I wasn't surprised when he said "boo" and game me $5/; q@I wasn't surprised when he said "boo" and game me $5@; q!I wasn't surprised when he said "boo" and game me $5!; q(I wasn't surprised when he said "boo" and game me $5); q[I wasn't surprised when he said "boo" and game me $5]; Interpolating quotes: qq/Hello, $name!/; qq!Hello, $name\!!; qq@Hello, $name!@; qq(Hello, $name!); qq[Hello, $name!]; qq'Hello, $name!'; Rule of thumb, single q for single quote string (non-interpolating normally), and two q's for a double quote string (normally interpolates).
- twic 6y agoLifted directly from sed, one of Perl's inspirations: s/replace #/with $/ s#replace /#with _#
- mycl 6y agoI think it was in ed already. The POSIX ed spec says: "Any character other than <space> or <newline> can be used instead of a slash to delimit the RE and the replacement."
- codesnik 6y agoand, AFAIR, in perl space and enter could be delimiters too! I think I've used it in some perlgolf task which didn't count whitespace.
- k_bx 6y agoOne thing I've figured is that in bash you can use $'These kinds of strings', without any variable expansion, but what you get is essentially what's present in most programming languages quote-wise. Example: $ echo $'hey there, it\'s "double quotes", \'single quotes\', and some \\, \', ", $ chars' hey there, it's "double quotes", 'single quotes', and some \, ', ", $ chars
- mkl 6y agoThose strings also support things like \0 to get a null byte, and \uxxxx to get a Unicode character. This is useful for working with filenames and other things with spaces, quotes, and so on using find, xargs, etc. E.g. find ... -print0 | while read -d $'\0' f; do ...; done
- bewuethr 6y ago"read" also supports the empty string to mean the null byte in this case: find ... -print0 | while read -d '' f; do ...; done And since Bash 4.4, also "readarray"/"mapfile" can specify a delimiter, which is great to read files safely into an array: readarray -d '' -a arr < <(find ... -print0)
- layoutIfNeeded 6y ago>Those strings also support things like \0 to get a null byte WARNING: this is not true in bash! You can have exactly one null byte in a bash string: the terminating null byte. Try this: echo $’foo\0bar’ It prints “foo”. So practically you can’t have null bytes in bash strings, as it will be mistaken for the terminating null of the underlying C string. In your example read -d ‘’ would work just the same; actually that’s the idiomatic way to iterate on zero-delimited input (or xargs -0). Why does the empty string work? Because -d takes the first char of the string as the delimiter, which for empty C strings is the terminating \0 - this is how bugs become features.
- twic 6y agoI turned off histexpand because i was fed up with exclamation marks doing random things i never wanted, so this doesn't work for me :( Still, i can use ctur's script approach.
- unityByFreedom 6y agoThis is hands down the coolest thing I've learned this year. Thank you
- gigatexal 6y agoThe only thing that trips me up more on becoming a true grey beard other than regex mastery is how and when to properly quote things in my command line incantations.
- bachmeier 6y agoYou can always take advantage of what I believe is the portable property of consecutive strings (without space) being concatenated together. Then you never need to escape anything in your scripts. For instance, to produce a double quote inside single quotes, you can do this echo "'"'"'"'" That's three quoted strings next to each other that produce '"'
- mkl 6y agoYou can get proper code formatting on HN by indenting two characters [1]: echo "'"'"'"'" '"' [1] https://news.ycombinator.com/formatdoc https://news.ycombinator.com/formatdoc
- jrib 6y agozsh has quote-line by default bound to alt-' which will escape your current command line: quote-line (ESC-’) (unbound) (unbound) Quote the current line; that is, put a ‘’’ character at the beginning and the end, and convert all ‘’’ characters to ‘’\’’’. http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html
- x87678r 6y agoJust another reminder bash scripts are unnecessarily complicated, use python instead.
- gj_78 6y agoIsn't this dangerous? Scripts with and without comments don't behave the same. Invariant violation.
- yonrg 6y agoI learned recently that escaping characters in zsh/bash also works for parameter expansion: # zsh using flags ${(flags)name} % string="This is a string with \"\"\" and ''' and \"\" again ''. Also such stuff as & % # ;" % echo $string This is a string with """ and ''' and "" again ''. Also such stuff as & % # ; % echo ${(q)string} This\ is\ a\ string\ with\ \"\"\"\ and\ \'\'\'\ and\ \"\"\ again\ \'\'.\ Also\ such\ stuff\ as\ \&\ %\ \#\ \; # bash using operators ${name@operator} % string="This is a string with \"\"\" and ''' and \"\" again ''. Also such stuff as & % # ;" % echo $string This is a string with """ and ''' and "" again ''. Also such stuff as & % # ; % echo ${string@Q} 'This is a string with """ and '\'''\'''\'' and "" again '\'''\''. Also such stuff as & % # ;' Wow, so easy then, I remember struggling so many times in the past when e.g. iterating over filenames with unusual cahracters. /edit: fix markup
- deleted 6y ago[deleted]
- w0mbat 6y agoI thought "!:q " was what you typed to quit vi.
- ORioN63 6y ago:q! : - Open command prompt. q - Quit. ! - Yes, I'm sure.
- jfhufl 6y agoThis works too: % cat /tmp/sh var=variables x=$( cat <<EOT This string has 'single' and "double" quotes and can interpolate '$var' EOT ) echo $x % bash /tmp/sh This string has 'single' and "double" quotes and can interpolate 'variables'
- bhpnpmg 6y agoI wrote bash function that leverages 'set -x' to get me the quoting in "$@" into a single bash env var say $job or in to a temp file. I use it from time to time -- pretty sure it's not perfect, but it works well enough to be useful. To use it it usually involves an 'eval'.
- deleted 6y ago[deleted]
- bhpnpmg 6y ago## key step: out=$((set -o xtrace;: "$@") 2>&1) # xtrace option shows us quoting for our args ## Cleanup $out # only 1 of the next 4 'left trims' is expected to change $out: out=${out#+++++ } # if $FUNCNAME is eval'd three times there are five "+" chars out=${out#++++ } # if $FUNCNAME is eval'd twice there are four "+" chars out=${out#+++ } # if $FUNCNAME is eval'd there are three "+" chars out=${out#++ } # xtrace prepends the '++ '; ': ' is from our code shown above. # We strip these 5 left most chars. # Would be nice to support any level of evals, (hence any no. # of '+' chars) [...]
- ladberg 6y agoIs anyone else annoyed by the amount of built in magic character strings in bash? There are times where I know a task is possible in a bash script but make it in Python (with no dependencies required and compatible with 2 and 3) because it's easier than looking up every random gotcha and running into issues later on when someone runs it in a directory with a space.
- timonoko 6y agoYou must think about the teletype era. One character commands are economical, even if you have to consult a manual to remember them all. Anything printed on the paper was expensive. I remember stealing color ribbons from the account reserved for manual typists and secretaries. It was also customary to use the paper roll all 4 ways, when editing.