3 ms·

It's fine, the content of an expanded variable isn't parsed further: $ touch "foo \"bar baz"; for f in *; do echo "$f"; done foo "bar baz # quotes

It's fine, the content of an expanded variable isn't parsed further: $ touch "foo \"bar baz"; for f in *; do echo "$f"; done foo "bar baz # quotes don't affect it either $ touch "foo \"bar baz"; for f in *; do echo $f; done foo "bar baz Though once you start passing args with quotes to other scripts, things get ugly. Rule of thumb is to always pass with "$@", and if that isn't enough to preserve quoting for whatever use case, write them out to a tempfile instead, or don't use a shell script for it in the first place.

kstrauser 2y ago
What about in the case of for f in `ls`; do echo "$f"; done Same behavior, for the same reason?
- chuckadams 2y ago
  The quotes are preserved, but backquote expansion fills the argument list using any whitespace as a delimiter. $ for f in `ls`; do echo "$f"; done foo "bar baz If you absolutely must parse ls (let's assume it's some other script that outputs items with spaces) and the output can contain spaces, you have a few options: $ ls | while read f; do echo "$f"; done foo "bar baz # parens keep the IFS change isolated to a subshell $ (IFS="\n"; for f in `ls`; do echo "$f"; done) foo "bar baz But if your filenames contain newlines, you'll really want to stick with the glob expansion, or output custom delimiters and set IFS to that.
  - ChoHag 2y ago
    > If you absolutely must parse ls ... stop and rethink your options. You may be able to get away with parsing the first columns of ls -l but even then a pathologically named file could make itself look like a line of ls output. It's simply not possible in all cases. If you can constrain your input then you may be able to make use of it but in the general case, that's why xargs and find grew a -0 option. Or glob.
    - chuckadams 2y ago
      Agreed when it comes to ls, but this applies to any script whose output you capture. I personally prefer “while read” loops but I’m probably screwed if someone smuggles in a newline.
      - unqueued 2y ago
        If you are iterating over a lot of files, a read while loop can be a major bottleneck. As long as you use the null options from find and pipe into xargs, you should be safe with any filename. I've found it can reduce minutes down to seconds for large operations. If you have to process a large number of files, you can let xargs minimize the number of times a program is run, instead of running it once per file. Something like: # Set the setgid bit for owner and group of all folders find . -type d -print0 | xargs -0 chmod g+s # Make the targets of symlinks immutable find . -type l -print 0 | xargs -0 readlink -z | xargs -0 chattr +i Way faster. But there are lots of caveats. Make sure your programs support it. Maybe read the xargs man page.
        amiga386 2y ago
        Personally I skip the middleman when I can with "find ... -exec cmd {} +" find . -type d -exec chmod g+s {} + Or even minimise arguments by including a test if the chmod is even needed: find . -type d \! -perm -g=s -exec chmod g+s {} + I actually have a script that fixes up permissions, and I was delighted to fit it in a single find invocation which only performs a single stat() on each file in the traversal, and only executes chown/chmod at all for files that need change: # - ensure owner is root:shared # - ensure dirs have 775 permissions (must have 775, must not have 002) # - ensure files have 775 (if w+x), 664 (if w), 555 (if x) otherwise 444 permissions find LIST OF DIRS \ '(' \! '(' -user root -group shared ')' -print -exec chown -ch root:shared {} + ')' , \ '(' -type d \! '(' -perm -775 \! -perm -002 ')' -print -exec chmod -c 775 {} + ')' , \ '(' -type f -perm /222 -perm /111 \! -perm 775 -print -exec chmod -c 775 {} + ')' , \ '(' -type f -perm /222 \! -perm /111 \! -perm 664 -print -exec chmod -c 664 {} + ')' , \ '(' -type f \! -perm /222 -perm /111 \! -perm 555 -print -exec chmod -c 555 {} + ')' , \ '(' -type f \! -perm /222 \! -perm /111 \! -perm 444 -print -exec chmod -c 444 {} + ')' But if you need multiple transformations of filenames in a pipeline like in your second example, then yes xargs will be involved.
  - kstrauser 2y ago
    Thanks for that. For my reputation’s sake, let me clarify that I do always use either globbing or `find -print0` since a more experienced sysadmin drilled that into my head decades ago. I was curious about other edge cases, but I don’t take any convincing.

kstrauser 2y ago

What about in the case of for f in `ls`; do echo "$f"; done Same behavior, for the same reason?

chuckadams 2y ago
The quotes are preserved, but backquote expansion fills the argument list using any whitespace as a delimiter. $ for f in `ls`; do echo "$f"; done foo "bar baz If you absolutely must parse ls (let's assume it's some other script that outputs items with spaces) and the output can contain spaces, you have a few options: $ ls | while read f; do echo "$f"; done foo "bar baz # parens keep the IFS change isolated to a subshell $ (IFS="\n"; for f in `ls`; do echo "$f"; done) foo "bar baz But if your filenames contain newlines, you'll really want to stick with the glob expansion, or output custom delimiters and set IFS to that.
- ChoHag 2y ago
  > If you absolutely must parse ls ... stop and rethink your options. You may be able to get away with parsing the first columns of ls -l but even then a pathologically named file could make itself look like a line of ls output. It's simply not possible in all cases. If you can constrain your input then you may be able to make use of it but in the general case, that's why xargs and find grew a -0 option. Or glob.
  - chuckadams 2y ago
    Agreed when it comes to ls, but this applies to any script whose output you capture. I personally prefer “while read” loops but I’m probably screwed if someone smuggles in a newline.
    - unqueued 2y ago
      If you are iterating over a lot of files, a read while loop can be a major bottleneck. As long as you use the null options from find and pipe into xargs, you should be safe with any filename. I've found it can reduce minutes down to seconds for large operations. If you have to process a large number of files, you can let xargs minimize the number of times a program is run, instead of running it once per file. Something like: # Set the setgid bit for owner and group of all folders find . -type d -print0 | xargs -0 chmod g+s # Make the targets of symlinks immutable find . -type l -print 0 | xargs -0 readlink -z | xargs -0 chattr +i Way faster. But there are lots of caveats. Make sure your programs support it. Maybe read the xargs man page.
      - amiga386 2y ago
        Personally I skip the middleman when I can with "find ... -exec cmd {} +" find . -type d -exec chmod g+s {} + Or even minimise arguments by including a test if the chmod is even needed: find . -type d \! -perm -g=s -exec chmod g+s {} + I actually have a script that fixes up permissions, and I was delighted to fit it in a single find invocation which only performs a single stat() on each file in the traversal, and only executes chown/chmod at all for files that need change: # - ensure owner is root:shared # - ensure dirs have 775 permissions (must have 775, must not have 002) # - ensure files have 775 (if w+x), 664 (if w), 555 (if x) otherwise 444 permissions find LIST OF DIRS \ '(' \! '(' -user root -group shared ')' -print -exec chown -ch root:shared {} + ')' , \ '(' -type d \! '(' -perm -775 \! -perm -002 ')' -print -exec chmod -c 775 {} + ')' , \ '(' -type f -perm /222 -perm /111 \! -perm 775 -print -exec chmod -c 775 {} + ')' , \ '(' -type f -perm /222 \! -perm /111 \! -perm 664 -print -exec chmod -c 664 {} + ')' , \ '(' -type f \! -perm /222 -perm /111 \! -perm 555 -print -exec chmod -c 555 {} + ')' , \ '(' -type f \! -perm /222 \! -perm /111 \! -perm 444 -print -exec chmod -c 444 {} + ')' But if you need multiple transformations of filenames in a pipeline like in your second example, then yes xargs will be involved.
- kstrauser 2y ago
  Thanks for that. For my reputation’s sake, let me clarify that I do always use either globbing or `find -print0` since a more experienced sysadmin drilled that into my head decades ago. I was curious about other edge cases, but I don’t take any convincing.