3 ms·

bash has handled spaces in filenames easily for a long time: #!/bin/bash # run this scri0t with 0 or more filename args for file in "$@" ; do

by pdkl95 7y ago

bash has handled spaces in filenames easily for a long time: #!/bin/bash # run this scri0t with 0 or more filename args for file in "$@" ; do do_something_with "${filename}" done > Nobody has a good solution! Bash is just permanently fucked when it comes to file name From the man page bash(1): Special Parameters @ Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is, "$@" is equivalent to "$1" "$2" ...

IshKebab 7y ago
It's possible to write bash that handles spaces in filenames properly, but it's really hard to do it consistently. Unless you test it then you've probably got it wrong. Most people don't test their bash on spaces. Look at the dismissive commenter above who just things "nah why would people use spaces in filenames?". Do you think he quotes everything properly? No chance. CMake is also really bad about this. Although it is arguable slightly better because it uses semicolons as list delimiters instead of spaces, and semicolons in filenames are going to be very uncommon. I would expect there isn't a single CMake project out there that works if you build it in a path containing a semicolon though!
- DonHopkins 7y ago
  To compound the problem, bash tends to attract the kind of care-free security-unconscious people who don't want to be bothered with going out of their way to handle unexpected "trivialities" like spaces in file names or carefully sanitizing user input or meticulously testing their code. The same kind of lazy care-free people who love the convenience of PHP's "register_globals", because it "saves typing". If they're too lazy to use something like Python in the first place, they're probably also too lazy to be conscientious and anal retentive enough to write and test bash scripts that handle spaces in file names properly, or do any kind of user input sanitization. And they don't think twice about running their scripts as root, either. Or writing blog postings telling other people to copy and paste their code snippets into a root shell, or worse yet download and run them without auditing first, by piping curl into a root shell. I recently had an Xcode project fail to build (blender, which uses cmake), because I was using a version of Xcode that had a space in its directory name (because I had foolishly added a space followed by the version number to the name "Xcode 9.4.1.app"). Of course that was because of some of the tools and custom build steps were calling out to bash. The problem wasn't in any of the source code or configuration files, it was in the file name of the tool itself! It took hours of frustration poring over machine generated project and makefiles to diagnose the problem from the obscure unhelpful error message, which I will never get back. In the long term, isn't it more efficient of a lazy person's time to simply use a scripting language that just doesn't have these problems in the first place, and doesn't require you to preemptively bend over backwards and dodge silent invisible bullets all the time? It's pretty ridiculous that a scripting language intended to be used to operate on files falls flat on its face and is so difficult to use when you have something as common and innocuous as files with spaces in their names (or worse yet, file names that begin with dashes -- think of the security implications of that: not something a lazy careless bash scripter would ever pause to reflect about)! It really is an extremely common and unsolved problem. Notice how each "better answer" is actually worse than the last and comes with its own host of problems and edge cases and work-arounds: https://unix.stackexchange.com/questions/9496/looping-through-files-with-spaces-in-the-names https://unix.stackexchange.com/questions/9496/looping-throug... https://stackoverflow.com/questions/3967707/file-names-with-spaces-in-bash https://stackoverflow.com/questions/3967707/file-names-with-... https://www.cyberciti.biz/tips/handling-filenames-with-spaces-in-bash.html https://www.cyberciti.biz/tips/handling-filenames-with-space... https://www.tecmint.com/manage-linux-filenames-with-special-characters/ https://www.tecmint.com/manage-linux-filenames-with-special-... https://www.hecticgeek.com/2014/02/spaces-file-names-command-line/ https://www.hecticgeek.com/2014/02/spaces-file-names-command...
DonHopkins 7y ago
And how do you safely iterate over the output of "find" (including properly handling file names beginning with dashes, while you're at it), without forking another sub-bash for each file name?
- pdkl95 7y ago
  As I said, that hasn't been a problem for a long time: $ find . -type f ./foo bar ./--stupid-filename ./-annoying dir name!/--very annoying path... $ echo "pid=$$" pid=31281 $ while IFS= read -r -d '' file ; do > echo "[pid=$$] do something with '${file}'" > list[i++]="$file" > done < <(find . -type f -print0) [pid=31281] do something with './foo bar' [pid=31281] do something with './--stupid-filename' [pid=31281] do something with './-annoying dir name!/--very annoying path...' $ declare -p list declare -a list='([0]="./foo bar" [1]="./--stupid-filename" \ [2]="./-annoying dir name!/--very annoying path..." [3]="./iter.sh")' Using the character literal $'\0' also works as the delimiter, but Bash doesn't support sending null bytes in args. In bash, $'\0' is the empty string: $ x=$'\0' $ declare -p x declare -- x=""

by pdkl95 7y ago

IshKebab 7y ago

It's possible to write bash that handles spaces in filenames properly, but it's really hard to do it consistently. Unless you test it then you've probably got it wrong. Most people don't test their bash on spaces. Look at the dismissive commenter above who just things "nah why would people use spaces in filenames?". Do you think he quotes everything properly? No chance. CMake is also really bad about this. Although it is arguable slightly better because it uses semicolons as list delimiters instead of spaces, and semicolons in filenames are going to be very uncommon. I would expect there isn't a single CMake project out there that works if you build it in a path containing a semicolon though!

DonHopkins 7y ago
To compound the problem, bash tends to attract the kind of care-free security-unconscious people who don't want to be bothered with going out of their way to handle unexpected "trivialities" like spaces in file names or carefully sanitizing user input or meticulously testing their code. The same kind of lazy care-free people who love the convenience of PHP's "register_globals", because it "saves typing". If they're too lazy to use something like Python in the first place, they're probably also too lazy to be conscientious and anal retentive enough to write and test bash scripts that handle spaces in file names properly, or do any kind of user input sanitization. And they don't think twice about running their scripts as root, either. Or writing blog postings telling other people to copy and paste their code snippets into a root shell, or worse yet download and run them without auditing first, by piping curl into a root shell. I recently had an Xcode project fail to build (blender, which uses cmake), because I was using a version of Xcode that had a space in its directory name (because I had foolishly added a space followed by the version number to the name "Xcode 9.4.1.app"). Of course that was because of some of the tools and custom build steps were calling out to bash. The problem wasn't in any of the source code or configuration files, it was in the file name of the tool itself! It took hours of frustration poring over machine generated project and makefiles to diagnose the problem from the obscure unhelpful error message, which I will never get back. In the long term, isn't it more efficient of a lazy person's time to simply use a scripting language that just doesn't have these problems in the first place, and doesn't require you to preemptively bend over backwards and dodge silent invisible bullets all the time? It's pretty ridiculous that a scripting language intended to be used to operate on files falls flat on its face and is so difficult to use when you have something as common and innocuous as files with spaces in their names (or worse yet, file names that begin with dashes -- think of the security implications of that: not something a lazy careless bash scripter would ever pause to reflect about)! It really is an extremely common and unsolved problem. Notice how each "better answer" is actually worse than the last and comes with its own host of problems and edge cases and work-arounds: https://unix.stackexchange.com/questions/9496/looping-through-files-with-spaces-in-the-names https://unix.stackexchange.com/questions/9496/looping-throug... https://stackoverflow.com/questions/3967707/file-names-with-spaces-in-bash https://stackoverflow.com/questions/3967707/file-names-with-... https://www.cyberciti.biz/tips/handling-filenames-with-spaces-in-bash.html https://www.cyberciti.biz/tips/handling-filenames-with-space... https://www.tecmint.com/manage-linux-filenames-with-special-characters/ https://www.tecmint.com/manage-linux-filenames-with-special-... https://www.hecticgeek.com/2014/02/spaces-file-names-command-line/ https://www.hecticgeek.com/2014/02/spaces-file-names-command...

DonHopkins 7y ago

And how do you safely iterate over the output of "find" (including properly handling file names beginning with dashes, while you're at it), without forking another sub-bash for each file name?

pdkl95 7y ago
As I said, that hasn't been a problem for a long time: $ find . -type f ./foo bar ./--stupid-filename ./-annoying dir name!/--very annoying path... $ echo "pid=$$" pid=31281 $ while IFS= read -r -d '' file ; do > echo "[pid=$$] do something with '${file}'" > list[i++]="$file" > done < <(find . -type f -print0) [pid=31281] do something with './foo bar' [pid=31281] do something with './--stupid-filename' [pid=31281] do something with './-annoying dir name!/--very annoying path...' $ declare -p list declare -a list='([0]="./foo bar" [1]="./--stupid-filename" \ [2]="./-annoying dir name!/--very annoying path..." [3]="./iter.sh")' Using the character literal $'\0' also works as the delimiter, but Bash doesn't support sending null bytes in args. In bash, $'\0' is the empty string: $ x=$'\0' $ declare -p x declare -- x=""