4 ms·
What if one of your file names has a space in it? Most bash scripts will crash when that happens. Please don't tell me never to use spaces in file names. How
by DonHopkins 8y ago
What if one of your file names has a space in it? Most bash scripts will crash when that happens.
Please don't tell me never to use spaces in file names.
How many human resources are required to track down and fix why a bash script suddenly starts failing mysteriously, years after it was written, when somebody mistakenly dares to use a space in a file name?
That kind of problem never happens with Python scripts (including problems with file names including other punctuation and unicode characters), because Python simply represents file names as strings.
For a scripting language that was indented to manipulate file names all the time, bash sure fucked up big time.
https://www.linuxjournal.com/article/10954 https://www.linuxjournal.com/article/10954
>Work the Shell - Dealing with Spaces in Filenames
>[...] I've outlined three possible solution paths herein: modifying the IFS value, ensuring that you always quote filenames where referenced, and rewriting filenames internally to replace spaces with unlikely character sequences, reversing it on your way out of the script.
>By the way, have you ever tried using the find|xargs pair with filenames that have spaces? It's sufficiently complicated that modern versions of these two commands have special arguments to denote that spaces might appear as part of the filenames: find -print and xargs -0 (and typically, they're not the same flags, but that's another story).
>During the years I've been writing this column, I've more than once tripped up on this particular problem and received e-mail messages from readers sharing how a sample script tosses up its bits when a file with a space in its name appears. They're right.
>My defensive reaction is “dude, don't use spaces in filenames”, but that's not really a good long-term solution, is it?
>What I'd like to do instead is open this up for discussion on the Linux Journal discussion boards: how do you solve this problem within your scripts? Or, do you just religiously avoid using spaces in your filenames?
>Dave Taylor has been hacking shell scripts for a really long time, 30 years. He's the author of the popular Wicked Cool Shell Scripts and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com.
(Dave wrote that 8 years ago, and there are still zero comments on it. Nobody has a good solution! Bash is just permanently fucked when it comes to file names.)
- pdkl95 8y agobash has handled spaces in filenames easily for a long time: #!/bin/bash # run this scri0t with 0 or more filename args for file in "$@" ; do do_something_with "${filename}" done > Nobody has a good solution! Bash is just permanently fucked when it comes to file name From the man page bash(1): Special Parameters @ Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is, "$@" is equivalent to "$1" "$2" ...
- IshKebab 8y agoIt's possible to write bash that handles spaces in filenames properly, but it's really hard to do it consistently. Unless you test it then you've probably got it wrong. Most people don't test their bash on spaces. Look at the dismissive commenter above who just things "nah why would people use spaces in filenames?". Do you think he quotes everything properly? No chance. CMake is also really bad about this. Although it is arguable slightly better because it uses semicolons as list delimiters instead of spaces, and semicolons in filenames are going to be very uncommon. I would expect there isn't a single CMake project out there that works if you build it in a path containing a semicolon though!
- DonHopkins 8y agoTo compound the problem, bash tends to attract the kind of care-free security-unconscious people who don't want to be bothered with going out of their way to handle unexpected "trivialities" like spaces in file names or carefully sanitizing user input or meticulously testing their code. The same kind of lazy care-free people who love the convenience of PHP's "register_globals", because it "saves typing". If they're too lazy to use something like Python in the first place, they're probably also too lazy to be conscientious and anal retentive enough to write and test bash scripts that handle spaces in file names properly, or do any kind of user input sanitization. And they don't think twice about running their scripts as root, either. Or writing blog postings telling other people to copy and paste their code snippets into a root shell, or worse yet download and run them without auditing first, by piping curl into a root shell. I recently had an Xcode project fail to build (blender, which uses cmake), because I was using a version of Xcode that had a space in its directory name (because I had foolishly added a space followed by the version number to the name "Xcode 9.4.1.app"). Of course that was because of some of the tools and custom build steps were calling out to bash. The problem wasn't in any of the source code or configuration files, it was in the file name of the tool itself! It took hours of frustration poring over machine generated project and makefiles to diagnose the problem from the obscure unhelpful error message, which I will never get back. In the long term, isn't it more efficient of a lazy person's time to simply use a scripting language that just doesn't have these problems in the first place, and doesn't require you to preemptively bend over backwards and dodge silent invisible bullets all the time? It's pretty ridiculous that a scripting language intended to be used to operate on files falls flat on its face and is so difficult to use when you have something as common and innocuous as files with spaces in their names (or worse yet, file names that begin with dashes -- think of the security implications of that: not something a lazy careless bash scripter would ever pause to reflect about)! It really is an extremely common and unsolved problem. Notice how each "better answer" is actually worse than the last and comes with its own host of problems and edge cases and work-arounds: https://unix.stackexchange.com/questions/9496/looping-through-files-with-spaces-in-the-names https://unix.stackexchange.com/questions/9496/looping-throug... https://stackoverflow.com/questions/3967707/file-names-with-spaces-in-bash https://stackoverflow.com/questions/3967707/file-names-with-... https://www.cyberciti.biz/tips/handling-filenames-with-spaces-in-bash.html https://www.cyberciti.biz/tips/handling-filenames-with-space... https://www.tecmint.com/manage-linux-filenames-with-special-characters/ https://www.tecmint.com/manage-linux-filenames-with-special-... https://www.hecticgeek.com/2014/02/spaces-file-names-command-line/ https://www.hecticgeek.com/2014/02/spaces-file-names-command...
- narnianal 8y agoI seriously would have never thought that anybody would really consider using spaces in filenames. For me it's a sign to not work in such an environment, which might be the reason I rarely even see it. But sure if you want a script that is widely used you might want to handle that case as well. In any case I don't say put in a Bash script and let it grow to ten million lines over 20 years. My approach to bash scripts is this: 1. start with a bash script to understand a problem and see how a solution might be structured. PoC level is enough here. 2. When you know you solve the right problem and the customer is willing to make something bigger, then put ~80% of the code in a good programming language (Python is my personal favorite, but Ruby, Java, PHP, C#, C++, Lisp might also work). That happens about 3 months into a project. 3. If you start to hit bottlenecks and performance ceilings put the 30-50% that are relatively stable and most performance hungry into C/C++ code and really optimize the heck out of it. This should be done only with an aged project, like 2 years in, and very pessimistically, since highly optimal code usually is badly readable and badly maintainable. You certainly don't want to go into that code 6 years later and drastically change its API to your python code. So a ten year project would have a thin layer of bash/docker at the top, a strong core of highly flexible code (of Python for me as well) in the middle, and a leg of high performance C code here and there. I honestly have never seen a project that really needed the C-layer though. In modern IT world it seems most problems are old at least from a political/management point of view between 2 weeks and 3 months. So in fact, although I count myself as a Python coder, the last few years I've spent writing way more bash and docker related stuff, which then is either never touched again or directly thrown away after a few weeks, no matter if successful or not. PS: Please don't use spaces in file names. Not just for scripts. It is also not clear for users where something ends, and they are even less used to putting string indicators around file names than you are about handling spaces with bash there. And last but not least stuff like spaces in filenames, starting scripts with upper case letter etc, they might even be just a small annoyance. But you can be sure that skilled developers will put you 2 classes below their average when they see this, because it simply shows a lack of the right spirit.
- saagarjha 8y agoNot everyone who uses files is a developer. People should be able to name their files what they want. When the time comes that somebody runs a script on it, it should not do something wonky because they decided that "Spring Pictures" was a much better name than "spring_pictures".
- enriquto 8y ago> What if one of your file names has a space in it? Most bash scripts will crash when that happens. And good for them! What kind of savage puts spaces on their filenames?
- DonHopkins 8y agoAll my Whitespace scripts have file names with nothing but spaces in them! https://en.wikipedia.org/wiki/Whitespace_(programming_language) https://en.wikipedia.org/wiki/Whitespace_(programming_langua... I also use spaces in the file names of all my C++2000 source code that takes advantage of the wonderful new whitespace overloading feature -- it's conventional to use the ".cpp " and ".h " suffixes with trailing spaces. http://www.stroustrup.com/whitespace98.pdf http://www.stroustrup.com/whitespace98.pdf
- enriquto 8y agoThat kind of savagery is deeply respected, of course! And it should go even further. Now that the modern compilers and interpreters (clang, python3, etc) allow unicode in variable names, the next step is certainly to accept spaces in variable names. With a slight modification, the python language can be adapted to support spaces inside variable names. After all, people have the right to name their variables however they want!
- DonHopkins 8y agoNot to mention that Unicode has several delicious varieties of invisible non-breaking, zero width spaces and shy hyphens to choose from! (Oooops, I mentioned it.) Tastes great, less filling, and perfectly justified! https://en.wikipedia.org/wiki/Non-breaking_space#Width_variation https://en.wikipedia.org/wiki/Non-breaking_space#Width_varia... https://en.wikipedia.org/wiki/Zero-width_space https://en.wikipedia.org/wiki/Zero-width_space https://en.wikipedia.org/wiki/Soft_hyphen https://en.wikipedia.org/wiki/Soft_hyphen