23 ms·
I work on a complex desktop application, and it's been astounding the number of bugs that have appeared over the years triggered by spaces and other unusual cha
by pimterry 5y ago
I work on a complex desktop application, and it's been astounding the number of bugs that have appeared over the years triggered by spaces and other unusual characters in file names. If you do anything with subprocesses or path processing, it's absurdly easy to hit in a thousand different ways, over and over again.
Pro tip: rename your development directory (or even better: the workspace path in CI) to put a space and/or special characters in it.
Forces you to deal with this properly, and immediately ensures that every automated test checks this case without you having to remember every time. Hasn't been particularly inconvenient, since I'm autocompleting it 99% of the time anyway, and I haven't shipped a single path parsing bug since.
- Izkata 5y ago> Pro tip: rename your development directory (or even better: the workspace path in CI) to put a space and/or special characters in it. A former co-worker changed his name in our auth system to include an apostrophe, so that whenever we handled names wrong he'd find it.
- floatingatoll 5y agoI set my nickname to U+FFFD at one point in one work system, resulting in a variety of bug reports and concerned emails. I think I dropped it since it was generating false reports from people who didn't check what character the page contained before reporting it.
- geoduck14 5y agoOh, I like this!
- ridaj 5y agoÁčçëñts hęlp tóø
- ygra 5y agoFor anyone curious, this is called Pseudo-localization (https://en.wikipedia.org/wiki/Pseudolocalization https://en.wikipedia.org/wiki/Pseudolocalization). I first singled across this in Raymond Chen's blog.
- deleted 5y ago[deleted]
- reaperducer 5y agoOne of the systems I built is being used by a group of younger people. I included an emoji in the superuser account name, just to make sure it would work. And to remind me to think more broadly about user input.
- ajmurmann 5y agoA related too for CI: change the system time to be a time zone that is during your work hours in a different day already than UTC. Really helped getting failures earlier than 4pm PST.
- brundolf 5y agoAt my last job we had a wild time-zone bug that only happened with your system location set to Mumbai. I left mine set to that for the rest of my time there.
- cpeterso 5y agoRelated: here's a recent Firefox bug about a test that failed during the daylight saving time change: https://bugzilla.mozilla.org/show_bug.cgi?id=1739847 https://bugzilla.mozilla.org/show_bug.cgi?id=1739847
- scubbo 5y agoCould you consider rephrasing this? It sounds like an interesting observation that I'd love to understand, but I'm genuinely not able to parse it. My best guess is "change the system time to be a timezone for which, during your work hours, the other-timezone is in a different day than UTC is" - but I'm still not sure what effect that would have on CI failures.
- Teknoman117 5y agoI read that as "set your CI to run earlier in your workday so you don't get new error reports at the end of the day." Midnight UTC being 4 pm/16:00 PST.
- ajmurmann 5y agoMaybe an example of the failure this detects helps: when I used to work on Rails apps in the olden days it was easy to call Time.now and get the local time instead of Time.zone.now to get UTC time. This often lead to wrong dates but tests would only fail once it was a new day in UTC land but still the old day in the local time zone. Making the CI machine's system time something Fiji time really helped in getting failures much sooner after changes were pushed.
- enragedcacti 5y agoTo have such thoughtful coworkers. On an old team I had two coworkers named Chris and once in a blue moon when they reviewed each other code master would start crashing because one of them accidentally left in an absolute path starting with "/home/chris/".
- curuinor 5y agothe proper name of the glorious sultan of slack, j. r. "bob" dobbs, has the quotation marks and therefore is a great subject for this
- fernandotakai 5y agomy test accounts always have emojis + accents + other weird characters. it keeps everybody on their toes lol.
- qwertox 5y agoI add a Japanese character into any .py, .js and .html file to ensure that Unicode is working properly through the entire chain. Mostly in form of a variable which gets passed along, even in URL parameters.
- soheil 5y agoObligatory xkcd https://xkcd.com/327/ https://xkcd.com/327/
- ygra 5y agoI've used to have a space in my user name and even contemplated to add a bit of non-1252 Unicode. You find a lot of issues, but unfortunately often in tools you have little control over and end up not being able to work effectively at times. It ended up being more frustrating than helpful.
- idatum 5y agoSomewhat related to injecting unusual characters, in my experience in localization efforts: Inject a Turkish 'I'. I don't know how to type or paste it here, but picture an English lower case 'i' that is upper case. It is a splendid way among many to shake out some loc bugs.
- InfiniteRand 5y agoIt's easy to tell users to make a folder with no spaces if you're setting up a global path, however if you have an application that runs in user directories things can become painful fast. Changing your user name is a pain and can leave things inconsistent, but having to handle all the variations in people's names with spaces, punctuation, international characters, can just be mind boggling.
- qwertox 5y agoIn that case, be thorough and insert a Chinese and an Arabic character to enforce a Unicode check.
- lifthrasiir 5y agoWhile I agree that we should do this in the ideal world, doing so will inevitably break other necessary tools so it is unworkable for me :(
- alpaca128 5y agoSeems like MS had the same idea according to an answer in the link: > Microsoft intentionally made programs install to C:\Program Files on Windows 95+ to force programmers to deal with spaces in filenames.
- vesinisa 5y agoExcept for programs that were too old / obscure to fix I guess. I think at least the Symbian Development Kit was such that builds would fail with strange errors unless you installed it in any other path than the default immediate subdirectory of C:\, let alone under "Program Files".
- dr-detroit 5y agoPlenty of new stuff does this. As long as youre not .net or javascript nobody scrutinizes the trash work developers charge money for.
- lifthrasiir 5y agoAnd yet they introduced C:\ProgramData in later versions.
- kitkat_new 5y agowhy "yet"? one occurrence is enough to make devs care about it
- jjoonathan 5y agoImagine if they made programmers put 64 bit DLLs in a "System32" directory and 32 bit DLLs in a "SysWoW64" directory. That would really keep 'em on their toes!
- eyegor 5y agoYou should look into the behavior of the /windows/sysnative link. It appears and disappears depending on whether your process is running as 32 bit or 64 bit.
- kitkat_new 5y agoPro tip2: Use std lib path processing utilities
- jeffwask 5y agoIt doesn't even have to be complex, often basic automation tasks fail with spaces and special characters. Honestly, treating a file system like a natural language processor is a bad idea. Besides at this point with how digital we have all become who can't understand... thisismyconfig.txt vs this is my config.txt or this_is_my_config.txt ...i've forced myself to stop using spaces, character, and even cap. They are all constructs that provide minimal value for the extra complexity.
- long_time_gone 5y ago> thisismyconfig.txt vs this is my config.txt or this_is_my_config.txt Just wondering, what is the readability of this for people who are dyslexic?
- reaperducer 5y agoOr in my case, people for whom English is a second language, or have low education levels. Saying, "who can't understand..." is arrogant, selfish, and an example of why normal people hate people in the SV echo chamber.
- jeffwask 5y agocestmaconfig.txt vs cest ma config.txt vs cest_ma_config.txt It's the same in any language. Hugs who hurt you. I'm also pretty sure most of us in any language use Slack, SMS or other forms of communication where text isn't necessarily presented in a grammatical correct manner and we all figure out what the person is saying.
- long_time_gone 5y ago> Saying, "who can't understand..." is arrogant, selfish, and an example of why normal people hate people in the SV echo chamber Exactly how I feel every time Economics is brought up on HN.
- throwaway2077 5y agoSV echo chamber is on your side here - it is very in vogue to denounce anglocentrism. they were defending hieroglyphs and emoji in variable names in that thread about invisible javascript backdoor a day or two ago if you'd like a recent example
- shane_b 5y agoMy Mac is formatted case sensitive when the default is case insensitive. This will also catch a ton of import related bugs. League of legends doesn’t run until I sed files for instance.
- memsom 5y agoI once returned a printer because the Mac driver and support software expected and enforced case insensitive access and basically couldn't install properly on my case-sensitive HFS+ volume. It half installed and blatantly just didn't work in any way when installed.
- NegativeLatency 5y agoAdobe software used to refuse to install on case sensitive file systems back in the not too distant past.
- dunham 5y agoCirca Y2k, I learned that the OSX Palm Pilot software didn't work with case sensitive. I've since given up and stuck with the default. (I'm anti-case folding in general, because of the ambiguity.)
- mdaniel 5y agoI also enjoyed doing that, but had to make a DMG just for Steam because it straight-up refuses to run on a case sensitive FS (that's true on Windows, also, which I suspect is how we all got here). I think the most recent Steam versions either caught wind of my trickery or -- more likely -- run something from $HOME/Library/SomethingOrOther and thus the work-around it no longer works When I got a new Mac, I just gave up and acquiesced to the case-retentive world :-(
- deckard1 5y agoI have coworkers on Mac that write node/JS code. Every once in awhile I'd pull down the latest code and it wouldn't run. I'm on Linux. Sure enough, they had SomeFile and were importing Somefile and it works fine on Mac but not on Linux (which, of course, is what our production servers use). It amazes me that "works fine on my machine" is still a thing when I definitely worked at companies that solved this back in the 2000s. It was solved. It was done. Then devs became enamored with running everything locally. Even dozens of microservices or databases. Even though JS is fairly isolated, you still have NPM packages that need built against the local OS and C/C++ library and compilers, etc. Which also has caused issues in the past.
- echelon 5y agoBetter solution: only allow ASCII, maybe dashes, and up to twelve characters. Problem solved. Enforce this in LDAP. Strict convention is better than flexibility and predicting obscure edge cases that can fail.
- pimterry 5y agoIn my case, and for many people writing desktop software, and for absolutely everybody writing open-source tools or libraries, unfortunately you can't control the environment. Non-ASCII paths are extremely common (e.g. the user's home directory on Windows, for the large majority of users outside the English-speaking world) and spaces, punctuation and weirder characters will definitely happen when you least expect it. Yes if you can avoid it then absolutely that's great, but I don't think most people can. It's also not usually very difficult to deal with, as long as you actually spot the issue in the first place.
- mikepurvis 5y agoUgh, we have the 15 character Active Directory limit now with hostnames, and a previous IT administration has imposed a convention that every name had to follow [prod|dev]-[ph|vm]-[service]-[nn]. So basically every production service is prod-vm-owtf-01— you get exactly four characters to actually describe what the machine does. Works great when the service is "jira" or "wiki", but there are a lot that are pretty mystical-sounding, like jkns, jwrk, cntr, hrbr, etc, where you kind of just have to know.
- HNo 5y agoI kind of like that honestly. No doubt you need some documentation so everyone knows what the service abbreviations are, but after you've been working there for a month you get it. Makes everything clean, consistent, and informational. You can quickly ascertain what a specific host is doing just from the name.
- mikepurvis 5y agoOh absolutely it makes sense to have a standard, and being able to tell at a glance if something is a VM or physical machine is of value also. But dedicating 2/3s of the character budget to such a scheme is madness. If the prod-vm- prefix simply become pv-, then you'd at least be able to do pv-jenkins-01 again. Anyway, all this was fine when we were on LDAP rather than Active Directory. So basically it's all Windows' fault.
- achn 5y agoI maintain a similar system, where a variety of companies submit files that get processed through multiple services - it is astounding how ridiculous people’s naming of files can be; spaces are the least concerning!
- josteink 5y ago> it's been astounding the number of bugs that have appeared over the years triggered by spaces and other unusual characters in file names If you consider spaces “unusual” I would say you haven’t encountered a single average user in your lifetime. Spaces in file-names is the single most common thing people have, outside programming environments. As a x-plat developer, the only platform where I (still) regularly encounter these kind of bugs are platforms where solving problems through scripting is common, like Linux, where the primary means of operation is through stringly-typed statements getting parsed and processed in a untyped-fashion. It's not very reliable. On Windows people more often use “real APIs” (because scripting doesn't really work as well), but then these problems just goes away. Pros and cons, I guess.
- SAI_Peregrinus 5y agoIt's especially funny that it affects Linux so much. Most file systems allow everything except `/` and NULL in file names. Early AT&T UNIX even allowed NULLs! POSIX shells use the IFS variable to perform field splitting, and it defaults to <space>, <tab>, and <newline>. The choice to perform field splitting by default (particularly with spaces in the default IFS set) has caused no end of headaches for developers and users.
- KronisLV 5y ago> Pro tip: rename your development directory (or even better: the workspace path in CI) to put a space and/or special characters in it. This will also break any code in external tools that are called during the builds of your application and do not handle spaces correctly for whatever reason, thus making it so that you won't be able to successfully finish the build. Then again, you probably shouldn't be relying on technologies like that, but when you're struggling to keep an old enterprise system alive, causing yourself more problems is not necessarily what you should do. Still a good idea in most cases, though.
- chris_wot 5y agoAnd yet OneDrive WP t allow fir spaces before or after a file name.
- sysadm1n 5y ago> other unusual characters in file names Saw a few hacks where malware authors used the RTL feature (which is baked into Windows) to obfuscate file extensions. It looked like .exe.innocuous-document.docx, but was actually .docx.innocuous-document.exe
- masklinn 5y agoThis exact vulnerability in most modern code editors just made the rounds, allowing smuggling malicious code right through review.
- mwcampbell 5y agoMy favorite filename special character bug was when I implemented CD ripping in 2005, and one of our beta testers ripped a CD with a song called "Have You Ever?". My code wasn't prepared to filter out the question mark on Windows.
- mixmastamyk 5y agoI just hit the one where an album folder ends in a period. Rsync copies every time because the period is dropped by the filesystem silently. :-/
- Foobar8568 5y agoLet's not forget return carriages in filenames within apps...
- wongarsu 5y ago> Pro tip: rename your development directory I changed my username to not contain a space because it was too annoying to deal with all the random dev tools breaking. The worst offender was probably npx on Windows [1] (resolved after four years by deprecating npx), but it was far from the only one (though the JS ecosystem was somehow the worst in this regard of all languages I worked with). 1: https://github.com/zkat/npx/issues/100 https://github.com/zkat/npx/issues/100
- kermire 5y agoSame, even I had to rename my user folder to not have a space because so many tools were breaking.
- dheera 5y agoOr not, which when bugs crop up will teach the businessy types to stop putting spaces in their filenames.
- cduzz 5y agoAnd add a emoji, a character in a right to left language ( א) and perhaps 太. Maybe italicize one of those too...
- 5faulker 5y agoFor those purposes I've found hyphen to be a nice substitute.
- redwall_hp 5y agoI don't know if it's still a problem, but it used to break Python virtualenv badly. If your working directory had a space anywhere in the path, it would throw a huge fit and not work. Which is problematic when the expected name for a Mac's boot drive is "Macintosh HD" (if you ever had a reason to run a virtualenv outside of your home directory).
- wldcordeiro 5y agoEven capitalization is a pain in the ass thanks to how OSes treat file names. I pretty much stick with either `file-name.ext` or `file_name.ext` exclusively now.
- Spooky23 5y agoSomeone should provide the OneDrive/SharePoint people some of this religion. Mysterious character requirements that do not conform with Microsoft’s OS limits, limits on tbe fully qualified pathname length, etc.
- ralphc 5y agoLate '90s I worked on Java software that got installed on several Unix platforms, including Linux for IBM mainframes. When you deal with the default en/de-coding of Unicode to EBCDIC you never have trouble with Java byte encodings ever again.
- BiteCode_dev 5y ago> Pro tip: rename your development directory (or even better: the workspace path in CI) to put a space and/or special characters in it. The problem with that is that YOUR code may handle it, but your tooling may not. If my code formatter break on spaces, I'm not going to change the formatter.
- ChrisSD 5y agoYou could submit a PR to their repo.
- BiteCode_dev 5y agoI could submit a PR to 5 tools a week on average. I actually have the time and resources to do it once a year. Last week I opened a ticket for a Firefox bug. Following up on the bug took me 2 hours in total. FOSS is not free, you pay it with your time. And as with everything you pay for, we all have a budget.
- cerved 5y agoSpaces are a pain in the ass when you're using CLI so I'd rather enforce a no space policy
- reayn 5y agoMost shells will behave just fine if you put a quote (single or double) before anything that has a space. A small extra step but something you get used to if you spend a lot of time in the cli.
- cerved 5y agoEscaping spaces is a pain. I have to do it every day. I set up symlinks which help navigating around but then the relative paths are wrong for git. No thanks. Friends don't let friends put spaces in paths
- agumonkey 5y agoSee the recent article about unicode invisible glyphs in JavaScript or bash. Naming freedom needs a stdlib module
- rossy 5y ago> anything with subprocesses I'm begging software developers to stop using subprocess APIs that take a string argument (system(), child_process.exec(), Process.Start(string)) and start using subprocess APIs that take an array of arguments (execvp(), child_process.execFile(), Process.Start(string, IEnumerable<string>).)
- WalterBright 5y agoSometimes / works as a path separator in Windows, sometimes it doesn't. It's not predictable. I never use / on Windows as a result.
- ygra 5y agoThe only common place where it doesn't work is in CMD for executing programs and as arguments for built-in commands. Everything else goes directly to the relevant APIs which don't care about / or \. These days using CMD instead of PowerShell should be rare enough and PowerShell certainly doesn't mind the slashes.
- dr-detroit 5y agothere are things you cant do in .net that you need the old Registry commands for and those don't accept spaces
- AlfeG 5y agoToday I learned that You cannot install Tailscale on windows if installer is inside path with non-latin chars.
- franga2000 5y agoMore importantly than your source files, put your testing data on such a path as well. Nobody uses absolute paths in testing so it doesn't matter how many spaces your absolute path has if your input is "./tests/file1". Put those files in a folder with spaces too and throw in a unicode character for good measure.
- uberswe 5y agoI did something similar on accident. I used to keep all my development work synced with Dropbox and I had a work and a personal account. So any of my own projects would have /Dropbox (Personal)/ in the path which did catch some bugs. Dropbox renamed my folder to "Dropbox (Personal)" automatically when connecting a work account.