4 ms·

I think that if an attacker can control filenames fully, this is the least of your worries. Otherwise this article could've just as well been titled "why you sh

• inopinatus 2y ago
  > Has anyone actually found a good use for filenames containing newlines? Yes - in tests for code that handles filenames, shell or otherwise.
  • fshbbdssbbgdd 2y ago
    Somebody does it once and now we all have to handle it forever!
  • kelnos 2y ago
    I think that falls under "no".
    • II2II 2y ago
      Seeming as newlines may appear in file names, it is a good use. Otherwise something as mundane as a bug in program A (which produces a file name with a newline) has a potential to expose a bug in program B (which fails to interpret a file name with a newline).
      • tyre 2y ago
        At some point it feels fine that code fails. If you’re writing shell scripts against file names that you don’t control and someone puts a new line, that’s on them. They’re asking for a bad time.
        • inopinatus 2y ago
          Many shell scripts we write run as root, or as a deployment user (which is often privileged in terms of its access to business-critical code, packages, data stores, user uploads). Or they may be supplied for use as utilities by folks with other domains of expertise. We should never plan to enable the possibility of uncontrolled outcomes due to bad/hostile third-party input; moralising blame after an incident isn't a productive activity.
          • Filligree 2y ago
            Maybe the kernel should be fixed to refuse filenames with newlines in them, if there are no other use cases…
            • inopinatus 2y ago
              The option (and it would have to be an option) to do so doesn't sound like an unreasonable defense-in-depth measure, but given the myriad Unix and Unix-like variants in the world, and the possibility that your code may not be running on such a configured target system[1], we still end up having to have adequately defensive basic coding standards. Honestly, though, it's one reason I write a lot of my own scripts in zsh these days. Zsh has its own quirks, and I don't touch oh-my-zsh with a barge pole, but it's a darn sight quicker to reason about expansions in zsh code compared to bash, especially if arrays are in the mix. Unfortunately zsh is not necessarily available everywhere and certainly remains outside the base install of many *nix distros. [1] we can prove the existence of at least one such device, by construction: it's the CI/CD host on which tests run, because those tests themselves may specifically and intentionally be for safe handling of such filenames, and therefore rely on being able to instantiate them
            • SAI_Peregrinus 2y ago
              Rather, the POSIX specification should be fixed to exclude `\n` from filenames, instead of just `/` and `\0`.
      • deleted 2y ago
        [deleted]
  • mminer237 2y ago
    We should also require all files to include metadata of their names translated into Klingon so we can properly test if programs are verifying the translation correctly.
    • inopinatus 2y ago
      "Klingons do not release software. Klingon software escapes, leaving a bloody trail of design engineers and quality assurance people in its path." This certainly clarifies that input validation & robustness is a security matter
      • blooalien 2y ago
        Well, this comment straight up wins my "favorite comment of the day" award. Take my upvote as your prize. :)
        • jerf 2y ago
          A complete list: https://www.smart-jokes.org/klingon-software.html https://www.smart-jokes.org/klingon-software.html Though I agree that's the standout quote from the list. I've used it before at work, as appropriate.
          • blooalien 2y ago
            After reading that list, I see that you chose by far the best one of the bunch. ;)
  • chipdart 2y ago
    > Yes - in tests for code that handles filenames, shell or otherwise. That sounds like a variant of the old broken window fallacy.
• tedunangst 2y ago
  Users can typically create any filename they like in /tmp, which causes trouble when the nightly cleanup script runs and tries to delete the old ones.
  • mozman 2y ago
    find -print0 | xargs -0
  • chipdart 2y ago
    > Users can typically create any filename they like in /tmp, which causes trouble when the nightly cleanup script runs and tries to delete the old ones. I don't think that's how /tmp works. Aren't those files garbage collected either at boot time or after X days of creation? I mean, if your program creates a file in /tmp and keeps it open for days then you clearly screwed up as you're misusing the whole concept of a temporary file. For that there are better places in the file system hierarchy.
    • pastage 2y ago
      It all depends, next Debian will mount tmp in ram, LWN has a great article about it as usual. They talk about the problem with clean up, e.g. tmux sessions are stored in tmp. https://lwn.net/Articles/975565/ https://lwn.net/Articles/975565/
      • chipdart 2y ago
        I don't think your reply holds any relevance. All it discusses is the file system used by a distro for it's /tmp. That's irrelevant for this discussion. The file system is still wiped on reboot and old files are garbage collected after X days of creation, which is handed by the likes of systemd and upstart. The article you quoted reiterates exactly the points I made.
• gruez 2y ago
  >I think that if an attacker can control filenames fully, this is the least of your worries. There's a lot of ways an arbitrarily named file can get on your system. You could be cloning a random git repo, extracting some sort of archive, or plugging in a usb drive.
  • IshKebab 2y ago
    Yes and more to the point I don't expect an attacker to be able to own my system just by having a particular filename on it. I mean... I do expect that on Linux because shitty Bash scripts are so common. But that is a problem that should be solved.
• dwheeler 2y ago
  > Has anyone actually found a good use for filenames containing newlines? They're great for attacking systems, because so many programs aren't prepared for them. Any program that uses filenames as keys (and there are many) can have this vulnerability if the input validation has a weakness. I don't think there are any legitimate uses. They also make it unnecessarily hard to write robust shell scripts.
  • userbinator 2y ago
    Any program that uses filenames as keys (and there are many) can have this vulnerability if the input validation has a weakness. That's why things like upload sites usually ignore the provided file name and generate their own (unique) one, or else limit it to a very safe subset like [0-9A-Za-z_]
    • o11c 2y ago
      The POSIX portable filename character set is: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z 0 1 2 3 4 5 6 7 8 9 . _ - Additionally: * The slash `/` is portable as a separator. * The filename should not be empty * The filenames `.` and `..` are special. (in many contexts it's reasonable to exclude all components that start with a dot) * The filename should not start with a `-` * The entire path should be no more than 256 bytes and no component should be more than 14 bytes (widely ignored since we assume modern filesystems).
  • IshKebab 2y ago
    Shell makes it unnecessarily hard to write robust shell scripts. I don't think filenames are to blame here. I mean it's not an issue in Python for example.
    • dwheeler 2y ago
      It is an issue for Python, though this specific issue isn't the main issue for Python. More generally, it's even harder to routinely call programs while connecting them with pipes and ensuring that the programs run in parallel. It's a pain to routinely call other programs and get results Python. It takes many many lines of Python to do tasks that are clearer one-liners in shell... and vice versa. Python and shell are different; each are good at different things.
• colechristensen 2y ago
  This is one of the ways you mitigate the “if they already have this you’re screwed” thing. The reason it’s not the least of your worries is if somebody can control a file like that the next step is some kind of escalation and this right here is one of those escalations.
• bbwbsb 2y ago
  Whether it matters if attackers control filenames depends on the threat model. I can think of a few where it doesn't matter. One example: you provide managed web application hosting and have some scripts that check something about the files so they get handled properly. If you have a good reason to create a file, and the value you want to identify it by contains a newline, then that is a good reason to create a filename containing new lines. Filenames are arbitrary bytes excluding null. Regardless at scale, eventually such a file will end up being created. Encoding bugs, libraries, user creation, messed up copy/paste, etc. File names may not even be utf8. `ls | cmd` is also just an sh code smell. Bash is big on discipline and suffering and not so much on correctness (like C and forth) so code smells matter more. When I worked with/maintained large bash/sh scripts, one `ls | cmd` and I would reread everything the person wrote because it was very likely it was horribly broken. Another one is 'set -euo pipefail`, which people love to set but often don't understand. A lot of sh/bash, maybe most of it, is written by people that don't take it seriously as a language, and it shows. Also basically everything on stack overflow about it is wrong, and probably everything chatgpt says. I remember it being uniquely bad in that regard. To the point that whenever someone says a neat bash fact, it is better to assume they are part of an elaborate conspiracy to get you to write catastrophic bugs until verified independently. Anything short of that level of paranoia and you eventually do the steam thing and delete everyone's files. The article could have been titled "why you shouldn't use sh".
• nine_k 2y ago
  I wonder what's the point to support newlines as a valid character in file names. And, for that matter, any other ASCII characters in the 0x00..0x1f range. Of course, file names are not characters but bytes, and this allows filesystems to transparently support various filename encodings, which are handled by locale, not by filesystem. On the other hand, there are few practical encodings that involve bytes that are control characters in ASCII. Obviously not UTF-8, not 8-bit stuff like ISO-8859-*, nor the widespread Japanese, Korean, or Chinese encodings, as much as I'm aware. UCS2 filenames would be impossible though, but I suspect nobody uses UCS2 as a locale in practice (even ironically). I think adding a mount flag that would disallow non-printable ASCII bytes in filenames could improve security, and is also really trivial to implement. I cannot be the first person to suggest it. I wonder why it was never implemented.
  • userbinator 2y ago
    I wonder what's the point to support newlines as a valid character in file names. And, for that matter, any other ASCII characters in the 0x00..0x1f range. 0 is reserved for the terminator so you can't have a null byte, but I think instead of looking at this from the perspective of "what's the point to support", you should think of it as "why add additional code just for restriction, when we can consider that an upper-layer concern". The same goes for case sensitivity --- it makes the FS code in the kernel more generic and simpler. Contrast this with the other extreme, filesystems that have to effectively implement a subset of Unicode because they're case-insensitive beyond ASCII. Even Apple abandoned that idea over time: https://news.ycombinator.com/item?id=13953800 https://news.ycombinator.com/item?id=13953800
    • coldtea 2y ago
      >you should think of it as "why add additional code just for restriction, when we can consider that an upper-layer concern". Because that way you solve the problem at the root? Of course that's not the "not my problem" Jersey-style approach. But tons of our current problems are because of that Jersey-style approach.
  • coldtea 2y ago
• Joker_vD 2y ago
  > Has anyone actually found a good use for filenames containing newlines? Yes. They are great for more legible names for document files on the Desktop, or even for using as post-it notes, also on Desktop. Of course, it only works for people who don't start fullscreen tmux session as the first thing they do after the boot and then never leave it.
• elric 2y ago
  > if an attacker can control filenames fully, this is the least of your worries Sure, but practicing defense in depth is often a good idea. Maybe today's version of your application sanitizes user input correctly, but tomorrow's version might suddenly allow funky characters. If your super duper bash backup script doesn't get updated, you might suddenly have a problem.