7 ms·
Spawn your shell like it's the 90s again
- nothrabannosir 10y agoThe posix file system API invites these race conditions. but when I want to access the fs from my app, it often feels like the only choice I have is to just accept it. What can I do to change it? Is there a reliable, widespread and usable successor to the posix API that offers transactions, or "compare and swap"-type operations? Or are we forever caught in this catch 22 between OS writers and application programmers ? I wouldn't mind an API that can be (unsafely) simulated on regular posix for compatibility, while being forward compatible with the next gen of file IO on systems that do support it, for example.
- qwertyuiop924 10y agothe problem is, you can't really avoid this kind of race condition. Not in a pre-emptive system. You can't stat a file and open it at the same time: the whole point of the stat is to see if you SHOULD open it. The best you could do is maybe stat the file after you open it. But I'm not sure what other problems that would cause.
- sebcat 10y agoyou call fstat on the opened file descriptor and compare device IDs and inodes, as was done in the referenced OpenBSD patch from -96: http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/libexec/mail.local/mail.local.c.diff?r1=1.1&r2=1.2&f=h http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/libexec/mail.lo... EDIT: corrected year
- ma2rten 10y agoI could think of multiple ways of fixing this problem changing the API. For instance they could have an api that opens the file only if it is not a synlink.
- tedunangst 10y agoLike open(O_NOFOLLOW)?
- nothrabannosir 10y agoThere are many possible solutions: Obtain a "session ID" through stat and always use that to access the file: id := stat("/a") read(id) write(id, ..) // doesn't protect race access to a file, // but at least you're not using the // pathname twice, which already solves a // lot of problems Use transactions: transid := ftransopen() f := stat("/a") read("/a") write("/a", ..) ftransend(transid) // fails if clashes with other fs changes Have "last modification ID" counters that change for every modification to a file, and allow CAS-style operations: mod := stat_mod("/a") open_mod("/a", mod) read_mod("/a", mod) mod2 := write_mod("/a", ..., mod) // fails if /a modified since mod mod3 := chown_mod("/a", ..., mod2) // fails if modified since mod2 etc. All with their own pros and cons, of course, but definitely possible. I'm just curious if there's a specific implementation that's gaining momentum.
- qwertyuiop924 10y agookay. neat. why did you use Go syntax?
- jandrese 10y agoWhat's so strange about stat() on the file after you open it? It's a security check against a known and abused exploit and it's not even hard to do. This seems like a simple fix to the problem. Just make sure you fstat() the file descriptor and not stat() the file to avoid a second race condition where a malicious actor undoes their attack immediately after you open() the file to hide the evidence.
- gpvos 10y agofstat is a more recent system call than stat, so old code bases do not use it, and mail.local is quite ancient. Although a quick lookup tells me that it appeared in 4.3BSD-Tahoe (June 1988) and SysVR4 (October 1988), so one would have expected all reasonable distributions to have gotten with the program by now.
- jandrese 10y agoA little digging suggests that mail.local appeared in Version 7 Unix from 1979, so it is not a surprise that it doesn't include a syscall invented 9 years later. Still, that syscall is 28 years old now. It's kind of embarrassing that nobody has gone though and checked for ancient and obvious privilege escalation issues like this. Or I guess they have, but on different OSes. This is one big downside to fragmentation, getting fixes distributed to all of the fragments.
- caf 10y agofstat() isn't the complete fix here - it doesn't protect you against opening/creating an unintended file through a symlink, for which you need O_NOFOLLOW (which is a bit more recent).
- qwertyuiop924 10y agoI had no idea if it was common or not, which is why I said I was unsure.
- tedunangst 10y agoThis depends entirely on what you want to do. You can open files with O_EXCL. You can atomically rename() them. You can use libsqlite3 which provides an abstraction for many operations.
- benmmurphy 10y agoin this situation you can do a stat on the file descriptor after you have opened the file to ensure that it is not a symlink. some of the races with the filesystem can be fixed using the file descriptor family of functions instead of the path family of functions.
- leni536 10y agoHow do you get a fd for the link and not the linked file?
- loeg 10y agoThere is the non-standard O_NOFOLLOW.
- adrianratnapala 10y agoCan you not do an ordinary open without O_NOFOLLOW and then lstat the filename?
- DblPlusUngood 10y agoWhy do you say "non-standard"? O_NOFOLLOW is specified by POSIX.
- loeg 10y ago
- sigil 10y ago> that offers transactions, or "compare and swap"-type operations? "Things UNIX can do atomically" https://rcrowley.org/2010/01/06/things-unix-can-do-atomically.html https://rcrowley.org/2010/01/06/things-unix-can-do-atomicall...
- quotemstr 10y ago> Is there a reliable, widespread and usable successor to the posix API that offers transactions, or "compare and swap"-type operations? https://msdn.microsoft.com/en-us/library/windows/desktop/aa363764(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
- asveikau 10y agoWaiting for someone to say this wouldn't be possible in rust. Spoiler alert: it would be.
- sdegutis 10y agoI don't think anyone's claiming you can't use your system's API in Rust, to do things like gain root privileges. Just that Rust makes memory bugs and race conditions near impossible.
- citizenterminal 10y agoRust doesn't prevent race conditions–it prevents data races. https://doc.rust-lang.org/nomicon/races.html https://doc.rust-lang.org/nomicon/races.html
- gue5t 10y agoTrue. The indeterminism provided by letting the OS and cache coherence reorder thread execution and memory accesses in response to external factors is one of the reasons parallel execution can provide speedups in practice, so benign races are important to permit. But this is a data race: the mail program performs a check on some data (in the filesystem) to establish some precondition, and then a concurrent thread of execution (another process) mutates that data before the mail program acts on its (now invalidated) belief. LLVM wouldn't call it a data race, because to LLVM all system calls are essentially opaque. But if the filesystem only existed within your process, and were written in idiomatic Rust style, using structs, borrowing, mutability, and lifetimes, then the analogous bug (two threads racing and mutating the filesystem at once) would be prevented. The real villain here is shared mutable state; if UNIX had been written by skilled Rust progammers, it would with any luck have some abstraction that provides more isolation and less potential for interference than the filesystem.
- steveklabnik 10y ago> on some data (in the filesystem) Traditionally, "data race" only refers to memory, not other forms of resources. So I agree with you that this is like a data race, but would dispute that it's actually a data race.
- a1k0n 10y agoWhy Korn shell? Is there something special about the way it handles being run w/ a setuid bit?
- mmarx 10y agoIt's one of the shells in the NetBSD base system. bash, for example, is not guaranteed to be installed.
- Retr0spectrum 10y agoBash drops privileges by default.
- 1_2__3 10y agoAuthor is correct, this is an old bug. I can confirm it was at least present in SunOS 4.1.3, because I worked for an ISP back in the 90s that offered shell accounts and we found evidence of attempts all over the place (when you "lose" the race a root-owned file is created, so unless you ever "win" and clean up after yourself there's evidence of your attempt).