10 ms·

Rust and Nix = easier Unix systems programming

255 points by kalmar 10y ago

subway 10y ago
Neat library, way too already-overloaded name.
- Ericson2314 10y ago
  Yeah as a Rust and NixOS fan, I was let down.
bigger_cheese 10y ago
Minor nit pick but don't you typically do something like this in C pid_t childPid; switch (childPid = fork()) { case -1: ... /error handling /; case 0: ... /Child Specific/ default: sleep (5); } edit - seems to mangle formatting but something like that seems fairly clean.
- bluejekyll 10y ago
  It's not that you can't write the code properly in C, it's that the language and function interfaces give you no handrails to protect you from a mistake. And in this case, doing the wrong thing could be disastrous to your system, especially if you had to run it as root. Great intro to Rust and Unix post!
- MaulingMonkey 10y ago
  Typically, I would hope so. A good code review process should typically catch when it isn't done like this. In context, "typically" means your devops people get to work on Christmas to patch a critical CVE being actively exploited in the wild pissing off all your customers that one time someone didn't do the typical pattern anywhere in your codebase or the codebases of any of your 3rd party libaries, frameworks, applications... Where "didn't do the typical pattern" might be if conditions as shown, or forgetting the "case -1", or missed a key and typed "case 1", or elided the "break;" from the case above (I note no "break;"s in your switch ;)), or didn't rtfm closely enough to see -1 was a special exit code, or mis-assumed kill(-1,...) was a noop, or ...
- cosmicexplorer 10y ago
  yes, but quite a lot of tutorials i've seen / intro to systems courses seem to think it looks "cleaner" to use an if statement; switch is definitely the move here but i've seen the if version quite a lot
- laughinghan 10y ago
  You're missing the point in actually a really important way. Nobody is claiming that C makes it impossible to cleanly do the right thing—obviously the whole world runs on C. The point is that nothing about the C language, libraries, or toolchain discourage the example given in the blogpost compared to your more correct code. Unless you remember exactly the right details from the manpages, there's nothing about the example in the blogpost that's less natural to write than your more correct code. (And people do forget those details: http://rachelbythebay.com/w/2014/08/19/fork/ http://rachelbythebay.com/w/2014/08/19/fork/ ) By contrast, as illustrated in the blogpost, the most natural way to do the same thing in Rust turns out to be the more correct thing. If you wanted the bad behavior to happen, you'd have to go out of your way to pass -1 to kill(). Hence in this example, Rust's design is an improvement. It's great that C gives you enough rope to hang yourself with, but it's even better if tying yourself to things safely is easy, and to hang yourself you have to really go out of your way.
SixSigma 10y ago
In reliability theory "X failed" is a poor error message. What we want to know is which failure mode has been triggered. The function of kill is to kill a given pid, so there are two failure modes : "the pid didn't exist" or "the pid didn't die"
- LukeShu 10y ago
  It's poorly named, because the function of kill isn't to kill a given PID; it's to send a signal to a given PID.
- meanary 10y ago
  Is there a logology/discipline where error messages are treated to medical-legal terminology?
  - SixSigma 10y ago
    I got exposed to reliability theory in Logistics Engineering. Mostly: Reliability Centered Maintenance https://en.wikipedia.org/wiki/Reliability-centered_maintenance https://en.wikipedia.org/wiki/Reliability-centered_maintenan...
- Sean1708 10y ago
  "kill failed" isn't actually the message which is printed. What you get is thread '<main>' panicked at 'kill failed: Sys(ESRCH)', ../src/libcore/result.rs:746 So you know that it failed because of ESRCH (no such process).
  - SixSigma 10y ago
    Ah, that makes much more sense, thanks
- masklinn 10y ago
  > The function of kill is to kill a given pid 1. there are more failure modes than these (POSIX kill(2) can set 3 different errnos) 2. kill(2) signals process, it doesn't usually kill them (let alone kill them outright in such a way that this information could ever be returned) 3. kill(2) can signal process sets of cardinality > 1 4. the `unwrap` panic message will print the unwrapped value, which includes the ERRNO
etrain 10y ago
type systems are great.
sergiolp 10y ago
I can't help but think they're trying to fix something that isn't broken at all. Adding new abstraction layers rarely helps when doing systems programming. You (as in "the developer") want to be as near to the machine as possible. C does this pretty well. Perhaps I'm just getting old :-(
- draven 10y ago
  In this case it seems like a very thin wrapper that leverages the type system to allow catching a whole class of errors at compile time, like using exhaustiveness checks to make sure a function call handles all possible return values. I think the small overhead is well worth it. The original API is not "broken" per se, it's just limited by the language features ("magical" return values vs. tagged unions or whatever they're called in Rust, I don't remember.)
  - wtallis 10y ago
    It's not even clear to me that there's any overhead to the Rust version. Checking error codes that should be checked isn't overhead. Checking them inefficiently would be overhead, but the Rust version looks like it should compile down to something pretty similar to what the equivalent C switch blocks would produce.
    - sergiolp 10y ago
      I'm not talking about the efficency of the resulting binary, but the "distance" from what the programmer is thinking, to what the machine will really do. Compiler optimizations aside, C does a pretty good job at this. It's way more efficient than writing assembly, but your still basically just moving memory around, while doing some arithmethic. Easy to understand in "machine" terms. Of course, this is only relevant when you're doing low-level stuff, like kernel or drivers programming. For the userland, Rust really looks like a nice language (I've played with it just a bit), and I'd be really happy if it pushes C++ away ;-)
      - iopq 10y ago
geocar 10y ago
I was very confused. I thought this had something to do with Rust and nix[1]. [1]: https://nixos.org/nix/ https://nixos.org/nix/
- tomp 10y ago
  Me too! It's actually about a library, nix, whose mission is "to provide ‘Rust friendly bindings to *nix APIs’". https://github.com/nix-rust/nix https://github.com/nix-rust/nix
- Keats 10y ago
  Same here. To be fair the Nix package manager has a name that is actively harming its growth since most people think *nix as in unix when you say nix.
  - ris 10y ago
    Google does too to an extent, making troubleshooting more of a pain than it should be.
- k__ 10y ago
  Same here. I thought "A match made in heaven" But it will probably never happen, because Cargo is too good, haha
  - wizeman 10y ago
    What do you mean, it will never happen? The nix package manager has (admittedly, undocumented) support for Rust / Cargo projects.
  - lolidaisuki 10y ago
    Cargo is bad. I hate how every language has to have it's own package manager. It's worse than just reinventing the wheel because it's actively harming everyone by requiring them to have a million package managers installed and know how to use them all. And often times these language specific package managers are insecure and lack many features. If we had just a few big package managers like nix and guix people could just package for those and we could have everything in one place. Projects like node, perl, python, rust etc. don't need to be the only people in control of their package managers. They could just host language specific repos or something.
larozin 10y ago
We have switched to Nix as internal dependency manager for our C++ project. It is really exciting! No more "after commit XXX you need to (re)build/update YYY with ZZZ". Developers just type `nix-shell` and get sane guaranted to work environment on their local machines corresponding to git HEAD. If we need to add or patch dependency we just edit and commit nix file. And if developer need to rollback to old commit/branch it will get old/custom environment from cache without submodule rebuilds.
- comex 10y ago
  That's pretty cool... but has nothing to do with the different project named "nix" discussed in the post.
zuzun 10y ago
I always find it a bit unfair when I see sloppy C programs used for shock value. What if the Rust developer uses fork().unwrap_or(default_value) in a hurry, or writes if let Some(child) = fork() { do_only_child_stuff(); } else { do_only_parent_stuff(); } or if let Some(ForkResult::Child) = fork() { do_only_child_stuff(); } else { do_only_parent_stuff(); } Now, if you're about to tell me that the examples above are totally stupid and no developer would do such a thing, then you know how I feel about the sloppy C versions. Doing a system call and not checking for error is totally stupid as well. By the way, you can also write your own wrapper functions in C, that transform the return value into something like struct fork_status { enum { ERROR, PARENT, CHILD } state; int ret; }; Then Clang and GCC will warn you about missing switch cases. That said, the libc bindings in Rust are pretty low-level and a project that offers higher-level wrappers can be very helpful, so I hope my comment doesn't create the impression that I'm ripping on the project itself.
- masklinn 10y ago
  > What if the Rust developer uses fork().unwrap_or(default_value) in a hurry The point here is that the language's tools and APIs can significantly better drive the developer towards the safe/right solution, that's a large point of type theory and static type systems after all. In this case rust's type system is used to split out the various "result cases" and notify the developer upfront of the various cases to handle. The return type pretty much tells you how the function will behave and what you need to take care of as the caller. That aside, why would you unwrap_or(default_value) if you're in a hurry when unwrap() is shorter (and you can later grep for "unwrap()" to find dodgy/hurried code, whereas unwrap_or is a perfectly legitimate recovery strategy). > Now, if you're about to tell me that the examples above are totally stupid and no developer would do such a thing, then you know how I feel about the sloppy C versions. Doing a system call and not checking for error is totally stupid as well. The issue being that even though you have a compiled statically typed language it's of absolutely no help in "checking for error", and interactions between syscalls can be hard to predict, not checking for fork(2)'s error isn't the end of the world... until you pass its result to kill(2) for instance (it might also give strange results if you pass specific pids to waidpid)
justincormack 10y ago
I maintain LuaJIT syscall bindings https://github.com/justincormack/ljsyscall https://github.com/justincormack/ljsyscall - they cover quite a lot, namespaces, netlink and so on. I spent quite a bit of time making them more intuitive than the raw bindings, with consistent error handling, also namespacing constants and so on. It is definitely useful to have these types of interfaces not in C.
- kalmar 10y ago
  This project looks really cool! I'm very curious to find out more about how you make sure constants are correct across platforms and architectures. I will be poking around!
  - justincormack 10y ago
    There are a bunch of tests but the whole Linux ABI spec is a mess.
superobserver 10y ago
This gets me thinking how awesome it would be to have functional programming on *nix systems, like Haskell (specifically). At least then it might be forcibly designed to be made more useful and ultimately get more people on board. One can dream.
- GhotiFish 10y ago
  Kinda like turtle! https://hackage.haskell.org/package/turtle https://hackage.haskell.org/package/turtle Oh by the way, I accidentally hit downvote on your post and HN doesn't let me undo that action... I was just trying to hide it! Sorry!
  - superobserver 10y ago
    Hey, thanks! Hadn't heard of turtle before, I don't believe. Time to install it on my crouton chroot and see what it can do. :)
peterwwillis 10y ago
"the return value is conveying three different things all at once. [...] That’s a lot of information for one poor little pid_t—usually a 32-bit integer—to convey!" Someone never had to bit-pack their programs to save memory, disk space, or bandwidth. In fact, it's a huge waste of memory; if you only need 3 bits, a 'char' would have sufficed. Saves 24 bits! Of course, we could use nibbles to make data structures where the fork return value only takes up 3 bits instead of a whole byte, but that could be considered micro-optimizing. (the compiler may do this for us anyway, though)
- Rusky 10y ago
  The return value is going to stay in a register the whole time anyway, so a char won't save you anything. But regardless, the point of that sentence is nothing to do with memory usage, but with semantics. Whether you or the compiler packs all the information into 3 bits or 3 words, that's fine, as long as the language helps you distinguish the parts.
ZephyrP 10y ago
I feel there are more promising options for a name than "Nix".
bogomipz 10y ago
The term NIX is becoming a bit overloaded - we've got the Nix package manger which run on NixOS, Nix the Rust library all of which can run on most 'Nix systems.

255 points by kalmar 10y ago

subway 10y ago

Neat library, way too already-overloaded name.

Ericson2314 10y ago
Yeah as a Rust and NixOS fan, I was let down.

bigger_cheese 10y ago

Minor nit pick but don't you typically do something like this in C pid_t childPid; switch (childPid = fork()) { case -1: ... /error handling /; case 0: ... /Child Specific/ default: sleep (5); } edit - seems to mangle formatting but something like that seems fairly clean.

bluejekyll 10y ago
It's not that you can't write the code properly in C, it's that the language and function interfaces give you no handrails to protect you from a mistake. And in this case, doing the wrong thing could be disastrous to your system, especially if you had to run it as root. Great intro to Rust and Unix post!
MaulingMonkey 10y ago
Typically, I would hope so. A good code review process should typically catch when it isn't done like this. In context, "typically" means your devops people get to work on Christmas to patch a critical CVE being actively exploited in the wild pissing off all your customers that one time someone didn't do the typical pattern anywhere in your codebase or the codebases of any of your 3rd party libaries, frameworks, applications... Where "didn't do the typical pattern" might be if conditions as shown, or forgetting the "case -1", or missed a key and typed "case 1", or elided the "break;" from the case above (I note no "break;"s in your switch ;)), or didn't rtfm closely enough to see -1 was a special exit code, or mis-assumed kill(-1,...) was a noop, or ...
cosmicexplorer 10y ago
yes, but quite a lot of tutorials i've seen / intro to systems courses seem to think it looks "cleaner" to use an if statement; switch is definitely the move here but i've seen the if version quite a lot
laughinghan 10y ago
You're missing the point in actually a really important way. Nobody is claiming that C makes it impossible to cleanly do the right thing—obviously the whole world runs on C. The point is that nothing about the C language, libraries, or toolchain discourage the example given in the blogpost compared to your more correct code. Unless you remember exactly the right details from the manpages, there's nothing about the example in the blogpost that's less natural to write than your more correct code. (And people do forget those details: http://rachelbythebay.com/w/2014/08/19/fork/ http://rachelbythebay.com/w/2014/08/19/fork/ ) By contrast, as illustrated in the blogpost, the most natural way to do the same thing in Rust turns out to be the more correct thing. If you wanted the bad behavior to happen, you'd have to go out of your way to pass -1 to kill(). Hence in this example, Rust's design is an improvement. It's great that C gives you enough rope to hang yourself with, but it's even better if tying yourself to things safely is easy, and to hang yourself you have to really go out of your way.

SixSigma 10y ago

In reliability theory "X failed" is a poor error message. What we want to know is which failure mode has been triggered. The function of kill is to kill a given pid, so there are two failure modes : "the pid didn't exist" or "the pid didn't die"

LukeShu 10y ago
It's poorly named, because the function of kill isn't to kill a given PID; it's to send a signal to a given PID.
meanary 10y ago
Is there a logology/discipline where error messages are treated to medical-legal terminology?
- SixSigma 10y ago
  I got exposed to reliability theory in Logistics Engineering. Mostly: Reliability Centered Maintenance https://en.wikipedia.org/wiki/Reliability-centered_maintenance https://en.wikipedia.org/wiki/Reliability-centered_maintenan...
Sean1708 10y ago
"kill failed" isn't actually the message which is printed. What you get is thread '<main>' panicked at 'kill failed: Sys(ESRCH)', ../src/libcore/result.rs:746 So you know that it failed because of ESRCH (no such process).
- SixSigma 10y ago
  Ah, that makes much more sense, thanks
masklinn 10y ago
> The function of kill is to kill a given pid 1. there are more failure modes than these (POSIX kill(2) can set 3 different errnos) 2. kill(2) signals process, it doesn't usually kill them (let alone kill them outright in such a way that this information could ever be returned) 3. kill(2) can signal process sets of cardinality > 1 4. the `unwrap` panic message will print the unwrapped value, which includes the ERRNO

etrain 10y ago

type systems are great.

sergiolp 10y ago

I can't help but think they're trying to fix something that isn't broken at all. Adding new abstraction layers rarely helps when doing systems programming. You (as in "the developer") want to be as near to the machine as possible. C does this pretty well. Perhaps I'm just getting old :-(

draven 10y ago
In this case it seems like a very thin wrapper that leverages the type system to allow catching a whole class of errors at compile time, like using exhaustiveness checks to make sure a function call handles all possible return values. I think the small overhead is well worth it. The original API is not "broken" per se, it's just limited by the language features ("magical" return values vs. tagged unions or whatever they're called in Rust, I don't remember.)
- wtallis 10y ago
  It's not even clear to me that there's any overhead to the Rust version. Checking error codes that should be checked isn't overhead. Checking them inefficiently would be overhead, but the Rust version looks like it should compile down to something pretty similar to what the equivalent C switch blocks would produce.
  - sergiolp 10y ago
    I'm not talking about the efficency of the resulting binary, but the "distance" from what the programmer is thinking, to what the machine will really do. Compiler optimizations aside, C does a pretty good job at this. It's way more efficient than writing assembly, but your still basically just moving memory around, while doing some arithmethic. Easy to understand in "machine" terms. Of course, this is only relevant when you're doing low-level stuff, like kernel or drivers programming. For the userland, Rust really looks like a nice language (I've played with it just a bit), and I'd be really happy if it pushes C++ away ;-)
    - iopq 10y ago

geocar 10y ago

I was very confused. I thought this had something to do with Rust and nix[1]. [1]: https://nixos.org/nix/ https://nixos.org/nix/

tomp 10y ago
Me too! It's actually about a library, nix, whose mission is "to provide ‘Rust friendly bindings to *nix APIs’". https://github.com/nix-rust/nix https://github.com/nix-rust/nix
Keats 10y ago
Same here. To be fair the Nix package manager has a name that is actively harming its growth since most people think *nix as in unix when you say nix.
- ris 10y ago
  Google does too to an extent, making troubleshooting more of a pain than it should be.
k__ 10y ago
Same here. I thought "A match made in heaven" But it will probably never happen, because Cargo is too good, haha
- wizeman 10y ago
  What do you mean, it will never happen? The nix package manager has (admittedly, undocumented) support for Rust / Cargo projects.
- lolidaisuki 10y ago
  Cargo is bad. I hate how every language has to have it's own package manager. It's worse than just reinventing the wheel because it's actively harming everyone by requiring them to have a million package managers installed and know how to use them all. And often times these language specific package managers are insecure and lack many features. If we had just a few big package managers like nix and guix people could just package for those and we could have everything in one place. Projects like node, perl, python, rust etc. don't need to be the only people in control of their package managers. They could just host language specific repos or something.

larozin 10y ago

We have switched to Nix as internal dependency manager for our C++ project. It is really exciting! No more "after commit XXX you need to (re)build/update YYY with ZZZ". Developers just type `nix-shell` and get sane guaranted to work environment on their local machines corresponding to git HEAD. If we need to add or patch dependency we just edit and commit nix file. And if developer need to rollback to old commit/branch it will get old/custom environment from cache without submodule rebuilds.

comex 10y ago
That's pretty cool... but has nothing to do with the different project named "nix" discussed in the post.

zuzun 10y ago

I always find it a bit unfair when I see sloppy C programs used for shock value. What if the Rust developer uses fork().unwrap_or(default_value) in a hurry, or writes if let Some(child) = fork() { do_only_child_stuff(); } else { do_only_parent_stuff(); } or if let Some(ForkResult::Child) = fork() { do_only_child_stuff(); } else { do_only_parent_stuff(); } Now, if you're about to tell me that the examples above are totally stupid and no developer would do such a thing, then you know how I feel about the sloppy C versions. Doing a system call and not checking for error is totally stupid as well. By the way, you can also write your own wrapper functions in C, that transform the return value into something like struct fork_status { enum { ERROR, PARENT, CHILD } state; int ret; }; Then Clang and GCC will warn you about missing switch cases. That said, the libc bindings in Rust are pretty low-level and a project that offers higher-level wrappers can be very helpful, so I hope my comment doesn't create the impression that I'm ripping on the project itself.

masklinn 10y ago
> What if the Rust developer uses fork().unwrap_or(default_value) in a hurry The point here is that the language's tools and APIs can significantly better drive the developer towards the safe/right solution, that's a large point of type theory and static type systems after all. In this case rust's type system is used to split out the various "result cases" and notify the developer upfront of the various cases to handle. The return type pretty much tells you how the function will behave and what you need to take care of as the caller. That aside, why would you unwrap_or(default_value) if you're in a hurry when unwrap() is shorter (and you can later grep for "unwrap()" to find dodgy/hurried code, whereas unwrap_or is a perfectly legitimate recovery strategy). > Now, if you're about to tell me that the examples above are totally stupid and no developer would do such a thing, then you know how I feel about the sloppy C versions. Doing a system call and not checking for error is totally stupid as well. The issue being that even though you have a compiled statically typed language it's of absolutely no help in "checking for error", and interactions between syscalls can be hard to predict, not checking for fork(2)'s error isn't the end of the world... until you pass its result to kill(2) for instance (it might also give strange results if you pass specific pids to waidpid)

justincormack 10y ago

I maintain LuaJIT syscall bindings https://github.com/justincormack/ljsyscall https://github.com/justincormack/ljsyscall - they cover quite a lot, namespaces, netlink and so on. I spent quite a bit of time making them more intuitive than the raw bindings, with consistent error handling, also namespacing constants and so on. It is definitely useful to have these types of interfaces not in C.

kalmar 10y ago
This project looks really cool! I'm very curious to find out more about how you make sure constants are correct across platforms and architectures. I will be poking around!
- justincormack 10y ago
  There are a bunch of tests but the whole Linux ABI spec is a mess.

superobserver 10y ago

This gets me thinking how awesome it would be to have functional programming on *nix systems, like Haskell (specifically). At least then it might be forcibly designed to be made more useful and ultimately get more people on board. One can dream.

GhotiFish 10y ago
Kinda like turtle! https://hackage.haskell.org/package/turtle https://hackage.haskell.org/package/turtle Oh by the way, I accidentally hit downvote on your post and HN doesn't let me undo that action... I was just trying to hide it! Sorry!
- superobserver 10y ago
  Hey, thanks! Hadn't heard of turtle before, I don't believe. Time to install it on my crouton chroot and see what it can do. :)

peterwwillis 10y ago

"the return value is conveying three different things all at once. [...] That’s a lot of information for one poor little pid_t—usually a 32-bit integer—to convey!" Someone never had to bit-pack their programs to save memory, disk space, or bandwidth. In fact, it's a huge waste of memory; if you only need 3 bits, a 'char' would have sufficed. Saves 24 bits! Of course, we could use nibbles to make data structures where the fork return value only takes up 3 bits instead of a whole byte, but that could be considered micro-optimizing. (the compiler may do this for us anyway, though)

Rusky 10y ago
The return value is going to stay in a register the whole time anyway, so a char won't save you anything. But regardless, the point of that sentence is nothing to do with memory usage, but with semantics. Whether you or the compiler packs all the information into 3 bits or 3 words, that's fine, as long as the language helps you distinguish the parts.

ZephyrP 10y ago

I feel there are more promising options for a name than "Nix".

bogomipz 10y ago

The term NIX is becoming a bit overloaded - we've got the Nix package manger which run on NixOS, Nix the Rust library all of which can run on most 'Nix systems.