7 ms·

Hardening Compiler Flags for NixOS

113 points by ivank 10y ago

cstrahan 10y ago
Awesome work, Franz et al!
- frio 10y ago
  Absolutely. NixOS is awesome. The lack of decent hardening by default was the only thing preventing me from using it more confidently. I'm thoroughly impressed by the many people who have worked hard to push this through. Thanks so much to everyone who's worked on this!
colemickens 10y ago
Excellent! Really, anything to propel Nix(OS) or Guix(SD) forward gets a huge thumbs up in my book. Nix is one of those things that I explain and then have people telling me I'm wrong and/or an idiot because they can't believe that you can have a distribution based on building packages from source, backed by a binary cache. If you think Snappy or Flatpak/Xdg-App are cool, you owe it to yourself to look at Nix/Guix, which provide a superset of the features provided by Snappy. And then some. NixOS + NixOps means I can have a single declarative file that describes the state of my cloud VMs, cloud load balancer, cloud Traffic Manager... plus the exact software on the VMs (all the way down to the exact checkout of gcc used to compile everything and the nginx configurations)... literally everything is snapshotted and revertable. If it weren't for a systemd-related bug, I'd have a one-command way of getting a Kubernetes cluster booted in Azure via NixOS, where upgrading a cluster would simply require editting a single file and re-running the nixops tool. I wish more people knew about NixOS. You know how everyone writes pie-in-the-sky blogposts about the "next-gen" Node.JS package manager? Yeah, well, Nix already packages NodeJS libraries/apps, plus Rust libraries/apps, plus Go libraries/apps, plus Python libraries/apps, etc, etc.
- bojo 10y ago
  > If it weren't for a systemd-related bug, I'd have a one-command way of getting a Kubernetes cluster booted Damn, could you explain what this is specifically? Running a Kubernetes cluster on NixOS was next on my list of architectures to test.
  - colemickens 10y ago
    The networking related services (Flannel) weren't waiting for the network to be available - I think I needed to configure one of the meta units that is meant to help block until network is ready. Now that there is Azure cloudprovider support in Kubernetes, there's no need for Flannel, so it's possible that this could be made to work... I ditched NixOS ~4 months ago; I think I had some patches on top of nixpkgs for updating Kube/flannel/etc, and then I have a branch of `nixops` that has the Kubernetes example. Might be a fun project to pick back up some weekend... I had a whole bunch of commits on top of nixpkgs, updating Flannel/Kubernetes and fixing up the Kubernetes module: https://github.com/colemickens/nixpkgs/commits/cmpkgs https://github.com/colemickens/nixpkgs/commits/cmpkgs And then on nixops, a branch that adds in the Kubernetes example. Keep in mind that it doesn't quite work right, and things are possibly broken since my nixpkgs fork is so stale at this point. https://github.com/colemickens/nixops/commit/9e97fd4166e7062ddc510cd36b16228d112c815d https://github.com/colemickens/nixops/commit/9e97fd4166e7062...
- aidenn0 10y ago
  "If it weren't for a systemd-related bug..." I've started so many sentences with those exact words, and it kind of saddens me. I'm still hoping that the problems with systemd aren't fundamental to its architecture, and it just needs some more polishing. It certainly is incredibly ambitious as it aims to replace on the order of a dozen different pieces of software, so some hiccups are to be expected.
  - digi_owl 10y ago
    Nah, its the mentality within the project. One that i think can be summed up as "code, compile, push, repeat". More and more it seems that the incoming generation treats everything like a web site that they can change at whim. Likely because they have had the internet and source repositories the whole time, and thus do not know the value of a truly stable and maintained system.
- boothead 10y ago
  I'd very much like to hear about using NixOS to provision kubernetes or mesos too.
  - colemickens 10y ago
    I added some info here: https://news.ycombinator.com/item?id=12388571 https://news.ycombinator.com/item?id=12388571
- cstrahan 10y ago
  > Nix is one of those things that I explain and then have people telling me I'm wrong and/or an idiot because they can't believe that you can have a distribution based on building packages from source, backed by a binary cache. Hehe, same here. The number of times I'e had people tell me "uh, no, your package-manager/OS can't do that because <reasons that assume FHS dumping ground of libs/headers and global, mutable config state>" -- too many to count :). I'm still trying to find a 30 second sales pitch that manages to get through people's preconceived notions. If I can get someone to look over my shoulder at my terminal for about 5 minutes, that usually blows them away, but that's not always an option (like here in the comments section, vs coworkers at the office).
AstralStorm 10y ago
Talking about stack protector providing any sort of guarantee makes me chuckle. Both this and fortify are relatively easy to work around. PIC/PIE are the real protection, and but still nowhere near foolproof. Also new Address Sanitizer is not mentioned.
- netheril96 10y ago
  Address Sanitizer slows down execution a lot. It should only used for testing purposes, the same as -O0.
  - masterleep 10y ago
    What if you care more about unsafe pointer access than speed?
    - anon1385 10y ago
      Address Sanitizer isn't designed to be used in production. See this thread: http://seclists.org/oss-sec/2016/q1/363 http://seclists.org/oss-sec/2016/q1/363 ASAN is great at detecting many unintentional memory errors, but it's not designed to thwart malicious attacks. If you need a specific example of ASAN binary being exploited then see: http://int3pids.blogspot.ch/2015/04/confidence-2015-teaser-quarantine-write.html http://int3pids.blogspot.ch/2015/04/confidence-2015-teaser-q...
      - AstralStorm 10y ago
        Correct, but it still prevents a large set of attacks. Instead, we need an equivalent that is more focused on security than bug detection. CPI is the mechanism for the future with somewhere below 10% overhead.
  - AstralStorm 10y ago
    This just shows how much unchecked access is done in any given application. Asan is still much faster than for instance Valgrind. Plus there are some parts that are actually static analysis, those have no impact on runtime.
oconnore 10y ago
If there are any other roadblocks keeping you from using Nix in your infrastructure: keep in mind you can overlay Nix on top of another distribution. This way you can get a more mature base system and still have all the benefits of Nix available when you want it.
- Joof 10y ago
  Good to know. Last I checked it had trouble with steam which kept me from trying it on my home machine.
  - Poldi 10y ago
    Just tested steam last week without any issues :)
    - Joof 10y ago
      Good to hear, I may have to give it another shot.
ramblenode 10y ago
I am glad to see this finally get merged--great work! For those who are curious about the declarative OS and package ecosystem offered by NixOS, the newest stable release (16.09) is coming in the following weeks.
- k__ 10y ago
  Is the new, more user friendly, CLI in 16.09?
  - ramblenode 10y ago
    Judging by the commits I've seen, I'm guessing no. Not sure what the time frame on this is, but it seems a lot of people are looking forward to it.
hannob 10y ago
anyone knows why they disable -pie by default? I know a lot of others do so as well, but this should be really pushed forward and be the default. Full ASLR is most likely the strongest exploit mitigation mechanism that's available right now.
- grhmc 10y ago
  We disabled -pie so we could get it merged in. We would like pie enabled by default (for the reasons you say) but didn't want perfect be the enemy of good here.

113 points by ivank 10y ago

cstrahan 10y ago

Awesome work, Franz et al!

frio 10y ago
Absolutely. NixOS is awesome. The lack of decent hardening by default was the only thing preventing me from using it more confidently. I'm thoroughly impressed by the many people who have worked hard to push this through. Thanks so much to everyone who's worked on this!

colemickens 10y ago

Excellent! Really, anything to propel Nix(OS) or Guix(SD) forward gets a huge thumbs up in my book. Nix is one of those things that I explain and then have people telling me I'm wrong and/or an idiot because they can't believe that you can have a distribution based on building packages from source, backed by a binary cache. If you think Snappy or Flatpak/Xdg-App are cool, you owe it to yourself to look at Nix/Guix, which provide a superset of the features provided by Snappy. And then some. NixOS + NixOps means I can have a single declarative file that describes the state of my cloud VMs, cloud load balancer, cloud Traffic Manager... plus the exact software on the VMs (all the way down to the exact checkout of gcc used to compile everything and the nginx configurations)... literally everything is snapshotted and revertable. If it weren't for a systemd-related bug, I'd have a one-command way of getting a Kubernetes cluster booted in Azure via NixOS, where upgrading a cluster would simply require editting a single file and re-running the nixops tool. I wish more people knew about NixOS. You know how everyone writes pie-in-the-sky blogposts about the "next-gen" Node.JS package manager? Yeah, well, Nix already packages NodeJS libraries/apps, plus Rust libraries/apps, plus Go libraries/apps, plus Python libraries/apps, etc, etc.

bojo 10y ago
> If it weren't for a systemd-related bug, I'd have a one-command way of getting a Kubernetes cluster booted Damn, could you explain what this is specifically? Running a Kubernetes cluster on NixOS was next on my list of architectures to test.
- colemickens 10y ago
  The networking related services (Flannel) weren't waiting for the network to be available - I think I needed to configure one of the meta units that is meant to help block until network is ready. Now that there is Azure cloudprovider support in Kubernetes, there's no need for Flannel, so it's possible that this could be made to work... I ditched NixOS ~4 months ago; I think I had some patches on top of nixpkgs for updating Kube/flannel/etc, and then I have a branch of `nixops` that has the Kubernetes example. Might be a fun project to pick back up some weekend... I had a whole bunch of commits on top of nixpkgs, updating Flannel/Kubernetes and fixing up the Kubernetes module: https://github.com/colemickens/nixpkgs/commits/cmpkgs https://github.com/colemickens/nixpkgs/commits/cmpkgs And then on nixops, a branch that adds in the Kubernetes example. Keep in mind that it doesn't quite work right, and things are possibly broken since my nixpkgs fork is so stale at this point. https://github.com/colemickens/nixops/commit/9e97fd4166e7062ddc510cd36b16228d112c815d https://github.com/colemickens/nixops/commit/9e97fd4166e7062...
aidenn0 10y ago
"If it weren't for a systemd-related bug..." I've started so many sentences with those exact words, and it kind of saddens me. I'm still hoping that the problems with systemd aren't fundamental to its architecture, and it just needs some more polishing. It certainly is incredibly ambitious as it aims to replace on the order of a dozen different pieces of software, so some hiccups are to be expected.
- digi_owl 10y ago
  Nah, its the mentality within the project. One that i think can be summed up as "code, compile, push, repeat". More and more it seems that the incoming generation treats everything like a web site that they can change at whim. Likely because they have had the internet and source repositories the whole time, and thus do not know the value of a truly stable and maintained system.
boothead 10y ago
I'd very much like to hear about using NixOS to provision kubernetes or mesos too.
- colemickens 10y ago
  I added some info here: https://news.ycombinator.com/item?id=12388571 https://news.ycombinator.com/item?id=12388571
cstrahan 10y ago
> Nix is one of those things that I explain and then have people telling me I'm wrong and/or an idiot because they can't believe that you can have a distribution based on building packages from source, backed by a binary cache. Hehe, same here. The number of times I'e had people tell me "uh, no, your package-manager/OS can't do that because <reasons that assume FHS dumping ground of libs/headers and global, mutable config state>" -- too many to count :). I'm still trying to find a 30 second sales pitch that manages to get through people's preconceived notions. If I can get someone to look over my shoulder at my terminal for about 5 minutes, that usually blows them away, but that's not always an option (like here in the comments section, vs coworkers at the office).

AstralStorm 10y ago

Talking about stack protector providing any sort of guarantee makes me chuckle. Both this and fortify are relatively easy to work around. PIC/PIE are the real protection, and but still nowhere near foolproof. Also new Address Sanitizer is not mentioned.

netheril96 10y ago
Address Sanitizer slows down execution a lot. It should only used for testing purposes, the same as -O0.
- masterleep 10y ago
  What if you care more about unsafe pointer access than speed?
  - anon1385 10y ago
    Address Sanitizer isn't designed to be used in production. See this thread: http://seclists.org/oss-sec/2016/q1/363 http://seclists.org/oss-sec/2016/q1/363 ASAN is great at detecting many unintentional memory errors, but it's not designed to thwart malicious attacks. If you need a specific example of ASAN binary being exploited then see: http://int3pids.blogspot.ch/2015/04/confidence-2015-teaser-quarantine-write.html http://int3pids.blogspot.ch/2015/04/confidence-2015-teaser-q...
    - AstralStorm 10y ago
      Correct, but it still prevents a large set of attacks. Instead, we need an equivalent that is more focused on security than bug detection. CPI is the mechanism for the future with somewhere below 10% overhead.
- AstralStorm 10y ago
  This just shows how much unchecked access is done in any given application. Asan is still much faster than for instance Valgrind. Plus there are some parts that are actually static analysis, those have no impact on runtime.

oconnore 10y ago

If there are any other roadblocks keeping you from using Nix in your infrastructure: keep in mind you can overlay Nix on top of another distribution. This way you can get a more mature base system and still have all the benefits of Nix available when you want it.

Joof 10y ago
Good to know. Last I checked it had trouble with steam which kept me from trying it on my home machine.
- Poldi 10y ago
  Just tested steam last week without any issues :)
  - Joof 10y ago
    Good to hear, I may have to give it another shot.

ramblenode 10y ago

I am glad to see this finally get merged--great work! For those who are curious about the declarative OS and package ecosystem offered by NixOS, the newest stable release (16.09) is coming in the following weeks.

k__ 10y ago
Is the new, more user friendly, CLI in 16.09?
- ramblenode 10y ago
  Judging by the commits I've seen, I'm guessing no. Not sure what the time frame on this is, but it seems a lot of people are looking forward to it.

hannob 10y ago

anyone knows why they disable -pie by default? I know a lot of others do so as well, but this should be really pushed forward and be the default. Full ASLR is most likely the strongest exploit mitigation mechanism that's available right now.

grhmc 10y ago
We disabled -pie so we could get it merged in. We would like pie enabled by default (for the reasons you say) but didn't want perfect be the enemy of good here.