5 ms·

Security challenges for the Qubes build process

64 points by kkl 10y ago

d33 10y ago
I heard that Qubes is riddled with undocumented scripts that have no automated tests for them. Does anybody know how true is that?
- otoburb 10y ago
  Qubes is open source[1] (including the Windows capabilities[2]) so your question should be fairly easy to prove/disprove yourself. I haven't delved into the source of any Qubes scripts or unit tests, but if one were to act on the rumours you heard perhaps they'd start with the Qubes Builder Scripts referenced in the article. If the Qubes team is able to implement a true append-only logging for their development+build process then perhaps any concerns around undocumented, unknown or untested scripts could be somewhat assuaged. [1] https://github.com/QubesOS/ https://github.com/QubesOS/ [2] https://www.qubes-os.org/news/2016/01/27/windows-tools-open-source/ https://www.qubes-os.org/news/2016/01/27/windows-tools-open-...
- kakwa_ 10y ago
  Just in case, I'm reposting my message from https://news.ycombinator.com/item?id=11652940 https://news.ycombinator.com/item?id=11652940 On the workstation part, it recommends QubesOS. Am I the only one who is skeptical about it? From what I saw superficially reading their source code, there are some frightening stuff going on: * tons of C code with nearly zero unit tests, same with the python code * lots glue in form of bash or python scripts * some not so beautiful stuff like: - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-firewall#L29 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (kill -9 on a daemon...) - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-firewall#L18 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (a daemon is a little bit more than an exe launched with '&' - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-core-appvm#L34 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (changing a config file in an init script, humm, weird...) - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-core-appvm#L32 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (starting a service inside the init of another service...) - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-qrexec-agent#L13 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... ("logging" with stderr redirection in a file) And it's just the init scripts... I'm too lazy to take a look further inside the C or python stuff. IMHO, as a proof of concept, it's interesting, as a finished, reliable and secure OS, it's frightening...
  - d33 10y ago
    Ah, yes, it's your comment that I was referring to.
  - nickpsecurity 10y ago
    It's hall-marks of a low-assurance, software project hacked together to get it working with OSS components. There could be strong review in scripts or C code as the team are talented coders and breakers from what I've read. The minimum in high-assurance security... even medium-assurance... was that every feature was tested for proper behavior during successful execution and failures. Plus, a description of why it's there that traces to a functional or security requirements. That's so evaluators can spot unnecessary code that might be a backdoor or just dead code. Python for those scripts is questionable, too, as it's too complex to analyze with known principles of security engineering. It should be a 3GL that maps well to hardware with easy way to spot code-level flaws or automated them out (esp static analysis). You can abstract it, even macro it, so long as final code can be subjected to whatever eyeballs or tools are out there with ease. Non-security-critical stuff can be written in something like Python, though.
- j_s 10y ago
  I think this is an important point based on the following taken from the article: our script for the verify-sources target in the Xen component contained a silly bug that resulted in the script not exiting upon failure
gaius 10y ago
Rutkowska is surely doing some of the most interesting research in this field.
- CiPHPerCoder 10y ago
  Rutkowska is among the crème de la crème of computer security researchers. That she's dedicating her time towards a project like Qubes is sorely under-appreciated by the public today, but is certainly something that inspires me to believe that, given enough time, defenders can win.
  - nickpsecurity 10y ago
    I'll give her credit for putting the time in more than most. Even I'm guilty on that one as my brain-damaged ass could be cloning UNIX apps in Rust or SPARK or something with MAC or Capsicum policies. My research is important but most of us doing such things don't do enough OSS coding.
- nickpsecurity 10y ago
  Barely. Most of her work is behind what was done in the 80's and 90's for security kernels then 2000's for separation kernels. I criticized her for not building on proven foundations and methods. She censors my stuff but did eventually say and do some of same things. My Xen gripes, GUI trusted path... these come to mind. For an example, here's an Orange Book A1-class VMM by legend Paul Karger. He's one of inventors of INFOSEC, genius designer/coder, and high-assurance veteran. Look at the design and assurance sections (p9 onward) of it to see what... in 90's... was necessary to secure a VMM via minimal privilege (POLA), correctness arguments, backdoor prevention, and covert channel suppression. Nothing today in OSS even has this baseline despite us discovering more problems and solutions. Re-reading it now, I noticed they were even doing continuous integration on it well before that became a fad. http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A-retrospective-on-the-VAX-VMM-security-kernel.pdf http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A... A modern example, one I cited on their mailing list, is INTEGRITY-178B. The features plus assurance activities are a nice illustration of high-assurance approach to microkernels for security or virtualization vs things like Xen. Quite a few things worth copying for security- or reliability-focused OSS projects. Approaches that got open-sourced from CompSci are in links below it. http://www.ghs.com/products/safety_critical/integrity-do-178b.html http://www.ghs.com/products/safety_critical/integrity-do-178... http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=9EE62F2088248DBB338582CF2822B3C8?doi=10.1.1.65.999&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=9EE... http://genode-labs.com/publications/nfeske-genode-fosdem-2012-02.pdf http://genode-labs.com/publications/nfeske-genode-fosdem-201... Note: GenodeOS is a competitor that uses components like above with architecture designed to lower risk in TCB like in high-assurance. It's nice work. Fundamental architecture needs peer review, though, to ensure it has claimed properties.
wongarsu 10y ago
Their build process sounds similar to Bitcoin Core's release process. Bitcoin downloads dependencies, checks them against their preconfigured hashes and then builds the different versions (Windows, Linux, Mac) in different VMs. The build is deterministic and thus produce the exact same files for everybody. Everyone signs the files they produced and uploads the signature. If all the signatures are valid for the same file you can be reasonably sure that the build process wasn't tampered with. Getting deterministic builds even for a project like Bitcoin Core with few dependencies was hard. On the scale of Qubes this would be a monumental task. But maybe Debian's initative for reproducible builds makes this easier in the future.
- j_s 10y ago
  To what degree are the Bitcoin build VMs reproducible vs. giant binary blobs?
  - wongarsu 10y ago
    The build VMs are predefined Ubuntu LTS releases. The build system that sets everything up is a bunch of barely documented scripts.
nickpsecurity 10y ago
Anyone interesting in securing repo's or build systems should start with Wheeler's landmark collection on the topic: http://www.dwheeler.com/essays/scm-security.html http://www.dwheeler.com/essays/scm-security.html Has basics in English, CompSci work, high-assurance considerations, and some example projects. A bright, security researcher that's very familiar with DVCS's should redo this in light of them with similar recommendations. More like a team of bright researchers but it needs to be done. I'm interested in any papers people already have on this that have similarly-thorough treatment of threat model and proposed mitigations. Once you know builds, you might want to address subversion, design, implementations, covert channels, and other things if you're trying to stop Five Eyes, Russia, or China. That requires "high-assurance" security methods... when it's even possible... Got a small list here to get people started on how deep the issue goes just at high-level and subversion aspects: https://news.ycombinator.com/item?id=10478742 https://news.ycombinator.com/item?id=10478742

64 points by kkl 10y ago

d33 10y ago

I heard that Qubes is riddled with undocumented scripts that have no automated tests for them. Does anybody know how true is that?

otoburb 10y ago
Qubes is open source[1] (including the Windows capabilities[2]) so your question should be fairly easy to prove/disprove yourself. I haven't delved into the source of any Qubes scripts or unit tests, but if one were to act on the rumours you heard perhaps they'd start with the Qubes Builder Scripts referenced in the article. If the Qubes team is able to implement a true append-only logging for their development+build process then perhaps any concerns around undocumented, unknown or untested scripts could be somewhat assuaged. [1] https://github.com/QubesOS/ https://github.com/QubesOS/ [2] https://www.qubes-os.org/news/2016/01/27/windows-tools-open-source/ https://www.qubes-os.org/news/2016/01/27/windows-tools-open-...
kakwa_ 10y ago
Just in case, I'm reposting my message from https://news.ycombinator.com/item?id=11652940 https://news.ycombinator.com/item?id=11652940 On the workstation part, it recommends QubesOS. Am I the only one who is skeptical about it? From what I saw superficially reading their source code, there are some frightening stuff going on: * tons of C code with nearly zero unit tests, same with the python code * lots glue in form of bash or python scripts * some not so beautiful stuff like: - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-firewall#L29 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (kill -9 on a daemon...) - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-firewall#L18 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (a daemon is a little bit more than an exe launched with '&' - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-core-appvm#L34 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (changing a config file in an init script, humm, weird...) - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-core-appvm#L32 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (starting a service inside the init of another service...) - https://github.com/QubesOS/qubes-core-agent-linux/blob/master/vm-init.d/qubes-qrexec-agent#L13 https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... ("logging" with stderr redirection in a file) And it's just the init scripts... I'm too lazy to take a look further inside the C or python stuff. IMHO, as a proof of concept, it's interesting, as a finished, reliable and secure OS, it's frightening...
- d33 10y ago
  Ah, yes, it's your comment that I was referring to.
- nickpsecurity 10y ago
  It's hall-marks of a low-assurance, software project hacked together to get it working with OSS components. There could be strong review in scripts or C code as the team are talented coders and breakers from what I've read. The minimum in high-assurance security... even medium-assurance... was that every feature was tested for proper behavior during successful execution and failures. Plus, a description of why it's there that traces to a functional or security requirements. That's so evaluators can spot unnecessary code that might be a backdoor or just dead code. Python for those scripts is questionable, too, as it's too complex to analyze with known principles of security engineering. It should be a 3GL that maps well to hardware with easy way to spot code-level flaws or automated them out (esp static analysis). You can abstract it, even macro it, so long as final code can be subjected to whatever eyeballs or tools are out there with ease. Non-security-critical stuff can be written in something like Python, though.
j_s 10y ago
I think this is an important point based on the following taken from the article: our script for the verify-sources target in the Xen component contained a silly bug that resulted in the script not exiting upon failure

gaius 10y ago

Rutkowska is surely doing some of the most interesting research in this field.

CiPHPerCoder 10y ago
Rutkowska is among the crème de la crème of computer security researchers. That she's dedicating her time towards a project like Qubes is sorely under-appreciated by the public today, but is certainly something that inspires me to believe that, given enough time, defenders can win.
- nickpsecurity 10y ago
  I'll give her credit for putting the time in more than most. Even I'm guilty on that one as my brain-damaged ass could be cloning UNIX apps in Rust or SPARK or something with MAC or Capsicum policies. My research is important but most of us doing such things don't do enough OSS coding.
nickpsecurity 10y ago
Barely. Most of her work is behind what was done in the 80's and 90's for security kernels then 2000's for separation kernels. I criticized her for not building on proven foundations and methods. She censors my stuff but did eventually say and do some of same things. My Xen gripes, GUI trusted path... these come to mind. For an example, here's an Orange Book A1-class VMM by legend Paul Karger. He's one of inventors of INFOSEC, genius designer/coder, and high-assurance veteran. Look at the design and assurance sections (p9 onward) of it to see what... in 90's... was necessary to secure a VMM via minimal privilege (POLA), correctness arguments, backdoor prevention, and covert channel suppression. Nothing today in OSS even has this baseline despite us discovering more problems and solutions. Re-reading it now, I noticed they were even doing continuous integration on it well before that became a fad. http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A-retrospective-on-the-VAX-VMM-security-kernel.pdf http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A... A modern example, one I cited on their mailing list, is INTEGRITY-178B. The features plus assurance activities are a nice illustration of high-assurance approach to microkernels for security or virtualization vs things like Xen. Quite a few things worth copying for security- or reliability-focused OSS projects. Approaches that got open-sourced from CompSci are in links below it. http://www.ghs.com/products/safety_critical/integrity-do-178b.html http://www.ghs.com/products/safety_critical/integrity-do-178... http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=9EE62F2088248DBB338582CF2822B3C8?doi=10.1.1.65.999&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=9EE... http://genode-labs.com/publications/nfeske-genode-fosdem-2012-02.pdf http://genode-labs.com/publications/nfeske-genode-fosdem-201... Note: GenodeOS is a competitor that uses components like above with architecture designed to lower risk in TCB like in high-assurance. It's nice work. Fundamental architecture needs peer review, though, to ensure it has claimed properties.

wongarsu 10y ago

Their build process sounds similar to Bitcoin Core's release process. Bitcoin downloads dependencies, checks them against their preconfigured hashes and then builds the different versions (Windows, Linux, Mac) in different VMs. The build is deterministic and thus produce the exact same files for everybody. Everyone signs the files they produced and uploads the signature. If all the signatures are valid for the same file you can be reasonably sure that the build process wasn't tampered with. Getting deterministic builds even for a project like Bitcoin Core with few dependencies was hard. On the scale of Qubes this would be a monumental task. But maybe Debian's initative for reproducible builds makes this easier in the future.

j_s 10y ago
To what degree are the Bitcoin build VMs reproducible vs. giant binary blobs?
- wongarsu 10y ago
  The build VMs are predefined Ubuntu LTS releases. The build system that sets everything up is a bunch of barely documented scripts.

nickpsecurity 10y ago

Anyone interesting in securing repo's or build systems should start with Wheeler's landmark collection on the topic: http://www.dwheeler.com/essays/scm-security.html http://www.dwheeler.com/essays/scm-security.html Has basics in English, CompSci work, high-assurance considerations, and some example projects. A bright, security researcher that's very familiar with DVCS's should redo this in light of them with similar recommendations. More like a team of bright researchers but it needs to be done. I'm interested in any papers people already have on this that have similarly-thorough treatment of threat model and proposed mitigations. Once you know builds, you might want to address subversion, design, implementations, covert channels, and other things if you're trying to stop Five Eyes, Russia, or China. That requires "high-assurance" security methods... when it's even possible... Got a small list here to get people started on how deep the issue goes just at high-level and subversion aspects: https://news.ycombinator.com/item?id=10478742 https://news.ycombinator.com/item?id=10478742