10 ms·

Given all of the security issues I am curious who thinks this is production ready ?

by geggam 7y ago

persistent 7y ago
Which of these issues strikes you as being a meaningful risk to a realistic deployment of Kubernetes?
- geggam 7y ago
  The better question is when you deploy k8s in a production how do you ensure none of the risks are being exploited. Given todays landscape of hardware and software exploits adding a complex orchestration layer with identified issues seems like less than prudent behavior. I currently work on kubernetes in production and am migrating large clients into these systems. I see the distinct lack of knowledge around securing systems and more so when adding kubernetes.
  - persistent 7y ago
    I'm not running antagonistic workloads in k8s though, I'm just running my own junk, each component of which also has its own laundry list of security nightmares.
  - raesene9 7y ago
    As with anything in security, this will depend on your threat model and the benefits of using k8s. There is some good information about securing k8s around, although we could always do with more. There's a free oReilly eBook on k8s security from aqua (https://info.aquasec.com/kubernetes-security https://info.aquasec.com/kubernetes-security) Also the CIS benchmark for k8s is reasonably up to date, although could use expanding, which should be happening for the next version. On top of that there have been quite a few conf. talks now about k8s security https://www.youtube.com/playlist?list=PLKDRii1YwXnLmd8ngltnf9Kzvbja3DJWx https://www.youtube.com/playlist?list=PLKDRii1YwXnLmd8ngltnf... for some examples
  - deleted 7y ago
    [deleted]
cj 7y ago
Not sure why this is being downvoted. Kubernetes is 5 years old. This is very, very young for mission-critical infrastructure management software. Having a certain level of doubt in young open source projects is responsible, in my opinion. I'm interested to hear other people's perspective on production-readiness of k8s for mission-critical applications.
- raesene9 7y ago
  If security got to be the number one concern for whether things were deployed or not, then sure we could likely take a more conservative view. However realistically k8s is in heavy deployment in a wide variety of industries including public sector, financial services, retail, technology ... and it's clear that this kind of concern is not the primary consideration. There were banks in the UK (Monzo) deploying k8s almost 3 years ago (https://monzo.com/blog/2016/09/19/building-a-modern-bank-backend https://monzo.com/blog/2016/09/19/building-a-modern-bank-bac...)
  - geggam 7y ago
    And its clear with the number of data dumps online of personal information that this needs to change. Good example is the recent offer for 147 million people to get 125 dollars out of a pool of 31 million ( Equifax ) As an industry IT is simply shamefully shoddy
  - cj 7y ago
    In most other industries, saying something is in "heavy development" is usually the same as "unstable". (Unstable usually is interpretted as Bad in software engineering -- but the dictionary definition of "unstable" only means "prone to change", which I think is an accurate characterization of k8s considering its degree of maturity) Whether or not something is a smart choice to use in mission-critical production applications doesn't depend on the number of big banks or big tech companies that use the technology. At the end of the day, Kubernetes is a tool that will change very rapidly over the next 5 years. I could see k8s being a decent choice to use in a tech project that you expect to actively maintain and improve for the next 5+ years, AND if you (and your developers) are willing to invest time (potentially a lot of time) every year keeping up to speed with how k8s evolves through every version release. That's the primary risk in using something like k8s.
geofft 7y ago
Are these more / worse security issues than anything else we call production-ready? The Linux kernel is full of security bugs, for instance.
- geggam 7y ago
  So adding more exploits is better ? OpenBSD security model is sound. Simplify and secure.
  - tptacek 7y ago
    "Only two remote holes in the default install, in a heck of a long time!" Due respect to smart acquaintances who work on OpenBSD, but to most people who secure application deployment environments, this is not the reassuring statement OpenBSD seems to think it is.
    - wolf550e 7y ago
      Who has a better track record?
      - tptacek 7y ago
        The problem is that it isn't a track record. What's funny about it is, if you're going to make up a benchmark (and theirs is contrived; it was "no remote vulnerabilities", as I recall, when I was involved with the project, then "no remote vulnerabilities in the default install", then "only one remote vulnerability in the default install"), make up one where your number is zero, not "just 2 in a heck of a long time". But more substantively: the reason you run an operating system is to do stuff on it. It isn't 1996 any more and nobody gets public shell accounts on Linux systems or OpenBSD systems; similarly, remotely-exploitable vulnerabilities in other operating systems are also exceedingly rare, and so OpenBSD's benchmark excludes the LPEs that actually make up the meaningful attack surface of a modern OS. What's a more important question is what features the operating system provides to harden the non-default programs that inevitably have to run on it. OpenBSD has historically lagged here, though they're upping their game recently. Despite briefly being involved with the project during "The OpenBSD Security Audit" in the late 1990s, I have a longstanding bias against OpenBSD that I should be up front with: we shipped an appliance on OpenBSD at Arbor Networks, and I spent several days debugging a VMM problem that would zombify pages of memory and gradually suffocate our systems. When I presented evidence to Theo, he said (not a literal quote) "don't bother me about this, Chuck Cranor" --- I think it's Chuck Cranor but could be wrong --- "wrote this VMM as his graduate project and I've got nothing to do with it". For whatever that's worth, I've felt OpenBSD is an unserious option for deploying real systems other than near-stateless network middleboxes ever since.
nova22033 7y ago
we keep finding 0-days and security issues in yourfavoritesoftware. Who thought yourfavoritesoftware was production ready?
- geggam 7y ago
  So you agree keeping things simple to reduce attack vectors is the best way to go ?
segmondy 7y ago
It's production ready, folks have been running it on production and will be running it on production. Sure it has issues from inside the cluster. But if you secure it and it's not accessible from outside, it's good to go. Probably more secure than trying to run 500 boxes at once.
- geggam 7y ago
  So you think kubernetes eliminates the need to run hardware or you think it reduces the number of machines ? I think it adds overhead but does allow you do maximize server density and usage allowing you to use all 500 machines more effectively. And it adds a plethora of attack vectors
  - raesene9 7y ago
    Out of curiousity, Which k8s attack vectors, do you think are particularly concerning?
  - segmondy 7y ago
    Yes, it significantly reduces the number of machines. That's the main benefit. You can binpack your pods by sizing it up well and maxing out resources on each machine.
    - geggam 7y ago
      I completely disagree based on multiple deployments at scale. You use more hardware with k8s not less.

Given all of the security issues I am curious who thinks this is production ready ?

by geggam 7y ago

Given all of the security issues I am curious who thinks this is production ready ?

persistent 7y ago

Which of these issues strikes you as being a meaningful risk to a realistic deployment of Kubernetes?

geggam 7y ago
The better question is when you deploy k8s in a production how do you ensure none of the risks are being exploited. Given todays landscape of hardware and software exploits adding a complex orchestration layer with identified issues seems like less than prudent behavior. I currently work on kubernetes in production and am migrating large clients into these systems. I see the distinct lack of knowledge around securing systems and more so when adding kubernetes.
- persistent 7y ago
  I'm not running antagonistic workloads in k8s though, I'm just running my own junk, each component of which also has its own laundry list of security nightmares.
- raesene9 7y ago
  As with anything in security, this will depend on your threat model and the benefits of using k8s. There is some good information about securing k8s around, although we could always do with more. There's a free oReilly eBook on k8s security from aqua (https://info.aquasec.com/kubernetes-security https://info.aquasec.com/kubernetes-security) Also the CIS benchmark for k8s is reasonably up to date, although could use expanding, which should be happening for the next version. On top of that there have been quite a few conf. talks now about k8s security https://www.youtube.com/playlist?list=PLKDRii1YwXnLmd8ngltnf9Kzvbja3DJWx https://www.youtube.com/playlist?list=PLKDRii1YwXnLmd8ngltnf... for some examples
- deleted 7y ago
  [deleted]

cj 7y ago

Not sure why this is being downvoted. Kubernetes is 5 years old. This is very, very young for mission-critical infrastructure management software. Having a certain level of doubt in young open source projects is responsible, in my opinion. I'm interested to hear other people's perspective on production-readiness of k8s for mission-critical applications.

raesene9 7y ago
If security got to be the number one concern for whether things were deployed or not, then sure we could likely take a more conservative view. However realistically k8s is in heavy deployment in a wide variety of industries including public sector, financial services, retail, technology ... and it's clear that this kind of concern is not the primary consideration. There were banks in the UK (Monzo) deploying k8s almost 3 years ago (https://monzo.com/blog/2016/09/19/building-a-modern-bank-backend https://monzo.com/blog/2016/09/19/building-a-modern-bank-bac...)
- geggam 7y ago
  And its clear with the number of data dumps online of personal information that this needs to change. Good example is the recent offer for 147 million people to get 125 dollars out of a pool of 31 million ( Equifax ) As an industry IT is simply shamefully shoddy
- cj 7y ago
  In most other industries, saying something is in "heavy development" is usually the same as "unstable". (Unstable usually is interpretted as Bad in software engineering -- but the dictionary definition of "unstable" only means "prone to change", which I think is an accurate characterization of k8s considering its degree of maturity) Whether or not something is a smart choice to use in mission-critical production applications doesn't depend on the number of big banks or big tech companies that use the technology. At the end of the day, Kubernetes is a tool that will change very rapidly over the next 5 years. I could see k8s being a decent choice to use in a tech project that you expect to actively maintain and improve for the next 5+ years, AND if you (and your developers) are willing to invest time (potentially a lot of time) every year keeping up to speed with how k8s evolves through every version release. That's the primary risk in using something like k8s.

geofft 7y ago

Are these more / worse security issues than anything else we call production-ready? The Linux kernel is full of security bugs, for instance.

geggam 7y ago
So adding more exploits is better ? OpenBSD security model is sound. Simplify and secure.
- tptacek 7y ago
  "Only two remote holes in the default install, in a heck of a long time!" Due respect to smart acquaintances who work on OpenBSD, but to most people who secure application deployment environments, this is not the reassuring statement OpenBSD seems to think it is.
  - wolf550e 7y ago
    Who has a better track record?
    - tptacek 7y ago
      The problem is that it isn't a track record. What's funny about it is, if you're going to make up a benchmark (and theirs is contrived; it was "no remote vulnerabilities", as I recall, when I was involved with the project, then "no remote vulnerabilities in the default install", then "only one remote vulnerability in the default install"), make up one where your number is zero, not "just 2 in a heck of a long time". But more substantively: the reason you run an operating system is to do stuff on it. It isn't 1996 any more and nobody gets public shell accounts on Linux systems or OpenBSD systems; similarly, remotely-exploitable vulnerabilities in other operating systems are also exceedingly rare, and so OpenBSD's benchmark excludes the LPEs that actually make up the meaningful attack surface of a modern OS. What's a more important question is what features the operating system provides to harden the non-default programs that inevitably have to run on it. OpenBSD has historically lagged here, though they're upping their game recently. Despite briefly being involved with the project during "The OpenBSD Security Audit" in the late 1990s, I have a longstanding bias against OpenBSD that I should be up front with: we shipped an appliance on OpenBSD at Arbor Networks, and I spent several days debugging a VMM problem that would zombify pages of memory and gradually suffocate our systems. When I presented evidence to Theo, he said (not a literal quote) "don't bother me about this, Chuck Cranor" --- I think it's Chuck Cranor but could be wrong --- "wrote this VMM as his graduate project and I've got nothing to do with it". For whatever that's worth, I've felt OpenBSD is an unserious option for deploying real systems other than near-stateless network middleboxes ever since.

nova22033 7y ago

we keep finding 0-days and security issues in yourfavoritesoftware. Who thought yourfavoritesoftware was production ready?

geggam 7y ago
So you agree keeping things simple to reduce attack vectors is the best way to go ?

segmondy 7y ago

It's production ready, folks have been running it on production and will be running it on production. Sure it has issues from inside the cluster. But if you secure it and it's not accessible from outside, it's good to go. Probably more secure than trying to run 500 boxes at once.

geggam 7y ago
So you think kubernetes eliminates the need to run hardware or you think it reduces the number of machines ? I think it adds overhead but does allow you do maximize server density and usage allowing you to use all 500 machines more effectively. And it adds a plethora of attack vectors
- raesene9 7y ago
  Out of curiousity, Which k8s attack vectors, do you think are particularly concerning?
- segmondy 7y ago
  Yes, it significantly reduces the number of machines. That's the main benefit. You can binpack your pods by sizing it up well and maxing out resources on each machine.
  - geggam 7y ago
    I completely disagree based on multiple deployments at scale. You use more hardware with k8s not less.