4 ms·

OpenShift is a fork of Kubernetes. Case in point. OpenShift has functionality called a Route. That isn't in Kubernetes. Kubernetes of course went and added

by breser 9y ago

OpenShift is a fork of Kubernetes. Case in point. OpenShift has functionality called a Route. That isn't in Kubernetes. Kubernetes of course went and added something similar called ingress. This means that anytime Kubernetes does a release that RedHat has to manage merging all those changes in with their local changes that aren't part of their project. This is exactly the same sort of thing that RedHat does with their Linux kernels. It's exactly why you RHEL has been such a terrible platform to run Docker on. Because RedHat is only pulling in some of the upstream changes into their kernels rather than upgrading everything. This sort of merging is slow, error prone and costly. If you use anything that they've added that isn't in the upstream it creates vendor lock-in. Sure OpenShift's Route code is open source, but you can't take that to any other vendor without building everything yourself. Want that new feature in the latest Kubernetes release, you better be prepared to wait. As time goes on it will become harder and harder for them to maintain this fork. If OpenShift doesn't become the dominate Kubernetes distribution then RedHat may lose interest in it and then you'll lose your maintenance of that fork. For that matter, if RedHat loses the maintainers as employees they may lose their ability to maintain this fork. It's my opinion that anyone buying OpenShift is playing with fire. The fact that RedHat has failed to get their modifications integrated upstream and have to maintain themselves is a massive failure on their part. This sort of thing was understandable in the past, but we really should expect more of them now.

wstrange 9y ago
My understanding is that OpenShift is more of a superset of k8s vs. a fork. I don't think the situation is going to be as bad as you are implying.
- breser 9y ago
  A fork can imply many things, but all supersets are forks. I realize that RedHat doesn't like calling OpenShift a fork, but it fundamentally is. The only people that will maintain their additional functionality is RedHat themselves. If you manage to avoid all of their added functionality then you can avoid the vendor lock in. But you can't avoid the delays in getting upstream features/bug fixes. If Kubernetes was super stable and you weren't likely to want any of that stuff I would agree it probably wouldn't be a big deal. But it's not. Significant improvements are coming in every single release. RedHat is managing a decent pace in keeping up with Kubernetes for now. But all that it takes to upset that apple cart is for something to get added in a way that breaks their additional functionality. If RedHat doesn't have the clout to get their improvements in upstream, why should I presume they have the clout to avoid their additions won't be broken by other changes?
  - Avshalom 9y ago
    The thing is some people want super stable, improvements are great but the ability to plan is also great. That breaking change that prevents Redhat from integrating Kubernetes into Openshift is just as likely to be a breaking change for Company X using Kubernetes That's why they pay Redhat, so they can plan.
    - dasil003 9y ago
      You're sort of twisting GP's point by saying that because K8s is not super stable, Red Hat will provide stability via OpenShift. I mean yes, Red Hat will certainly provide a more stable version, but it's a double-edged sword, because K8s might leave OpenShift in the dust with some incompatible change, and then a few years down the line you could end up with serious buyer's remorse when K8s is an order of magnitude better and OpenShift is left in some well-maintained purgatory.
      - breser 9y ago
        Yup, when I talk about stability I'm not talking about stability of function, which I presume is what people buying OpenShift want. I'm talking about stability as in few changes. Anyone doing things in the Container space shouldn't be expecting lack of changes, even if you're using OpenShift.
      - Avshalom 9y ago
        You could equally end up a few years down the line with Kubernetes being an order of magnitude better but also having required two orders of magnitude of work (over those years) porting your companies infrastructure to it. or a few years down the line the zeitgeist has moved to Locutus and no one but Red Hat is driving anymore.
      - smarterclayton 9y ago
        I can assure you that openshift will always be Kube++. It's just a Kube distro. The fact that today you need to compile in those extensions is a detail that we and others spend most of the time addressing. Odds are, most of the things you use in Kubernetes were because someone working on OpenShift wrote, tested, performance tested, and stress tested in production. When LTS OpenShift is a thing, there will still be an OpenShift trucking along right behind the latest Kube. We always try to strike the balance between being on the bleeding edge and making sure end user clusters continue to work. In fact, a lot of the bugs in patch releases are found by the teams working on openshift and opened upstream right away. But an OpenShift user never sees that, because we only ship once it's stable.
- oso2k 9y ago
  [Full disclosure: I work for Red Hat as an OpenShift Consultant] The truth is a bit more nuanced as Kubernetes and OpenShift are actually made up of dozens of projects and integrations. Our company contributes to (as upstreams) in support of our OpenShift Enterprise offering: Kubernetes, Linux kernel, HAProxy, Jenkins, Hawkular, Heapster, Cassandra, Elasticsearch, FluentD, Kibana, Jenkins, JBoss, Tomcat, Apache, Ansible, Go-lang, and probably many more. We do almost all of our work completely in the open (our docs, container images, templates, examples, blogs) via github and trello. In fact, you can run just about the same OpenShift (officially called OpenShift Container Platform (OCP or OSCP) or OpenShift Enterprise (OSE)) we sell by using our upstream project for it, OpenShift Origin [0][1]. If that looks complicated, you can try minishift [2][3] to start which also has an upstream Kubernetes project in minikube [4]. In terms of superset vs. fork: it's not quite a superset because almost everything we commit to OpenShift gets committed to Kubernetes and/or vicea versa. You can almost always say if it works in OpenShift, it works in Kubernetes; if it works in Kubernetes, it works in OpenShift. It's not really a fork either (as we say often "Best idea wins!") and so our people (including management!) try to make sure we are adding value to Kubernetes so that our customers & the community can then extract that value. OpenShift/Kubernetes metrics is one place that affects me & my customers that we're following the community's lead on that and implementing new developments in OpenShift as Tech Previews when appropriate. Our code is not diverging from Kubernetes as much as you might think in supporting some of the "enterprise" features we've added. So, I would say OpenShift is a Distro of Kubernetes in the same ways RHEL, SuSE, et. al., are GNU/Linux Distros. You might say Kubernetes provides the "kernel" for a modern Data Center (compute resource scheduling and management, internal/external data structures and interfaces to use such compute resources). OpenShift is intended to help provide everything else you expect your data center to do for you or to support your Application Development in Java, .Net, node.js, Ruby, Python, PHP, Perl, etc (UI & CLI management interfaces, simplified build and deployment processes (S2I), Jenkins integration, external logging integration, external monitoring integration, sample 12 Factor Applications, etc.). We partner with companies when they want help bringing on their storage systems, frameworks, databases, applications, etc., just like you'd expect when companies provide drivers for their databases, hardware, or storage systems for OS kernels. [0] https://www.openshift.org/ https://www.openshift.org/ [1] https://github.com/openshift/origin https://github.com/openshift/origin [2] https://docs.openshift.org/latest/minishift/getting-started/installing.html https://docs.openshift.org/latest/minishift/getting-started/... [3] https://github.com/minishift/minishift https://github.com/minishift/minishift [4] https://kubernetes.io/docs/tasks/tools/install-minikube/ https://kubernetes.io/docs/tasks/tools/install-minikube/
yeukhon 9y ago
It is important to note OpenShift predated Kubernetes. RH reimplemented the underlying platform by adopting Kubernetes only a few years ago.
- breser 9y ago
  That is a fair point. I'm only talking about the "Kubernetes" implementation within OpenShift. It's modified from the upstream one and includes things that aren't part of upstream Kubernetes.
  - smarterclayton 9y ago
    Routes were created about a year before ingress. Part of what we do is give users a path of adoption. For instance, routes will continue to work forever. The openshift routers have almost every bell and whistle possible for routes - and we've recently added ingress support. We also took all of the security features of routes and applied them to ingress - for example, if two different namespaces ask for the same hostname with a route, the oldest one always gets served. We also adapted the security rbac around routes to ingress, so you can set a role that controls whether end users can use custom host names or custom certs. We also are in the process of adding all of our extended cert validation to the router for both routes and ingress, so if a user puts in a bad cert other users aren't impacted. Basically, you can use vanilla Kubernetes if you want. But openshift is a sundae with sprinkles. Come for the ice cream, stay for the toppings? Edit: also, every add on in openshift is either something that will eventually be in Kube, or an extension to Kube. Red Hatters are the ones who added extensibility to Kube (with help from others) like api extensions, CRD, initializers, web hook extensions, binary CLI plugins. We did that so that OpenShift can extend Kube to solve real user problems, and also so that everyone else in the ecosystem can do the same.
smarterclayton 9y ago
> The fact that RedHat has failed to get their modifications integrated upstream and have to maintain themselves is a massive failure on their part. Not really correct. We treated ingress as routes v2. Some of the design choices for ingress (which is still beta, and may change again before reaching stable) were improvements, but others created more problems. RBAC, most of the authentication code, a huge amount of performance work, podsecuritypolicy, egress network policy, and many others all originated in OpenShift, and then we moved or helped move them into Kube. When we did that, we worked in the community to improve those features. And then we do all the work in OpenShift to make them transparently (mostly) available to the early adopters. For instance, OpenShift RBAC APIs in 3.7 now sit on top of Kube RBAC, and you can use either API. We'll continue supporting that for a long time so that users can switch at their leisure. It's just what we do. Edit: templates are the only thing that didn't get upstream, and it was because Helm was good enough at that point that we didn't need it in Kube. We continue to support templates, and they are exposed under the new service catalog work as a broker so users can be completely oblivious to their consumption like we're hoping to do for Helm. Everyone wins. Edit2: deployment configs also are an example of predating Kube deployments - the fundamental design choice is actually different (DC can fail and inform you something is wrong, deployments just try forever). We continue to add capability to deployments to make them better than DCs, and then add the same improvements to DCs. If we picked a new name it would be DeploymentJob - it can run hooks, have custom logic, and fail. It's not upstream, but will be an extension API soon.
- shaklee3 9y ago
  This is easily confirmed. Just look at the companies upstreaming into k8s, and you'll see redhat is dominating. They have people in almost all SIGs, and are very active in the community. Thanks for all the contributions.
- tapirl 9y ago
  The extension API feature is cool, it will make k8s ecosystem grow more rapidly. BTW, will all the extra features in OpenShift be ported as extension APIs?
nvarsj 9y ago
Kubernetes owes its success largely to Openshift and Redhat's efforts. Without Openshift, Kubernetes would just be an interesting POC. Google doesn't dog food Kubernetes. Openshift has since the beginning, and contributed significantly to K8s as a result of actual production usage. Just take a look at the top contributors and you can see the kind of contributions of the Redhat guys. While I sort of agree with you on RHEL, I don't think this is the case with Openshift at all. I wouldn't hesitate to recommend it to people looking for a full solution.
- dominotw 9y ago
  > Kubernetes owes its success largely to Openshift and Redhat's efforts. Without Openshift, Kubernetes would just be an interesting POC. How did you arrive at this conclusion. Curious to know more about redhats role in this.
  - jacques_chester 9y ago
    Red Hat have done a lot of the productionising and packaging and got it running at a lot of companies. I don't fully agree that Red Hat has more ownership of the success of Kubernetes, though. They may have been necessary, but by no means sufficient. The aura of Google has probably had far more importance in the momentum to date.
    - nvarsj 9y ago
      CNCF owns the Kubernetes project. Google provides a lot of resources, and so does Redhat. Both are heavily involved in k8s, and both contributed significantly to its success.
  - nvarsj 9y ago
    My involvement w/ Kubernetes over the last few years. I've always considered k8s co-led by Google and Redhat. It's not hard to find this out for yourself - just take a look at the Github and the mailing lists, it's all open. K8s changed dramatically with Redhat's involvement.
dankohn1 9y ago
You might be interested in CNCF's Kubernetes Software Conformance Working Group, which has been working closely with the Kubernetes's architecture and testing SIGs and providers of most of the Kubernetes distributions (including Red Hat) to ensure interoperability. https://www.cncf.io/certification/software-conformance/ https://www.cncf.io/certification/software-conformance/

by breser 9y ago

wstrange 9y ago

My understanding is that OpenShift is more of a superset of k8s vs. a fork. I don't think the situation is going to be as bad as you are implying.

breser 9y ago
A fork can imply many things, but all supersets are forks. I realize that RedHat doesn't like calling OpenShift a fork, but it fundamentally is. The only people that will maintain their additional functionality is RedHat themselves. If you manage to avoid all of their added functionality then you can avoid the vendor lock in. But you can't avoid the delays in getting upstream features/bug fixes. If Kubernetes was super stable and you weren't likely to want any of that stuff I would agree it probably wouldn't be a big deal. But it's not. Significant improvements are coming in every single release. RedHat is managing a decent pace in keeping up with Kubernetes for now. But all that it takes to upset that apple cart is for something to get added in a way that breaks their additional functionality. If RedHat doesn't have the clout to get their improvements in upstream, why should I presume they have the clout to avoid their additions won't be broken by other changes?
- Avshalom 9y ago
  The thing is some people want super stable, improvements are great but the ability to plan is also great. That breaking change that prevents Redhat from integrating Kubernetes into Openshift is just as likely to be a breaking change for Company X using Kubernetes That's why they pay Redhat, so they can plan.
  - dasil003 9y ago
    You're sort of twisting GP's point by saying that because K8s is not super stable, Red Hat will provide stability via OpenShift. I mean yes, Red Hat will certainly provide a more stable version, but it's a double-edged sword, because K8s might leave OpenShift in the dust with some incompatible change, and then a few years down the line you could end up with serious buyer's remorse when K8s is an order of magnitude better and OpenShift is left in some well-maintained purgatory.
    - breser 9y ago
      Yup, when I talk about stability I'm not talking about stability of function, which I presume is what people buying OpenShift want. I'm talking about stability as in few changes. Anyone doing things in the Container space shouldn't be expecting lack of changes, even if you're using OpenShift.
    - Avshalom 9y ago
      You could equally end up a few years down the line with Kubernetes being an order of magnitude better but also having required two orders of magnitude of work (over those years) porting your companies infrastructure to it. or a few years down the line the zeitgeist has moved to Locutus and no one but Red Hat is driving anymore.
    - smarterclayton 9y ago
      I can assure you that openshift will always be Kube++. It's just a Kube distro. The fact that today you need to compile in those extensions is a detail that we and others spend most of the time addressing. Odds are, most of the things you use in Kubernetes were because someone working on OpenShift wrote, tested, performance tested, and stress tested in production. When LTS OpenShift is a thing, there will still be an OpenShift trucking along right behind the latest Kube. We always try to strike the balance between being on the bleeding edge and making sure end user clusters continue to work. In fact, a lot of the bugs in patch releases are found by the teams working on openshift and opened upstream right away. But an OpenShift user never sees that, because we only ship once it's stable.
oso2k 9y ago
[Full disclosure: I work for Red Hat as an OpenShift Consultant] The truth is a bit more nuanced as Kubernetes and OpenShift are actually made up of dozens of projects and integrations. Our company contributes to (as upstreams) in support of our OpenShift Enterprise offering: Kubernetes, Linux kernel, HAProxy, Jenkins, Hawkular, Heapster, Cassandra, Elasticsearch, FluentD, Kibana, Jenkins, JBoss, Tomcat, Apache, Ansible, Go-lang, and probably many more. We do almost all of our work completely in the open (our docs, container images, templates, examples, blogs) via github and trello. In fact, you can run just about the same OpenShift (officially called OpenShift Container Platform (OCP or OSCP) or OpenShift Enterprise (OSE)) we sell by using our upstream project for it, OpenShift Origin [0][1]. If that looks complicated, you can try minishift [2][3] to start which also has an upstream Kubernetes project in minikube [4]. In terms of superset vs. fork: it's not quite a superset because almost everything we commit to OpenShift gets committed to Kubernetes and/or vicea versa. You can almost always say if it works in OpenShift, it works in Kubernetes; if it works in Kubernetes, it works in OpenShift. It's not really a fork either (as we say often "Best idea wins!") and so our people (including management!) try to make sure we are adding value to Kubernetes so that our customers & the community can then extract that value. OpenShift/Kubernetes metrics is one place that affects me & my customers that we're following the community's lead on that and implementing new developments in OpenShift as Tech Previews when appropriate. Our code is not diverging from Kubernetes as much as you might think in supporting some of the "enterprise" features we've added. So, I would say OpenShift is a Distro of Kubernetes in the same ways RHEL, SuSE, et. al., are GNU/Linux Distros. You might say Kubernetes provides the "kernel" for a modern Data Center (compute resource scheduling and management, internal/external data structures and interfaces to use such compute resources). OpenShift is intended to help provide everything else you expect your data center to do for you or to support your Application Development in Java, .Net, node.js, Ruby, Python, PHP, Perl, etc (UI & CLI management interfaces, simplified build and deployment processes (S2I), Jenkins integration, external logging integration, external monitoring integration, sample 12 Factor Applications, etc.). We partner with companies when they want help bringing on their storage systems, frameworks, databases, applications, etc., just like you'd expect when companies provide drivers for their databases, hardware, or storage systems for OS kernels. [0] https://www.openshift.org/ https://www.openshift.org/ [1] https://github.com/openshift/origin https://github.com/openshift/origin [2] https://docs.openshift.org/latest/minishift/getting-started/installing.html https://docs.openshift.org/latest/minishift/getting-started/... [3] https://github.com/minishift/minishift https://github.com/minishift/minishift [4] https://kubernetes.io/docs/tasks/tools/install-minikube/ https://kubernetes.io/docs/tasks/tools/install-minikube/

yeukhon 9y ago

It is important to note OpenShift predated Kubernetes. RH reimplemented the underlying platform by adopting Kubernetes only a few years ago.

breser 9y ago
That is a fair point. I'm only talking about the "Kubernetes" implementation within OpenShift. It's modified from the upstream one and includes things that aren't part of upstream Kubernetes.
- smarterclayton 9y ago
  Routes were created about a year before ingress. Part of what we do is give users a path of adoption. For instance, routes will continue to work forever. The openshift routers have almost every bell and whistle possible for routes - and we've recently added ingress support. We also took all of the security features of routes and applied them to ingress - for example, if two different namespaces ask for the same hostname with a route, the oldest one always gets served. We also adapted the security rbac around routes to ingress, so you can set a role that controls whether end users can use custom host names or custom certs. We also are in the process of adding all of our extended cert validation to the router for both routes and ingress, so if a user puts in a bad cert other users aren't impacted. Basically, you can use vanilla Kubernetes if you want. But openshift is a sundae with sprinkles. Come for the ice cream, stay for the toppings? Edit: also, every add on in openshift is either something that will eventually be in Kube, or an extension to Kube. Red Hatters are the ones who added extensibility to Kube (with help from others) like api extensions, CRD, initializers, web hook extensions, binary CLI plugins. We did that so that OpenShift can extend Kube to solve real user problems, and also so that everyone else in the ecosystem can do the same.

smarterclayton 9y ago

> The fact that RedHat has failed to get their modifications integrated upstream and have to maintain themselves is a massive failure on their part. Not really correct. We treated ingress as routes v2. Some of the design choices for ingress (which is still beta, and may change again before reaching stable) were improvements, but others created more problems. RBAC, most of the authentication code, a huge amount of performance work, podsecuritypolicy, egress network policy, and many others all originated in OpenShift, and then we moved or helped move them into Kube. When we did that, we worked in the community to improve those features. And then we do all the work in OpenShift to make them transparently (mostly) available to the early adopters. For instance, OpenShift RBAC APIs in 3.7 now sit on top of Kube RBAC, and you can use either API. We'll continue supporting that for a long time so that users can switch at their leisure. It's just what we do. Edit: templates are the only thing that didn't get upstream, and it was because Helm was good enough at that point that we didn't need it in Kube. We continue to support templates, and they are exposed under the new service catalog work as a broker so users can be completely oblivious to their consumption like we're hoping to do for Helm. Everyone wins. Edit2: deployment configs also are an example of predating Kube deployments - the fundamental design choice is actually different (DC can fail and inform you something is wrong, deployments just try forever). We continue to add capability to deployments to make them better than DCs, and then add the same improvements to DCs. If we picked a new name it would be DeploymentJob - it can run hooks, have custom logic, and fail. It's not upstream, but will be an extension API soon.

shaklee3 9y ago
This is easily confirmed. Just look at the companies upstreaming into k8s, and you'll see redhat is dominating. They have people in almost all SIGs, and are very active in the community. Thanks for all the contributions.
tapirl 9y ago
The extension API feature is cool, it will make k8s ecosystem grow more rapidly. BTW, will all the extra features in OpenShift be ported as extension APIs?

nvarsj 9y ago

Kubernetes owes its success largely to Openshift and Redhat's efforts. Without Openshift, Kubernetes would just be an interesting POC. Google doesn't dog food Kubernetes. Openshift has since the beginning, and contributed significantly to K8s as a result of actual production usage. Just take a look at the top contributors and you can see the kind of contributions of the Redhat guys. While I sort of agree with you on RHEL, I don't think this is the case with Openshift at all. I wouldn't hesitate to recommend it to people looking for a full solution.

dominotw 9y ago
> Kubernetes owes its success largely to Openshift and Redhat's efforts. Without Openshift, Kubernetes would just be an interesting POC. How did you arrive at this conclusion. Curious to know more about redhats role in this.
- jacques_chester 9y ago
  Red Hat have done a lot of the productionising and packaging and got it running at a lot of companies. I don't fully agree that Red Hat has more ownership of the success of Kubernetes, though. They may have been necessary, but by no means sufficient. The aura of Google has probably had far more importance in the momentum to date.
  - nvarsj 9y ago
    CNCF owns the Kubernetes project. Google provides a lot of resources, and so does Redhat. Both are heavily involved in k8s, and both contributed significantly to its success.
- nvarsj 9y ago
  My involvement w/ Kubernetes over the last few years. I've always considered k8s co-led by Google and Redhat. It's not hard to find this out for yourself - just take a look at the Github and the mailing lists, it's all open. K8s changed dramatically with Redhat's involvement.

dankohn1 9y ago

You might be interested in CNCF's Kubernetes Software Conformance Working Group, which has been working closely with the Kubernetes's architecture and testing SIGs and providers of most of the Kubernetes distributions (including Red Hat) to ensure interoperability. https://www.cncf.io/certification/software-conformance/ https://www.cncf.io/certification/software-conformance/