5 ms·

So you move from vendor lock in with the cloud provider, to vendor lock in with an expensive, proprietary* IBM k8s distribution with its strange nonstandard opi

So you move from vendor lock in with the cloud provider, to vendor lock in with an expensive, proprietary* IBM k8s distribution with its strange nonstandard opinions about workflows that you have to manage yourself? Don't get me wrong, I appreciate RHAT's code contributions very much, they have done a lot for k8s! But running OKD on one's own is a bad idea, while paying for IBM support makes you as much a hostage as anything Google will do to you. Better to just stick with a distribution with better community support and wait for RHAT's useful innovations to be merged upstream (while avoiding the pitfalls of their failed experiments...) * Yes it's open source, but the community version (okd) isn't really supported, nor is it widely used, so if you're serious about running this you're doing so for the Enterprise support and you're going to be writing those checks

rcarmo 7y ago
I work for Microsoft and I quite like OpenShift (although we have AKS as the default, "vanilla" managed k8s service, you can also run OpenShift on Azure). Sure it's opinionated, but at this point, which flavor of k8s isn't? Even k3s (which I play around with on https://github.com/rcarmo/azure-k3s-cluster https://github.com/rcarmo/azure-k3s-cluster) has its quirks. Everyone who's targeting the enterprise market has a "value-added" brew of k8s at this point, so kudos to RH for the engineering effort :)
freedomben 7y ago
Thanks for the edits (and acknowledging our contributions). I wasn't sure if you were just trolling or not before, so I didn't want to engage. Your concern is valid, and I agree with you that OKD is not supported enough. I have my own theories as to why, but I will keep my criticism "in the family" (but do know there are people that want to see OKD be a first-class citizen, and know we are falling short right now). We had some challenges supporting OKD 4.x because the masters in 4.x now require Red Hat CoreOS (and nodes it is highly recommended), but RHCOS was not yet freely available. This is obvoiusly a big problem. Now that Fedora CoreOS is out, there is a freely distributable host OS on which to build OKD, so it will be better supported and usable. FWIW I have a personal goal to have a production-ish OKD cluster running for myself by end of the year. I'll admit I am a little offended at being called a "proprietary IBM K8s distribution," but I don't think you meant to be offensive. IBM has nothing to do with OpenShift, beyond the fact that they are becoming customers of it. Every bit of our OpenShift code is open source and available. You are right that it's not in a production-usable state (although there are people using it) but it's a lot better than you'll get from other vendors. We are at least trying to get it to a usable state, unlike many of them. We are strapped for resources like everyone else, and K8s runs a mile a minute and requires significant effort to stay ahead). This space is still really young and really hot, and I am confident we'll get the open source in a good, usable, state, much like Fedora and CentOS are. I also don't think OpenShift is really all that expensive considering the enormous value it provides. The value really does shine at scale, and may not be there for smaller shops. I don't blame you for waiting, I probably would too. Our current offering is made for enterprise scale, so isn't tailored to everyone. I've heard OpenShift online has gotten better, but haven't tried it myself. Eventually I plan to run all my personal apps on OKD (I have maybe a dozen, mostly with the only users being me and my family), but until then I've been using podman pods with systemd, which will be trivial to port to OKD once it's in a good state.
- JeremyNT 7y ago
  Yes, I realize I will come across as being overly negative here, and I apologize for this. It's not that openshift is bad per se, I just don't imagine it solves many problems an org that is fretting about lock in or gcp pricing will have. Such an org is probably cost sensitive and looking for flexibility, but openshift is expensive, and if you adopt its differentiating features you are de facto locking yourself in. And if you do not leverage those features out of a desire to avoid lock in, you are effectively paying a whole lot just for k8s support... And I really should say, for certain orgs (especially bigcos) this may well be worth it, I just don't think it is a good option for anybody worried primarily about avoiding vendor lock in and keeping costs in check.
- shaklee3 7y ago
  Honestly, the fact that it requires rhel or centos was enough to make it not feasible for us. I wish that would change, since I can't think of any reason the distribution should affect openshift.
  - freedomben 7y ago
    There are several reasons, many of them are that the nodes themselves are managed by OpenShift operators. If you run (as cluster admin) `oc get clusteroperators` you'll see plenty that are for hosts, such as an upgrader and a tuner. If the operators had to be distro agnostic it would be a support nightmare, and we wouldn't be able to do it. With RHCOS (immutable OS) we also have enough guarantees to safely upgrade systems without human intervention. Can you imagine doing that in an Enterprise environment while trying to support multiple distributions? I can't.
    - shaklee3 7y ago
      Can you describe what kind of tuning the rhel tuning tools do that are not available using the normal kernel constructs? Last I checked tuna and others did everything you could do in Ubuntu, but without knowing the guts of the system. Again, I think the idea of OS is great, but you've lost us, and likely other big customers because of that restriction. Having old kernels is just not an option for some people.

rcarmo 7y ago

I work for Microsoft and I quite like OpenShift (although we have AKS as the default, "vanilla" managed k8s service, you can also run OpenShift on Azure). Sure it's opinionated, but at this point, which flavor of k8s isn't? Even k3s (which I play around with on https://github.com/rcarmo/azure-k3s-cluster https://github.com/rcarmo/azure-k3s-cluster) has its quirks. Everyone who's targeting the enterprise market has a "value-added" brew of k8s at this point, so kudos to RH for the engineering effort :)

freedomben 7y ago

Thanks for the edits (and acknowledging our contributions). I wasn't sure if you were just trolling or not before, so I didn't want to engage. Your concern is valid, and I agree with you that OKD is not supported enough. I have my own theories as to why, but I will keep my criticism "in the family" (but do know there are people that want to see OKD be a first-class citizen, and know we are falling short right now). We had some challenges supporting OKD 4.x because the masters in 4.x now require Red Hat CoreOS (and nodes it is highly recommended), but RHCOS was not yet freely available. This is obvoiusly a big problem. Now that Fedora CoreOS is out, there is a freely distributable host OS on which to build OKD, so it will be better supported and usable. FWIW I have a personal goal to have a production-ish OKD cluster running for myself by end of the year. I'll admit I am a little offended at being called a "proprietary IBM K8s distribution," but I don't think you meant to be offensive. IBM has nothing to do with OpenShift, beyond the fact that they are becoming customers of it. Every bit of our OpenShift code is open source and available. You are right that it's not in a production-usable state (although there are people using it) but it's a lot better than you'll get from other vendors. We are at least trying to get it to a usable state, unlike many of them. We are strapped for resources like everyone else, and K8s runs a mile a minute and requires significant effort to stay ahead). This space is still really young and really hot, and I am confident we'll get the open source in a good, usable, state, much like Fedora and CentOS are. I also don't think OpenShift is really all that expensive considering the enormous value it provides. The value really does shine at scale, and may not be there for smaller shops. I don't blame you for waiting, I probably would too. Our current offering is made for enterprise scale, so isn't tailored to everyone. I've heard OpenShift online has gotten better, but haven't tried it myself. Eventually I plan to run all my personal apps on OKD (I have maybe a dozen, mostly with the only users being me and my family), but until then I've been using podman pods with systemd, which will be trivial to port to OKD once it's in a good state.

JeremyNT 7y ago
Yes, I realize I will come across as being overly negative here, and I apologize for this. It's not that openshift is bad per se, I just don't imagine it solves many problems an org that is fretting about lock in or gcp pricing will have. Such an org is probably cost sensitive and looking for flexibility, but openshift is expensive, and if you adopt its differentiating features you are de facto locking yourself in. And if you do not leverage those features out of a desire to avoid lock in, you are effectively paying a whole lot just for k8s support... And I really should say, for certain orgs (especially bigcos) this may well be worth it, I just don't think it is a good option for anybody worried primarily about avoiding vendor lock in and keeping costs in check.
shaklee3 7y ago
Honestly, the fact that it requires rhel or centos was enough to make it not feasible for us. I wish that would change, since I can't think of any reason the distribution should affect openshift.
- freedomben 7y ago
  There are several reasons, many of them are that the nodes themselves are managed by OpenShift operators. If you run (as cluster admin) `oc get clusteroperators` you'll see plenty that are for hosts, such as an upgrader and a tuner. If the operators had to be distro agnostic it would be a support nightmare, and we wouldn't be able to do it. With RHCOS (immutable OS) we also have enough guarantees to safely upgrade systems without human intervention. Can you imagine doing that in an Enterprise environment while trying to support multiple distributions? I can't.
  - shaklee3 7y ago
    Can you describe what kind of tuning the rhel tuning tools do that are not available using the normal kernel constructs? Last I checked tuna and others did everything you could do in Ubuntu, but without knowing the guts of the system. Again, I think the idea of OS is great, but you've lost us, and likely other big customers because of that restriction. Having old kernels is just not an option for some people.