5 ms·

I had to deal with it past 2 years - it is extremely bloated, not very well thought out and anything that needs decently performing storage is screwed over by G

I had to deal with it past 2 years - it is extremely bloated, not very well thought out and anything that needs decently performing storage is screwed over by GlusterFS and now I guess Ceph. All in all you really want to carefully analyze which apps you want to put on it and how much benefit you are really getting out of it - because it isn't free in any sense of the word - licensing, maintenance, operations - you are adding a crap ton of complexity and giving developers more ways to shoot themselves in the foot. [Wow, looks like we have OpenShift fans here. Still, I prefer leaving the f out of OpenShift like everyone who has actually worked on it trying to get complex apps running smoothly :)]

pestaa 6y ago
I want to get into OpenShift/OKD, but the distributed storage systems are quite frightening. There are plenty nightmare stories related to GlusterFS performance. However, strange as it sounds, I'm looking to deploy a cluster on a single node, primarily for the Kubernetes API & ecosystem (so many tooling options). What is your opinion on running K8s this way, and having only local RAID storage?
- blinkingled 6y ago
  I have tried running OKD/OpenShift locally - even using the latest retake on it - Code Ready Containers. It is way too resource intensive for a single VM. That is before you start the prometheus/monitoring/alerting stuff or metrics server - at that point your single VM is good as dead. You can just learn plain old K8S - get an Ubuntu VM and snap install microk8s. Add another vm as a node and it works well for most part and sans storage stuff (you can do NFS though) you can learn almost everything. After that catching up to OpenShift specifics is not that big of a deal - it's not all that different.
- oso2k 6y ago
  As mentioned, CodeReady Containers (CRC) is the easy button for a single node setup [0]. But it is heavy for a single node. Alternatives for labs and demos are learn.openshift.com [1] and the All-in-One VM blog [2]. Storage isn't too important in a single node setup unless you're looking for Metrics, Logging, and Monitoring to be turned on. You could use NFS if you have to. Just don't do that in Production, IMO. [0] https://developers.redhat.com/blog/2019/09/05/red-hat-openshift-4-on-your-laptop-introducing-red-hat-codeready-containers/ https://developers.redhat.com/blog/2019/09/05/red-hat-opensh... [1] https://learn.openshift.com/ https://learn.openshift.com/ [2] https://www.openshift.com/blog/revamped-openshift-all-in-one-aio-for-labs-and-fun https://www.openshift.com/blog/revamped-openshift-all-in-one...
- bennofs 6y ago
  This sounds like you want https://k3s.io https://k3s.io, a kubernetes distribution specially made for single node and probably much simpler.
oso2k 6y ago
Disclaimer: I work for Red Hat Consulting where my day job is 100% OpenShift. Storage isn't easy on OpenShift unless you're in one of the public clouds. Even then, there's caveats based on (mis-)use cases and what nots [0]. But on complexity, I think companies and developers do it to themselves by getting excited about 12 Factor Apps, Microservices, and Event Driven Architectures and not understanding what that all means. OpenShift & Kubernetes just make that easier to roll out. At the end of the day, OpenShift & Kubernetes is just ELBs, App Routers, scheduling, monitoring, and logging systems, some orchestration tools [1]. [0] https://docs.openshift.com/container-platform/4.3/scalability_and_performance/optimizing-storage.html https://docs.openshift.com/container-platform/4.3/scalabilit... [1] http://uncontained.io/articles/openshift-ha-installation/ http://uncontained.io/articles/openshift-ha-installation/
- blinkingled 6y ago
  > OpenShift & Kubernetes is just ELBs, App Routers, scheduling, monitoring, and logging systems, some orchestration tools True but consider this - there used to be people whose job it was to do most of that stuff and the app developers did not have to think about it for the most part. Kubernetes/OpenShift just moved all of the complexity on the developer's plate - and the mind numbing stuff that results from that is worthy of few articles. And to say most developers have no way to actually run this at their home and become familiar with the intricacies is even more asinine. This isn't a OpenShift/K8s critique per se but the whole idea that you can somehow make all the complexity disappear by writing more software and more abstractions and then let the developers deal with all things that organizations failed to account for in the need for cheap and fast - it doesn't really work all that well.
  - acdha 6y ago
    > > OpenShift & Kubernetes is just ELBs, App Routers, scheduling, monitoring, and logging systems, some orchestration tools > True but consider this - there used to be people whose job it was to do most of that stuff and the app developers did not have to think about it for the most part. Kubernetes/OpenShift just moved all of the complexity on the developer's plate - and the mind numbing stuff that results from that is worthy of few articles Ignoring OpenShift, I don't think the comparison is quite that simple because it's leaving out all of the drawbacks from that traditional approach. Instead of writing a few lines of YAML you have to wrangle time and support from central IT, debugging is often a multi-party finger-pointing exercise with access limitations, and because you're talking about separate services and usually hardware changes and updates are hard because they need to be coordinated across multiple applications and teams. That's not saying that there aren't legitimate areas for concern but I've seen fairly large amounts of developer time and downtime go into working around the limitations of that model (“<vendor> isn't compliant with this 15 year old RFC so we need to distribute forks of these 15 common tools for years until we can replace those $$$ appliances”).

pestaa 6y ago

I want to get into OpenShift/OKD, but the distributed storage systems are quite frightening. There are plenty nightmare stories related to GlusterFS performance. However, strange as it sounds, I'm looking to deploy a cluster on a single node, primarily for the Kubernetes API & ecosystem (so many tooling options). What is your opinion on running K8s this way, and having only local RAID storage?

blinkingled 6y ago
I have tried running OKD/OpenShift locally - even using the latest retake on it - Code Ready Containers. It is way too resource intensive for a single VM. That is before you start the prometheus/monitoring/alerting stuff or metrics server - at that point your single VM is good as dead. You can just learn plain old K8S - get an Ubuntu VM and snap install microk8s. Add another vm as a node and it works well for most part and sans storage stuff (you can do NFS though) you can learn almost everything. After that catching up to OpenShift specifics is not that big of a deal - it's not all that different.
oso2k 6y ago
As mentioned, CodeReady Containers (CRC) is the easy button for a single node setup [0]. But it is heavy for a single node. Alternatives for labs and demos are learn.openshift.com [1] and the All-in-One VM blog [2]. Storage isn't too important in a single node setup unless you're looking for Metrics, Logging, and Monitoring to be turned on. You could use NFS if you have to. Just don't do that in Production, IMO. [0] https://developers.redhat.com/blog/2019/09/05/red-hat-openshift-4-on-your-laptop-introducing-red-hat-codeready-containers/ https://developers.redhat.com/blog/2019/09/05/red-hat-opensh... [1] https://learn.openshift.com/ https://learn.openshift.com/ [2] https://www.openshift.com/blog/revamped-openshift-all-in-one-aio-for-labs-and-fun https://www.openshift.com/blog/revamped-openshift-all-in-one...
bennofs 6y ago
This sounds like you want https://k3s.io https://k3s.io, a kubernetes distribution specially made for single node and probably much simpler.

oso2k 6y ago

Disclaimer: I work for Red Hat Consulting where my day job is 100% OpenShift. Storage isn't easy on OpenShift unless you're in one of the public clouds. Even then, there's caveats based on (mis-)use cases and what nots [0]. But on complexity, I think companies and developers do it to themselves by getting excited about 12 Factor Apps, Microservices, and Event Driven Architectures and not understanding what that all means. OpenShift & Kubernetes just make that easier to roll out. At the end of the day, OpenShift & Kubernetes is just ELBs, App Routers, scheduling, monitoring, and logging systems, some orchestration tools [1]. [0] https://docs.openshift.com/container-platform/4.3/scalability_and_performance/optimizing-storage.html https://docs.openshift.com/container-platform/4.3/scalabilit... [1] http://uncontained.io/articles/openshift-ha-installation/ http://uncontained.io/articles/openshift-ha-installation/

blinkingled 6y ago
> OpenShift & Kubernetes is just ELBs, App Routers, scheduling, monitoring, and logging systems, some orchestration tools True but consider this - there used to be people whose job it was to do most of that stuff and the app developers did not have to think about it for the most part. Kubernetes/OpenShift just moved all of the complexity on the developer's plate - and the mind numbing stuff that results from that is worthy of few articles. And to say most developers have no way to actually run this at their home and become familiar with the intricacies is even more asinine. This isn't a OpenShift/K8s critique per se but the whole idea that you can somehow make all the complexity disappear by writing more software and more abstractions and then let the developers deal with all things that organizations failed to account for in the need for cheap and fast - it doesn't really work all that well.
- acdha 6y ago
  > > OpenShift & Kubernetes is just ELBs, App Routers, scheduling, monitoring, and logging systems, some orchestration tools > True but consider this - there used to be people whose job it was to do most of that stuff and the app developers did not have to think about it for the most part. Kubernetes/OpenShift just moved all of the complexity on the developer's plate - and the mind numbing stuff that results from that is worthy of few articles Ignoring OpenShift, I don't think the comparison is quite that simple because it's leaving out all of the drawbacks from that traditional approach. Instead of writing a few lines of YAML you have to wrangle time and support from central IT, debugging is often a multi-party finger-pointing exercise with access limitations, and because you're talking about separate services and usually hardware changes and updates are hard because they need to be coordinated across multiple applications and teams. That's not saying that there aren't legitimate areas for concern but I've seen fairly large amounts of developer time and downtime go into working around the limitations of that model (“<vendor> isn't compliant with this 15 year old RFC so we need to distribute forks of these 15 common tools for years until we can replace those $$$ appliances”).