3 ms·

Openshift Virtualization is based on kubevirt not ovirt. The future of ovirt/rhv is uncertain as redhat has indicated openshift virtualization is the path forwa

by davkan 3y ago

Openshift Virtualization is based on kubevirt not ovirt. The future of ovirt/rhv is uncertain as redhat has indicated openshift virtualization is the path forward. And OKD is a mess as of a year ago when I last tried it. Red Hat seems to be uninterested in maintaining it as a viable platform out of the box. OKD/OCP is an opinionated distribution and many Kubernetes operators that would work fine out of the box on a cluster that deviated less from upstream need significant effort to get working on OKD. That’s not an issue on OCP as redhat maintains their own versions of major operators via OLM and their own catalog. But on OKD you don’t have access to the subscription catalog. And the free catalog is missing many operators or they’re extremely out of date. I needed a service mesh for my cluster and wanted to use istio. On OCP istio is Openshift Service Mesh, on OKD you must use the upstream project Maistra. I was having issues with installation when I realized that Maistra had not been updated in the catalog in like a year. There was no simple workaround. Redhat employees were aware of the mess and keen to address but management was not interested in devoting time to fix it. Also the OKD docs are just copy-paste from OCP and in many instances especially during install the instructions are just inaccurate. I’ve always used RH upstreams in my lab. Used ovirt briefly, keycloak for a good while, freeipa for a decade. All of those products are pretty much 1:1 full featured clones. Keycloak’s documentation is great. Freeipa is identical to IDM so you can use the RH docs. OKD is a far cry from any of those and I don’t think it’s worth looking at unless you are an organization like CERN who can devote the resources needed and has unique needs.

nanibot 3y ago
Fellow okd user here (at home). Just curious about what do you think of Talos?
- davkan 3y ago
  Currently using Talos. I like it but, k3s would be less effort. Unlike OKD, Talos is vanilla k8s, so most applications work out of the box. But you’ll still run into some hiccups when it comes to deploying applications that need host access like CNI/CSI etc. Most recently I wanted to use OVS bridges for kubevirt networking. Since there’s no way to install applications on talos you must deploy the OVS mgmt tools as a daemonset. I couldn’t get it working and eventually gave up on ovs. Now looking to use the macvtap cni, but haven’t started yet. When I was looking at storage Longhorn didn’t even support Talos, though it was being worked on. In short, if what you really want to do at the end of the day is run your applications, then k3s or kubeadm seem like the best bet to avoid getting bogged down in the lower level details. At least looking from the outside anyway, as I haven’t migrated off talos.
emptysongglass 3y ago
This may not be the answer you want to hear but just use real OpenShift, not upstream. OpenShift SAs and above have indicated on numerous calls with me that there is no intention to chase after users who use "real" OpenShift in their homelab setups. And why bother? We wouldn't otherwise pay them money and the knowledge we earn in our homelabs is transferable to our time at our companies that do pay them money.
- davkan 3y ago
  Yeah, I would agree OCP is the way to go if you want to use the RH solution. At the time I couldn't really find any information on whether updates and management would continue to work past the 60 day trial. In the end I didn't go that route because I typically try to make sure that core systems in my lab are FOSS and not in a licensing gray area. E.g. I use freeipa for my dns, certs, etc instead of the AD environment I also have running on windows server eval systems. Availability of non FOSS software is always subject to change, and I like the idea of being able to quickly rip out any non-kosher systems in my lab should I choose to start selling services or run a business off it.

by davkan 3y ago

nanibot 3y ago

Fellow okd user here (at home). Just curious about what do you think of Talos?

davkan 3y ago
Currently using Talos. I like it but, k3s would be less effort. Unlike OKD, Talos is vanilla k8s, so most applications work out of the box. But you’ll still run into some hiccups when it comes to deploying applications that need host access like CNI/CSI etc. Most recently I wanted to use OVS bridges for kubevirt networking. Since there’s no way to install applications on talos you must deploy the OVS mgmt tools as a daemonset. I couldn’t get it working and eventually gave up on ovs. Now looking to use the macvtap cni, but haven’t started yet. When I was looking at storage Longhorn didn’t even support Talos, though it was being worked on. In short, if what you really want to do at the end of the day is run your applications, then k3s or kubeadm seem like the best bet to avoid getting bogged down in the lower level details. At least looking from the outside anyway, as I haven’t migrated off talos.

emptysongglass 3y ago

This may not be the answer you want to hear but just use real OpenShift, not upstream. OpenShift SAs and above have indicated on numerous calls with me that there is no intention to chase after users who use "real" OpenShift in their homelab setups. And why bother? We wouldn't otherwise pay them money and the knowledge we earn in our homelabs is transferable to our time at our companies that do pay them money.

davkan 3y ago
Yeah, I would agree OCP is the way to go if you want to use the RH solution. At the time I couldn't really find any information on whether updates and management would continue to work past the 60 day trial. In the end I didn't go that route because I typically try to make sure that core systems in my lab are FOSS and not in a licensing gray area. E.g. I use freeipa for my dns, certs, etc instead of the AD environment I also have running on windows server eval systems. Availability of non FOSS software is always subject to change, and I like the idea of being able to quickly rip out any non-kosher systems in my lab should I choose to start selling services or run a business off it.