4 ms·

> Thinly provisioned compute in the presence of other VMs is one reason That's the only reason I can think of. > I think this is aimed at people with multiple

> Thinly provisioned compute in the presence of other VMs is one reason That's the only reason I can think of. > I think this is aimed at people with multiple proxmox nodes in a cluster I think so too, but then we're back to the original question again (i.e. why not just have kubernetes nodes instead of having proxmox nodes with kubernetes VMs on them).

maneesh 3y ago
If you host multiple proxmox nodes, you can host other VMs on each bare metal node in addition to the kid cluster, right?
- RamRodification 3y ago
  Yes. Seems reasonable if you have much more compute than you can saturate with just things in your kubernetes cluster. Otherwise just make those hosts kubernetes nodes instead and call it a day.
  - moondev 3y ago
    Why make pets when you are already set up for cattle
    - RamRodification 3y ago
      A kubernetes worker node (on bare metal) is not more of a pet than a proxmox node IMO. It's just an unnecessary layer if you're gonna fill it with kubernetes pods anyway. Again, only if you are not planning to run non-kubernetes VMs. If you are then Proxmox with kubernetes VMs seems good.
sgarland 3y ago
You can also do things like live VM migration, where the K8s nodes get transferred to a different Proxmox node without interruption. Arguably niche, but if you have a small cluster (I have 3 physical nodes in my Proxmox cluster at home), it's sometimes nice to be able to just shift everything to do maintenance without having to worry about losing quorum or disrupting workloads.
- RamRodification 3y ago
  True! Didn't consider that.
J_tt 3y ago
Something a lot of other comments seem to be skipping out on is storage, proxmox provides an easy hyperconverged option with its support for CEPH, which provides a much more battle tested option when compared to k8s solutions like longhorn.
- RamRodification 3y ago
  Good point

maneesh 3y ago

If you host multiple proxmox nodes, you can host other VMs on each bare metal node in addition to the kid cluster, right?

RamRodification 3y ago
Yes. Seems reasonable if you have much more compute than you can saturate with just things in your kubernetes cluster. Otherwise just make those hosts kubernetes nodes instead and call it a day.
- moondev 3y ago
  Why make pets when you are already set up for cattle
  - RamRodification 3y ago
    A kubernetes worker node (on bare metal) is not more of a pet than a proxmox node IMO. It's just an unnecessary layer if you're gonna fill it with kubernetes pods anyway. Again, only if you are not planning to run non-kubernetes VMs. If you are then Proxmox with kubernetes VMs seems good.

sgarland 3y ago

You can also do things like live VM migration, where the K8s nodes get transferred to a different Proxmox node without interruption. Arguably niche, but if you have a small cluster (I have 3 physical nodes in my Proxmox cluster at home), it's sometimes nice to be able to just shift everything to do maintenance without having to worry about losing quorum or disrupting workloads.

RamRodification 3y ago
True! Didn't consider that.

J_tt 3y ago

Something a lot of other comments seem to be skipping out on is storage, proxmox provides an easy hyperconverged option with its support for CEPH, which provides a much more battle tested option when compared to k8s solutions like longhorn.

RamRodification 3y ago
Good point