4 ms·

Serious question: What is the advantage of Proxmox for power linux users that can run a debian or ubuntu server box + KVM ? I'm always reading stuff about peop

by fb03 3y ago

Serious question: What is the advantage of Proxmox for power linux users that can run a debian or ubuntu server box + KVM ? I'm always reading stuff about people using proxmox but don't really understand its 'edge'

plagiarist 3y ago
Me either, although I am using it for my home. It does allow configuring VMs, containers, logical volumes, and such. I think you can have multiple proxmox in a cluster. I might not have reached the point where I am doing enough with it to understand the benefit.
Modified3019 3y ago
For me who uses it as a NAS/VM platform, it’s basically Debian with some virtualization relevant packages and an opt in kernel that are more reasonably up to date, with solid ZFS support out of the box so I don't care about the "update kernel, reboot, oops version mismatch I need to revert" nonsense. While things like TrueNAS exist with ZFS support, in the past iX systems has made some overeager patches to ZFS, to bring in new features that weren't appropriate to include yet IMO. I hate to dig up old news, but that's why I've avoided them so far. Basically I've got Proxmox "figured out", which lets me fuck around with the things I want, rather than the things I don’t want. This reduces my very subjective mental overhead. All the high availability stuff is wasted on me though, and I just disable the related services. Note to new users experimenting with proxmox The most common "gotcha" with proxmox, is that it does NOT automatically adjust network stuff after install. If you switch out a network card or change what physical port it's connected with, you will lose network access, and need to manually edit `/etc/network/interfaces` to make sure your using a correct port ID and/or assigned IP are valid. iface vmbr0 inet static ... bridge-ports <The port ID, like "enp130s0f0"> ...
- solarkraft 3y ago
  > While things like TrueNAS exist with ZFS support, in the past iX systems has made some overeager patches to ZFS, to bring in new features that weren't appropriate to include yet IMO. I hate to dig up old news, but that's why I've avoided them so far. I've also not enjoyed reading their forums at all, while information about Proxmox tends to be actually helpful rather than berating people over their hardware choice or spreading misinformation about ZFS.
pbnsh 3y ago
UI, zfs out of the box, has an api.
- cassianoleal 3y ago
  Also clustering, VM/LXC migration between nodes, backups, access control, software-defined firewall...
kube-system 3y ago
It's a product instead of a box of parts. My brain only has so much cognitive power. If I'm spinning up a VM, I'm trying to accomplish some other work.
- deleted 3y ago
  [deleted]
darken 3y ago
I was in a similar boat and finally got around to trying out Proxmox in the last year or so. I just wanted share my own experience here, since it's a bit different than the "standard" uses I've seen. I had been running a k3s cluster (k8s flavor) on some Raspberry Pis, but decided I needed some non-ARM nodes, and "invested" in a few low power "1L" AMD64 PCs (6-8 core + hyper-threading). I was initially going to just install Ubuntu and base my setup off my existing Ansible automation to make things less inefficient. But I figured I'd play around with Proxmox first and see if there was any benefit to using that as a base layer since I'd heard a lot about it. I'm so glad I did. I ended up learning quite a bit in the process. Some quick highlights about using Proxmox for VMs in general: * Proxmox supports creating a "cluster", so you can login though one machine to administer them all. You can conveniently "move" VMs between machines pretty seamlessly. * If you install the para-virtualization drivers for e.g. Windows or Ubuntu VMs, you can do pretty fast remote KVM. E.g. I could run Youtube on a Windows VM in my basement over "Spice" and it almost looks like it's running locally. (Not that it's a use case I care about, mostly just shows the fact it's performant.) In terms of actually getting around to deploying k3s on top of the infra: * I ended up learning HPC-Packer, and HPC-Terraform, which integrated nicely with my existing Ansible experience. * Packer turns an Ubuntu ISO + my "base" Ansible setup playbooks into a pre-baked machine template directly in Proxmox. (My local machine's Packer binary just orchestrates the process.) * Terraform deploys the machine template into the Proxmox cluster. Basically a config file of machine names + IPs + mac adresses, and a few other params and initial setup. * Ansible then installs any final dependencies (anything not in the base template), setups up the first k3s master, grabs the join token, and adds in 2 more master nodes for a proper `etcd` backend. * Ansible then installs my base Kubernetes services (cert-manager, Rancher, Longhron storage, etc) via running helm commands on one of the nodes. * This is where I'm at now; the next step is for me to deploy my existing Flux.cd-automated "Gitops" apps (built for ARM64+AMD64 via Gitlab runner, also in Proxmox). These _had_ been running on my now-quite-crusty-seeming Pi cluster. I can run a single command to delete all the VMs, and rebuild + setup everything (full HA cluster + apps deployed and running) from scratch in ~6 minutes without any manual input required from me, just a few secrets/params in a config file. This has made exploring the horizon of possibilities _so much easier_ without getting locked in; I can try to weird Longhorn storage configs, or try out k8s monitoring stacks without worrying about needing to "back out" my changes if I picked bad settings. (Just blow it up and try again!) I can change how VLANs are configured in early steps, or try adding a library to the base Ubuntu install cluster-wide super easily, etc. I am primarily a software-engineer, so it has been really nice to delve into the operational side of things, and get a proper reproducible setup. It really has transformed how I think about the cluster in that it's no longer a "thing to carefully maintain", but instead a great sandbox to explore AND deploy my own k8s applications on top of without playing cloud bills. My Proxmox journey in the past few months definitely turned into more than a rabbit hole than I'd expected.
znpy 3y ago
it makes hard stuff simple. yeah you can write weird shell script and re-implement hot vm migration (moving a physical machine from one physical host to another without shutting it down) but what's the point, really? you might as well use proxmox. if you ever actually need to see how it's done you could just see what's proxmox doing under the hood (it's open source after all)
peddling-brink 3y ago
Like most useful things, it provides abstraction, simplicity, and organization. It has a webui with a bunch of features. If you don't like or want those things, that's fine. This question feels like a "no true power user would...".
gosub100 3y ago
It's a sandbox that lets you play without stepping in the mud or throwing rocks. See my other post in this thread for what happens when you try to do it the hard way.
mrsilencedogood 3y ago
With proxmox, I click the plus sign, pick an iso image, cpu/mem, and i'm good to go and I have a remote console viewer and tons of other neat mgmt stuff. In the same amount of time, if I were diving in to DIYing it, I'd probably be about 5 minutes into reading the arch linux wiki page for KVM/qemu.
jtriangle 3y ago
Easier, and it's debian, so, stays easy, very low overhead. Like any distro, it has some quirks, but, those are worth dealing with because the day to day is almost always frictionless.

by fb03 3y ago

plagiarist 3y ago

Me either, although I am using it for my home. It does allow configuring VMs, containers, logical volumes, and such. I think you can have multiple proxmox in a cluster. I might not have reached the point where I am doing enough with it to understand the benefit.

Modified3019 3y ago

For me who uses it as a NAS/VM platform, it’s basically Debian with some virtualization relevant packages and an opt in kernel that are more reasonably up to date, with solid ZFS support out of the box so I don't care about the "update kernel, reboot, oops version mismatch I need to revert" nonsense. While things like TrueNAS exist with ZFS support, in the past iX systems has made some overeager patches to ZFS, to bring in new features that weren't appropriate to include yet IMO. I hate to dig up old news, but that's why I've avoided them so far. Basically I've got Proxmox "figured out", which lets me fuck around with the things I want, rather than the things I don’t want. This reduces my very subjective mental overhead. All the high availability stuff is wasted on me though, and I just disable the related services. Note to new users experimenting with proxmox The most common "gotcha" with proxmox, is that it does NOT automatically adjust network stuff after install. If you switch out a network card or change what physical port it's connected with, you will lose network access, and need to manually edit `/etc/network/interfaces` to make sure your using a correct port ID and/or assigned IP are valid. iface vmbr0 inet static ... bridge-ports <The port ID, like "enp130s0f0"> ...

solarkraft 3y ago
> While things like TrueNAS exist with ZFS support, in the past iX systems has made some overeager patches to ZFS, to bring in new features that weren't appropriate to include yet IMO. I hate to dig up old news, but that's why I've avoided them so far. I've also not enjoyed reading their forums at all, while information about Proxmox tends to be actually helpful rather than berating people over their hardware choice or spreading misinformation about ZFS.

pbnsh 3y ago

UI, zfs out of the box, has an api.

cassianoleal 3y ago
Also clustering, VM/LXC migration between nodes, backups, access control, software-defined firewall...

kube-system 3y ago

It's a product instead of a box of parts. My brain only has so much cognitive power. If I'm spinning up a VM, I'm trying to accomplish some other work.

deleted 3y ago
[deleted]

darken 3y ago

I was in a similar boat and finally got around to trying out Proxmox in the last year or so. I just wanted share my own experience here, since it's a bit different than the "standard" uses I've seen. I had been running a k3s cluster (k8s flavor) on some Raspberry Pis, but decided I needed some non-ARM nodes, and "invested" in a few low power "1L" AMD64 PCs (6-8 core + hyper-threading). I was initially going to just install Ubuntu and base my setup off my existing Ansible automation to make things less inefficient. But I figured I'd play around with Proxmox first and see if there was any benefit to using that as a base layer since I'd heard a lot about it. I'm so glad I did. I ended up learning quite a bit in the process. Some quick highlights about using Proxmox for VMs in general: * Proxmox supports creating a "cluster", so you can login though one machine to administer them all. You can conveniently "move" VMs between machines pretty seamlessly. * If you install the para-virtualization drivers for e.g. Windows or Ubuntu VMs, you can do pretty fast remote KVM. E.g. I could run Youtube on a Windows VM in my basement over "Spice" and it almost looks like it's running locally. (Not that it's a use case I care about, mostly just shows the fact it's performant.) In terms of actually getting around to deploying k3s on top of the infra: * I ended up learning HPC-Packer, and HPC-Terraform, which integrated nicely with my existing Ansible experience. * Packer turns an Ubuntu ISO + my "base" Ansible setup playbooks into a pre-baked machine template directly in Proxmox. (My local machine's Packer binary just orchestrates the process.) * Terraform deploys the machine template into the Proxmox cluster. Basically a config file of machine names + IPs + mac adresses, and a few other params and initial setup. * Ansible then installs any final dependencies (anything not in the base template), setups up the first k3s master, grabs the join token, and adds in 2 more master nodes for a proper `etcd` backend. * Ansible then installs my base Kubernetes services (cert-manager, Rancher, Longhron storage, etc) via running helm commands on one of the nodes. * This is where I'm at now; the next step is for me to deploy my existing Flux.cd-automated "Gitops" apps (built for ARM64+AMD64 via Gitlab runner, also in Proxmox). These _had_ been running on my now-quite-crusty-seeming Pi cluster. I can run a single command to delete all the VMs, and rebuild + setup everything (full HA cluster + apps deployed and running) from scratch in ~6 minutes without any manual input required from me, just a few secrets/params in a config file. This has made exploring the horizon of possibilities _so much easier_ without getting locked in; I can try to weird Longhorn storage configs, or try out k8s monitoring stacks without worrying about needing to "back out" my changes if I picked bad settings. (Just blow it up and try again!) I can change how VLANs are configured in early steps, or try adding a library to the base Ubuntu install cluster-wide super easily, etc. I am primarily a software-engineer, so it has been really nice to delve into the operational side of things, and get a proper reproducible setup. It really has transformed how I think about the cluster in that it's no longer a "thing to carefully maintain", but instead a great sandbox to explore AND deploy my own k8s applications on top of without playing cloud bills. My Proxmox journey in the past few months definitely turned into more than a rabbit hole than I'd expected.

znpy 3y ago

it makes hard stuff simple. yeah you can write weird shell script and re-implement hot vm migration (moving a physical machine from one physical host to another without shutting it down) but what's the point, really? you might as well use proxmox. if you ever actually need to see how it's done you could just see what's proxmox doing under the hood (it's open source after all)

peddling-brink 3y ago

Like most useful things, it provides abstraction, simplicity, and organization. It has a webui with a bunch of features. If you don't like or want those things, that's fine. This question feels like a "no true power user would...".

gosub100 3y ago

It's a sandbox that lets you play without stepping in the mud or throwing rocks. See my other post in this thread for what happens when you try to do it the hard way.

mrsilencedogood 3y ago

With proxmox, I click the plus sign, pick an iso image, cpu/mem, and i'm good to go and I have a remote console viewer and tons of other neat mgmt stuff. In the same amount of time, if I were diving in to DIYing it, I'd probably be about 5 minutes into reading the arch linux wiki page for KVM/qemu.

jtriangle 3y ago

Easier, and it's debian, so, stays easy, very low overhead. Like any distro, it has some quirks, but, those are worth dealing with because the day to day is almost always frictionless.