6 ms·

Proxmox VE 6.2

95 points by lysp 6y ago

tlamponi 6y ago
Community forum thread open for discussion: https://forum.proxmox.com/threads/proxmox-ve-6-2-released.69647/ https://forum.proxmox.com/threads/proxmox-ve-6-2-released.69...
unixhero 6y ago
Great hypervisor solution! Really excited about this release.
polskibus 6y ago
Anyone using it with Kubernetes and Ceph in production? What are the advantages to other open source? What's worse in Proxmox than in Hyper-V or VMSphere?
- tyingq 6y ago
  "What are the advantages to other open source?" I don't think there is another notable, proven, open source clustered manager of both actual VMs and containers. OpenNebula might be the closest: https://opennebula.io/get-your-hands-on-v-5-8-edge/ https://opennebula.io/get-your-hands-on-v-5-8-edge/ LXD is similar to proxmox if you only need containers, and pretty slick. They use their own distributed version of sqlite to propagate state, so it's much simpler than proxmox.
  - elcomet 6y ago
    Wouldn't OpenStack be an alternative ?
    - tyingq 6y ago
      Ah, yes, that's true. I'd left it out because it seems more focused on the end managed services, but you could use "Nova" and "Zun" more directly.
    - ymse 6y ago
      Only if you don't care about stability or maintenance costs.
  - ymse 6y ago
    > I don't think there is another notable, proven, open source clustered manager of both actual VMs and containers. Googles Ganeti is strikingly similar to Proxmox, to the point where I suspect Proxmox started as a reimplementation: http://www.ganeti.org/ http://www.ganeti.org/ (developed at https://github.com/ganeti/ganeti https://github.com/ganeti/ganeti) It does not come with a GUI, though there are a few third party ones. It has API endpoints for everything, a fully featured CLI client, and is extremely stable. Googles marketing department does not want you to know this, but Ganeti runs most of its internal infrastructure (not public facing stuff). No affiliation, just a happy user.
    - tyingq 6y ago
      Interesting...they don't seem to market it well, I hadn't heard of it. Looks like it started in 2007, where Proxmox started in 2008. It doesn't seem to support containers though. Just Xen and KVM.
      - ymse 6y ago
        LXC is supported too. Thanks for checking the dates, I thought Proxmox was newer.
    - tlamponi 6y ago
      Proxmox VE dev here. Honestly, heard of Ganeti the first time today. The basis of Proxmox VE is the real time replicated clustered configuration file system: https://pve.proxmox.com/pve-docs/chapter-pmxcfs.html https://pve.proxmox.com/pve-docs/chapter-pmxcfs.html I do not see anything similar for ganeti, also not much similar in general besides that it manages VMs?
      - polskibus 6y ago
        Hi there! Are there any plans for Proxmox to support Firecracker (the Amazon's lightweight VM for containers).?
      - ymse 6y ago
        Hi, thanks for the link, very interesting. Indeed the similarity with Ganeti is only on the surface: being a scalable infrastructure management tool, DRBD as a first-class supported disk backend, etc. The cluster replication and consensus model is very different: Ganeti has a single "master" node that makes all the decisions and replicates cluster state to other nodes.
gbuk2013 6y ago
Proxmox is awesome - VMs and containers with an API that just works. Network file system with a UI that just works. We once built an internal system to make network solution demos in it - a VM image running Proxmox that would be configured with any number of other VMs and containers all inside of that image, so you could literally take the while demo environment with you on a USB stick when you presented it, work on it while on a plane etc. I also built a mini cloud on a Proxmox cluster using Ceph that would instantiate these images from a template using a web UI - a bit of Proxmox recursion fun. That was a cool project to work on. :)
icefo 6y ago
I love Proxmox but the absence of incremental backups is very annoying. If you have a large vm whose contents don't change much your options are limited as storage (edited: and bandwidth) is not infinite. I also don't want to use aryufan patches to enable incremental backup. I feel like they could probably partner with duplicacy to ship that feature. Duplicacy already released a paid version specifically designed to backup VMware hosts so it should be possible to adapt it to backup Proxmox hosts. (Thank you all for your comments ! They gave me some ideas)
- JustFinishedBSG 6y ago
  Use ZFS as backend and tada your problem stops existing
  - icefo 6y ago
    I could only use snapshots and regularly send the pool somewhere but it feel a bit hacky
    - jblwps 6y ago
      You don't send the whole pool--you send the datasets (ZFS filesystems) that you're concerned with.
- iggldiggl 6y ago
  I've been wondering about the very same problem (and stumbled across aryufan's patches as well), and discovered that they're now actually working on an expanded backup solution of their own: https://git.proxmox.com/?p=proxmox-backup.git;a=tree https://git.proxmox.com/?p=proxmox-backup.git;a=tree https://forum.proxmox.com/threads/proxmox-backup-server-and-client.68781/ https://forum.proxmox.com/threads/proxmox-backup-server-and-...
- beagle3 6y ago
  Use a CoW/snapshotting backing file system for your backup (btrfs/zfs), or one of the modern de-duplicating backup systems (bup/borg/restic), and the space taken will be proportional to the changed data, while still providing random-access to any snapshot -- Not much more you can expect in terms of space usage. snapshotting filesystems will be much more efficient in terms of time/effort compared to de-duplicating later, but other than that, all of these solutions are mostly equivalent.
atomicnumber3 6y ago
I've been using proxmox in my homelab for ~2y, and was doing everything bare metal for the ~3y prior to that (which is when I started my homelab). At this point, I'd never use anything else. Proxmox "just works" in the best sense of the phrase. Never had uptime issues or anything (knock on wood). The only bare-metal install at this point is my plex server, which has a special level of importance in my close and extended family, so I baby it a little bit. A lotta bit. I use it for a lot of things - a windows VM for running the mega client (so it's always on + isn't on a personal machine to snoop around), a postgres server that I use for the dev/qa versions of my personal stuff, a jenkins+wekan server, a vm just for minecraft servers, etc etc. It's so easy to just spin up a new VM when I need a clean slate for something random. If I at some point actually start my own business, we'd be a proxmox shop for any non-cloud infra. And the cost-effectiveness of even a handful of 1-gen-old Dell servers on-prem (for things that don't need super high availability) is really hard to overstate when your budget isn't attached to VC funding. I would not run prod infra on-prem at this point, though: DigitalOcean makes the cloud far too affordable for that. But I'd probably have all CI, data analysis, etc etc local for a good long while. One gripe I have is their subscription/license nag. It's annoying and tacky. I would gripe less if they had an option that made it feasible for me, a guy running it in his basement with 0 profit motive, to buy a license. Right now, it'd be (~$85 * 2 sockets * 4 servers) for me. Sorry, that's not happening, I could actually use DigitalOcean cheaper than that (and maybe even AWS). And it would be a huge % of my house's tech-related OpEx (compare that to ~80/yr for domains, $5/mo for my one droplet, $100/mo for internet, etc). If I had to name a price, I'd say $25/yr would be about the most I'd drop (as a guy with a basement server room, of course). Contrast this with Plex which I got a lifetime license for for like 150 bucks or something, and plex doesn't even really have an alternative. I could live without proxmox (libvirt is really annoying, but not that annoying), but I couldn't live without plex. The pricing seems fine for businesses, though. I'd argue it's probably good RoI.
- lysp 6y ago
  > One gripe I have is their subscription/license nag. It's annoying and tacky root@pve01:~# cd /etc/cron.daily/ root@pve01:/etc/cron.daily# cat proxmox-nosub #!/bin/sh sed -i "s|if (data.status !== 'Active')|if (false \&\& data.status !== 'Active')|g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js I put it in daily in case I forget to run it during an update.
unixhero 6y ago
Does anybody here know how to encrypt the root partition of the Proxmox install? Because data at rest needs to be encrypted.
- grawlinson 6y ago
  You could install Debian with an encrypted root, then add the Proxmox repositories afterwards.
  - unixhero 6y ago
    Ah cool. Will try that, thanks a bunch.
apple4ever 6y ago
I really like Proxmox, except for one glaring issue: dealign with storage. It is all command line for anything else but local storage. Trying to set up iSCSI can be frustrating. I hope 7.0 adds that feature. I used Proxmox in production on a decently sized ecommerce site, and it worked well otherwise.

95 points by lysp 6y ago

Community forum thread open for discussion: https://forum.proxmox.com/threads/proxmox-ve-6-2-released.69647/ https://forum.proxmox.com/threads/proxmox-ve-6-2-released.69...

unixhero 6y ago

Great hypervisor solution! Really excited about this release.

polskibus 6y ago

Anyone using it with Kubernetes and Ceph in production? What are the advantages to other open source? What's worse in Proxmox than in Hyper-V or VMSphere?

tyingq 6y ago
"What are the advantages to other open source?" I don't think there is another notable, proven, open source clustered manager of both actual VMs and containers. OpenNebula might be the closest: https://opennebula.io/get-your-hands-on-v-5-8-edge/ https://opennebula.io/get-your-hands-on-v-5-8-edge/ LXD is similar to proxmox if you only need containers, and pretty slick. They use their own distributed version of sqlite to propagate state, so it's much simpler than proxmox.
- elcomet 6y ago
  Wouldn't OpenStack be an alternative ?
  - tyingq 6y ago
    Ah, yes, that's true. I'd left it out because it seems more focused on the end managed services, but you could use "Nova" and "Zun" more directly.
  - ymse 6y ago
    Only if you don't care about stability or maintenance costs.
- ymse 6y ago
  > I don't think there is another notable, proven, open source clustered manager of both actual VMs and containers. Googles Ganeti is strikingly similar to Proxmox, to the point where I suspect Proxmox started as a reimplementation: http://www.ganeti.org/ http://www.ganeti.org/ (developed at https://github.com/ganeti/ganeti https://github.com/ganeti/ganeti) It does not come with a GUI, though there are a few third party ones. It has API endpoints for everything, a fully featured CLI client, and is extremely stable. Googles marketing department does not want you to know this, but Ganeti runs most of its internal infrastructure (not public facing stuff). No affiliation, just a happy user.
  - tyingq 6y ago
    Interesting...they don't seem to market it well, I hadn't heard of it. Looks like it started in 2007, where Proxmox started in 2008. It doesn't seem to support containers though. Just Xen and KVM.
    - ymse 6y ago
      LXC is supported too. Thanks for checking the dates, I thought Proxmox was newer.
  - tlamponi 6y ago
    Proxmox VE dev here. Honestly, heard of Ganeti the first time today. The basis of Proxmox VE is the real time replicated clustered configuration file system: https://pve.proxmox.com/pve-docs/chapter-pmxcfs.html https://pve.proxmox.com/pve-docs/chapter-pmxcfs.html I do not see anything similar for ganeti, also not much similar in general besides that it manages VMs?
    - polskibus 6y ago
      Hi there! Are there any plans for Proxmox to support Firecracker (the Amazon's lightweight VM for containers).?
    - ymse 6y ago
      Hi, thanks for the link, very interesting. Indeed the similarity with Ganeti is only on the surface: being a scalable infrastructure management tool, DRBD as a first-class supported disk backend, etc. The cluster replication and consensus model is very different: Ganeti has a single "master" node that makes all the decisions and replicates cluster state to other nodes.

gbuk2013 6y ago

Proxmox is awesome - VMs and containers with an API that just works. Network file system with a UI that just works. We once built an internal system to make network solution demos in it - a VM image running Proxmox that would be configured with any number of other VMs and containers all inside of that image, so you could literally take the while demo environment with you on a USB stick when you presented it, work on it while on a plane etc. I also built a mini cloud on a Proxmox cluster using Ceph that would instantiate these images from a template using a web UI - a bit of Proxmox recursion fun. That was a cool project to work on. :)

icefo 6y ago

I love Proxmox but the absence of incremental backups is very annoying. If you have a large vm whose contents don't change much your options are limited as storage (edited: and bandwidth) is not infinite. I also don't want to use aryufan patches to enable incremental backup. I feel like they could probably partner with duplicacy to ship that feature. Duplicacy already released a paid version specifically designed to backup VMware hosts so it should be possible to adapt it to backup Proxmox hosts. (Thank you all for your comments ! They gave me some ideas)

JustFinishedBSG 6y ago
Use ZFS as backend and tada your problem stops existing
- icefo 6y ago
  I could only use snapshots and regularly send the pool somewhere but it feel a bit hacky
  - jblwps 6y ago
    You don't send the whole pool--you send the datasets (ZFS filesystems) that you're concerned with.
iggldiggl 6y ago
I've been wondering about the very same problem (and stumbled across aryufan's patches as well), and discovered that they're now actually working on an expanded backup solution of their own: https://git.proxmox.com/?p=proxmox-backup.git;a=tree https://git.proxmox.com/?p=proxmox-backup.git;a=tree https://forum.proxmox.com/threads/proxmox-backup-server-and-client.68781/ https://forum.proxmox.com/threads/proxmox-backup-server-and-...
beagle3 6y ago
Use a CoW/snapshotting backing file system for your backup (btrfs/zfs), or one of the modern de-duplicating backup systems (bup/borg/restic), and the space taken will be proportional to the changed data, while still providing random-access to any snapshot -- Not much more you can expect in terms of space usage. snapshotting filesystems will be much more efficient in terms of time/effort compared to de-duplicating later, but other than that, all of these solutions are mostly equivalent.

atomicnumber3 6y ago

I've been using proxmox in my homelab for ~2y, and was doing everything bare metal for the ~3y prior to that (which is when I started my homelab). At this point, I'd never use anything else. Proxmox "just works" in the best sense of the phrase. Never had uptime issues or anything (knock on wood). The only bare-metal install at this point is my plex server, which has a special level of importance in my close and extended family, so I baby it a little bit. A lotta bit. I use it for a lot of things - a windows VM for running the mega client (so it's always on + isn't on a personal machine to snoop around), a postgres server that I use for the dev/qa versions of my personal stuff, a jenkins+wekan server, a vm just for minecraft servers, etc etc. It's so easy to just spin up a new VM when I need a clean slate for something random. If I at some point actually start my own business, we'd be a proxmox shop for any non-cloud infra. And the cost-effectiveness of even a handful of 1-gen-old Dell servers on-prem (for things that don't need super high availability) is really hard to overstate when your budget isn't attached to VC funding. I would not run prod infra on-prem at this point, though: DigitalOcean makes the cloud far too affordable for that. But I'd probably have all CI, data analysis, etc etc local for a good long while. One gripe I have is their subscription/license nag. It's annoying and tacky. I would gripe less if they had an option that made it feasible for me, a guy running it in his basement with 0 profit motive, to buy a license. Right now, it'd be (~$85 * 2 sockets * 4 servers) for me. Sorry, that's not happening, I could actually use DigitalOcean cheaper than that (and maybe even AWS). And it would be a huge % of my house's tech-related OpEx (compare that to ~80/yr for domains, $5/mo for my one droplet, $100/mo for internet, etc). If I had to name a price, I'd say $25/yr would be about the most I'd drop (as a guy with a basement server room, of course). Contrast this with Plex which I got a lifetime license for for like 150 bucks or something, and plex doesn't even really have an alternative. I could live without proxmox (libvirt is really annoying, but not that annoying), but I couldn't live without plex. The pricing seems fine for businesses, though. I'd argue it's probably good RoI.

lysp 6y ago
> One gripe I have is their subscription/license nag. It's annoying and tacky root@pve01:~# cd /etc/cron.daily/ root@pve01:/etc/cron.daily# cat proxmox-nosub #!/bin/sh sed -i "s|if (data.status !== 'Active')|if (false \&\& data.status !== 'Active')|g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js I put it in daily in case I forget to run it during an update.

unixhero 6y ago

Does anybody here know how to encrypt the root partition of the Proxmox install? Because data at rest needs to be encrypted.

grawlinson 6y ago
You could install Debian with an encrypted root, then add the Proxmox repositories afterwards.
- unixhero 6y ago
  Ah cool. Will try that, thanks a bunch.

apple4ever 6y ago

I really like Proxmox, except for one glaring issue: dealign with storage. It is all command line for anything else but local storage. Trying to set up iSCSI can be frustrating. I hope 7.0 adds that feature. I used Proxmox in production on a decently sized ecommerce site, and it worked well otherwise.