11 ms·

Proxmox VE 6.1 – Open-Source virtualization platform

152 points by tlamponi 7y ago

anon9001 7y ago
Proxmox is pretty cool, but it doesn't make sense unless you're running many machines and utilizing ceph. For a home user, Debian (or some other stable distro) is still going to be the best distro for doing hypervisor stuff.
- regecks 7y ago
  I run it on a single node to run LXC guests for all sorts of non-production workloads. That it takes care of all the networking, storage, OS templates, backups, KVM access and has a comprehensive web UI is all very useful to me. What else is there to take its place? oVirt can't do LXC.
  - anon9001 7y ago
    virt-manager can do lxc. I rdesktop to the server and set up things with virt-manager. You can also install pve on a debian system if you want the proxmox gui. If you want to do full disk encryption, that's the recommended approach, because proxmox's installer doesn't support it.
- BuildTheRobots 7y ago
  I'd disagree entirely. If you want an extremely easy to use gui that gives you access to KVM and LXC virtualisations, along with easy access web remote consoles, snapshots and backups, then Proxmox has been a fantastic go-to for over 6 years. My only objection is that it's a 'full-fat' linux distro that needs to run off a real disk (unlike ESXi), however on every other level it's far superior.
  - dngray 7y ago
    > Proxmox is pretty cool, but it doesn't make sense unless you're running many machines and utilizing ceph. For a home user, Debian (or some other stable distro) is still going to be the best distro for doing hypervisor stuff. > My only objection is that it's a 'full-fat' linux distro that needs to run off a real disk (unlike ESXi), however on every other level it's far superior. Why not Alpine Linux? I use that in diskless mode https://wiki.alpinelinux.org/wiki/Installation#diskless_mode https://wiki.alpinelinux.org/wiki/Installation#diskless_mode
    - BuildTheRobots 7y ago
      > If you want an extremely easy to use gui that gives you access to KVM and LXC virtualisations, along with easy access web remote consoles, snapshots and backups My understanding is that alpine is a very light weight linux ditro and isn't a productised, easy to use hypervisor with full control via a gui. It's a bit like me saying "I like music editing using xxx because it has a tonne of value added, makes it easy, but it requires an amount of hardware resource" and someone replying "pah, I just use a reel-to-reel, some sharp scissors and a roll of tape!" - unless I'm missing something, it's not really comparable. edit: as tfigment pointed out along side my comment, I'm slightly annoyed that proxmox is a full-fat linux install for the hypervisor. The comparison I tried to make was with vmware ESXi which boots from an SD card or USB stick and is almost entirely held in memory after boot. ESXi has a whole number of other issues attached to it, but the fact I can throw a £10 usb stick in a server and use that as a boot drive, rather than "wasting" two of my SATA/SAS bays to a pair of real disks with the hypervisor installed on is nice.
    - tfigment 7y ago
      They mean proxmox requires full debian. Pretty sure its not a choice. In VMs you can run anything qemu runs. Also no Docker. The devs are opinionated, not that that is necessarily an issue.
      - roboyoshi 7y ago
        Docker has been on the wishlist for a while but it seems like the developers refuse to integrate it. I just installed docker-ce manually and added portainer as a systemd service. That way I can manage containers with a 2nd webui.
        regecks 7y ago
        Oh, neat. I've been wanting to do something like that, but I was afraid that the combination of LXC + Docker would blow up due to networking or cgroups or something. Are you running Docker and LXC containers side by side? Any documentation on how you did it?
        7y ago
  - tfigment 7y ago
    Mostly agree but proxmox still has quite a few quirks. Its cloud-init support is difficult to use unless you just need hostname and single user and single NIC which does cover a lot of cases but not mine. Its web api is non-standard and cannot upload full cloud-init config files but requires files already on disk. Also no Docker, just LxC. I found ESXi free tier better for test lab for somethings like backup and snapshots but no creating vms from scratch via api was an issue which is why I used Proxmox in first place.
- F00Fbug 7y ago
  But you don't get that great UI. I'm the world's biggest Linux cheerleader; one thing you never hear is that almost every 'modern' Linux has a hypervisor built-in (KVM/QEMU) to it. PVE is basically Debian with a wonderful web-UI wrapped around KVM/QEMU. You don't really need it, but it sure makes life easy!
  - reinhardt1053 7y ago
    Unless you need clustering and shared storage. Without Proxmox you are gonna need to setup corosync/ceph yourself.
- unixhero 7y ago
  I disagree completely. It makes sense to use Proxmox at home or in a business on a lot of planes. Ceph usage is not required, and it comes with excellent ZFS support. There are a lot of other great features which Proxmox facilitates.
- Proven 7y ago
  Yes. It's nice for LXC and Linux VMs and use cases that fit its feature set, but not nearly as versatile and integrated as commercial hypervisors. Most enterprise users would eventually have to also have - or leave in place - an (enterprise) virtualization platform. For others, it may or may not work.
- gmuslera 7y ago
  You can see it as Debian, with a web gui for managing virtualized resources. Ceph is not required, but it is something nice to have if you have many hypervisors and no central storage. But it is meant for using the machine mainly as an hypervisor, not as a desktop with the ability to ocasionally run a few virtual machines (in that case kvm+virt_manager or virtualbox are more desktop friendly).
mvip 7y ago
Love Proxmox. It has its quirks but overall a great platform for home usage (labs etc) and for small businesses.
slifin 7y ago
I installed Proxmox at a previous job, doing snapshots in seconds on a database container was nice for backups and the UI makes you feel informed and in control I would recommend it for most businesses honestly
F00Fbug 7y ago
I can't say enough good things about PVE! I started managing VMware in 2003 and ESXi is a wonderful product. I was running ESXi at home on the free license which only gives you eight vCPUs (if I remember correctly). After hitting that limit, I slowly decided to move to PVE. I was able to migrate my machines quite easily and have never looked back. For so many use cases, PVE is just as good as ESXi. If you're looking to virtualize and have relatively straightforward requirements, it's worth a look. Free is great and it's rock solid. I made that switch three years ago and haven't had a single issue.
godzillabrennus 7y ago
Used proxmox in its earliest iterations within a startup and found it fantastic. Recently I tried it again when attempting to install Untangle on a system with EFI. Untangle didn’t support EFI so I needed to virtualize it. I found it too difficult to get software network adapters to work with different physical network cards. Switched over to VMware ESXi and had vswitch configured within minutes. YMMV and I’d still recommend Proxmox for simple virtualization projects.
gboone 7y ago
Can't say enough about how I appreciate Proxmox. I had tried Xenserver but it didn't play well with my setup. Proxmox is robust from GUI and CLI. I run two small businesses from my home and the setup allows me to efficiently host pFsense, a mail server, multiple web and DB servers. But even better for me, the ease and ability to pass through a GPU and run Windows in a VM while using modifiable resources makes it so flexible. I've been a customer for over two years and they provide support with a super reasonable yearly subscription of just $99. The forum is active and the updates are recent and up to speed.
- listic 7y ago
  What do you mean by "using modifiable resources"?
  - gboone 7y ago
    Yes, unclear sorry. I meant modifying the availability of resources, and simply praising the general value of VMs. But also in the cases when the VMs are never running at the same time. Sometimes the "shop" closes and gaming starts...
- dangerboysteve 7y ago
  The Xenserver fork is really great. https://xcp-ng.org/ https://xcp-ng.org/ very active community and timely updates without all the licensing restrictions.
- dan_quixote 7y ago
  > Proxmox is robust from GUI and CLI. I'm looking through the CLI docs now and I can't even figure out how to launch a VM. At face value, it doesn't look like a good CLI (well actually they list a bunch of CLI tools for VM management). Am I just looking in the wrong place? I also want the ability to run setup commands (think AWS user-data) at first boot. The docs indicate user-data support though cloud-init, but I see forum discussions that indicate that you can only set up basic things like IP addresses, SSH keys, etc. Is there a programmatic way to inject bash commands at first boot?
deleted 7y ago
[deleted]
mlrtime 7y ago
Proxmox running on a nuc i7 with 64GB of ram and a fast nvme drive. The system is very fast and takes very little space. Just wish I had an extra nic.
- moondev 7y ago
  You can easily add a USB gigabit nic, or even 10 GbE via TB3 https://medium.com/@moondev/my-adventure-adding-10gbe-networking-to-an-intel-nuc-for-esxi-via-thunderbolt-3-pcie-expansion-1d6a627ffea4 https://medium.com/@moondev/my-adventure-adding-10gbe-networ...
- roboyoshi 7y ago
  The Hades Canyon NUC actually has 2 RJ45 ports: https://simplynuc.co.uk/hades-canyon/ https://simplynuc.co.uk/hades-canyon/ But it does cost you some serious money. I have a small i3 NUC and would rather plugin a USB3.0 to ethernet adapter.
  - oneplane 7y ago
    I'm using a miniPCIe ethernet card and added a hole to the case. Works well, but wouldn't trust it in prod unless some serious testing was done.
  - walterbell 7y ago
    There's a dual (Realtek) NIC ASRock Ryzen Embedded for $410, don't know how well it supports Linux, https://www.newegg.com/p/N82E16856158065 https://www.newegg.com/p/N82E16856158065
gbuk2013 7y ago
Proxmox is awesome. :) Back several years ago I worked on a project to build a system for making demos of network solutions. We ran Proxmox inside of an ESX virtual machine and then used it to virtualise various pieces of the demo architecture. The ESX image could simply be coped to an external drive for the consultant to use. Ultimately the company went for a central-lab demo environment instead but it was pretty cool anyway. The demo itself was stored as a bunch of Ansible playbooks and assorted scrips so it was Git-friendly, to be applied to a generic ESX template. We built cool things like a canvas-based Visio-like UI that could be used to define VMs / containers / networks and provision at the touch of a button. Similarly the solution itself would be visualised using a diagram that supported dynamic components like links showing traffic, counters, graphs etc that would show traffic running between the VMs inside the demo. We also had a central server that would store pre-created demo templates using Ceph that could checked-out for use via a web UI. Fun times :)
- walterbell 7y ago
  > We built cool things like a canvas-based Visio-like UI that could be used to define VMs / containers / networks and provision at the touch of a button. Similarly the solution itself would be visualised using a diagram that supported dynamic components like links showing traffic, counters, graphs etc that would show traffic running between the VMs inside the demo. Nifty! What frontend/backend dev stack did you use? Does anything like that exist in OSS, for any virt platform?
  - gbuk2013 7y ago
    No, I built it for this project. The canvas diagram was plain JS and for logos we used SVGs from company official PowerPoint icon pack to make it look consistent. The backend was Node.js that ran Ansible playbooks that provisioned things via Proxmox API and then configured the VMs as needed.
sekh60 7y ago
I had a brief stint with proxmox a few weeks ago and didn't like it much. I didn't like the nag popup when you don't have a subscription (and I had installed the nosubscription package!). I migrated to it to see if it would be easier to manage than OpenStack, but I honestly found OpenStack easier to deal with, probably due to familiarity.
- roboyoshi 7y ago
  Yeah that nag is my only nitpick with Proxmox. It's a bit too much in the face everytime I login. You can easily disable it with a code snippet, but I think the devs should just advertise the non-subscription model differently.
roboyoshi 7y ago
I've been running proxmox since v3.0 with lots of success. I admit that back then you could break stuff easily if you did not know what you're doing. Over the time I've been running it continuously with every version in my homelab and it just got more awesome. I tried ESXI and Microsoft Server.. But in the end I always went back to Proxmox and the very stable debian that runs it. Now with the latest version ZFS encryption is also native and I have pretty much everything I could ask for. Can only recommend!
zigzaggy 7y ago
Just set up Proxmox on my home lab last week. I’m slowly ramping up the learning curve. Sounds like I made the right choice.
babayega2 7y ago
Last year I had applied for a job where we would have to manage Windows servers. Stuff with Active Directory and storage... I'm Linux guy but have extended experience in WinServer 2012. I just set up a server, boot ProxMox, spun up as many as Win Server 2016 instances I wanted to experiment with all the aspects of the new Win Server and guess what? I was second in the job interview. Thank you ProxMox. You is the best.
cremp 7y ago
Proxmox is good with the large exception of high speed interconnects. If I have Mellanox IB cards in my servers, proxmox fails to handle ipoib without a lot of legwork. Compare that to something like oVirt; that supports it out of the box. There is very little incentive for me to recommend a proxmox subscription to any of my clients because having >= 40 gbit interconnects is far better than using lags on single gbit. High traffic internal applications, (and migration!) benefit so much from those interconnects.
- arminiusreturns 7y ago
  I have run dual 100gb mellanox in it with no problems without the IPoIB. It seems a pretty specific problem that is probably related to the mellanox cards and not to proxmox, as similar bugs show up with mellanox in oVirt.
mister_hn 7y ago
We use it at work since Proxmox 3. The only thing I miss is a truly compatible Terraform module that lets you create accounts, machines and infrastructure. That would be cool
self_awareness 7y ago
I've used Proxmox on my dedicated host and it was working really well, however one problem I've encountered is that it doesn't really play well with guest OpenBSD (guest randomly was hanging, with no apparent pattern). But Linux and Windows worked perfectly.
- wil421 7y ago
  I agree, FreeBSD (FreeNAS) doesn’t play well with it and iSCSI did not work well according to a lot of forum postings and blogs. I wanted a robust NAS and doing it through a VM did not provide the peace of mind I was looking. The only upside is now I can build a dedicated visualization machine or grab some old enterprise hardware from Craigslist. My wife is not so amused.
ralala 7y ago
What is your experience with major proxmox and OS updates?
tyingq 7y ago
Probably one of the more notable current applications that still use a lot of Perl. The admin interface is mostly Perl, as are many of the command line utilities.
- dan_quixote 7y ago
  Zoneminder uses a lot of Perl still IIRC.
  - tyingq 7y ago
    Yep. Also, Booking.com is probably the most notable current Perl user.
    - _jal 7y ago
      Craigslist.
cevn 7y ago
I gave proxmox a solid try for virtualizing a mattermost instance. But it was just way too slow compared to bare metal. Chats took seconds to load instead of ms. I'm sure there's some explanation for it but I couldn't be assed to figure it out and reverted.
- isodude 7y ago
  You were probably missing the virtualization flag if it is that slow.
  - cevn 7y ago
    Hmm possible. I will give that a try.
zoobab 7y ago
The company behind it in Vienna tried to recruit developers for 3200eur bruto/month, I found this super low compared to current standards.
- deleted 7y ago
  [deleted]
MayeulC 7y ago
I'm currently using Yunohost, but I'm finding the system a bit brittle for some use-cases, and some choices rather arbitrary (which was also the point, as I initially chose it to spend less time managing my server). I would like something a bit more fleximble, and was seriously thinking about jumping to Guix or Nix. Does any of you have any input on this? Proxmox is also a candidate, as I could run my old Yunohost side-by-side with Nix, Guix, and others, I guess. I'm just afraid of overcomplexifying my nginx config. Allso, if I run Guix/Nix on Proxmox, I'll lose the declarative config advantage... Is the opposite possible?
- hippich 7y ago
  I am sure usecases are slightly different for these two. At least for me personally, "plug and play" offer was the key - so I could easily move stuff around and not think about it. And it has browser access so non-tech people can use email/Nextcloud/etc. But perhaps Yunohost can be used to model what you have in mind, and once solidified, use it as a base to build VM with precise setup. Ansible is perhaps the closest you can get to declarative config... See "sovereign" on github - I migrated from it to Yunohost, as things kept breaking and I was looking hands off setup (hence Yunohost), but given that Sovereign is ansible-based - perhaps you can leverage it.
Evidlo 7y ago
One thing I've never understood with Proxmox is why networking can't be configured host-side. It seems you are expected to go into each new VM and manually set network settings. Why not have a DHCP server on the host that hands out prescribed addresses by VM id? If Proxmox could do this in addition to maintaining port forwarding to these VMs I would be happy.
- adamfeldman 7y ago
  I've set up Proxmox a couple times. I do exactly what you described, using dnsmasq. There are a few blog posts out there, this is one [1] I used for inspiration. I too wish it was a built-in Proxmox feature, or at least officially documented... [1]: https://blog.jenningsga.com/private-network-with-proxmox/ https://blog.jenningsga.com/private-network-with-proxmox/
- altmind 7y ago
  you can configure the network in the guests in cloud-init tab.
whalesalad 7y ago
I cannot stand the Proxmox software. It’s a total pain in the ass to use. XCP-NG is also a nightmare due to how half baked Xen Orchestra is. VMWare ESXi is the only type 1 hypervisor that I would ever recommend to a fellow hacker. You can get a free license for personal use.
- GuyPostington 7y ago
  Can you give an example of the half baked-ness please?
  - whalesalad 7y ago
    Have you used it? It’s the perfect example of Twitter bootstrap being all the rope to hang yourself.
harryruhr 7y ago
What's wrong with using plain libvirt tools, like virsh? Is there some other advantage to Proxmox besides having a nice WebUI?
- tecleandor 7y ago
  Nobody said it's wrong. It's perfectly OK. Proxmox is precisely a nice web UI for that and a lot of stuff integrated, such as LXC, CEPH, HA, Live Migration...
tracker1 7y ago
Would be cool to see an addon for NAS services in Proxmox... I'm setting up a home server when I get the chance and just want it to run a VM, a handful of docker containers and provide NAS storage at home. In the end, will probably add Docker support, run the NAS in a container or vm and leave it at that.
- antsar 7y ago
  Why integrate the NAS? I run a Synology full of spinning rust for backups, archives, and shared file storage. It works reliably and is stupid simple to maintain. Adjacent to that is a server running Proxmox, and I'm currently working to add Kubernetes/Rancher as a layer on top of that. The Proxmox server has a local SSD array so the VMs have fast low-latency storage. VMs that touch larger files (media server, etc) have an NFS mount to the NAS. IMHO the financial/maintenance overhead of a second unit is totally outweighed by the easy opportunity to have two performance and reliability tiers (fast homelab vs reliable slow storage), ability to upgrade/change the two units independently of each other, and ability for cohabitants to use the NAS without having to understand/deal with the virtualization stuff.
  - tracker1 7y ago
    I have an older Synology NAS, it's really slow, and TBH need more compute for a few things at home... but don't want to spend on yet another box for this. The main role of the new server is NAS stuff, but if I can do a few other things in the background, so much the better. I'll be running proxmox and VMs off of an nvme drive in a 4x pcie adapter, and the NAS drives using the hotswap bays in the front.. it's a used R710 (iirc), and has 6 3.5" bays (came with 2TB drives, but replacing the raid controller with one that will support larger drives and upgrading to 8TB as time/cost permits). I could totally drop the home server compute requirement before the NAS requirement as my desktop could handle those chores.
    - unethical_ban 7y ago
      TBH FreeNAS is a great NAS/SAN solution. I'm running a 4x4 RAIDZ2 (8TB effective) with a 10 year old quad core AMD and 16GB DDR2 RAM. If you are okay with the limited VM hosting capabilities of Freenas (headless VMs and containers on the same network as the file server) then you can use it for that as well.
apple4ever 7y ago
Proxmox is pretty nice, except it has two glaring issues: Setting up networks is a pain in the butt. Setting up storage is even worse. Both really require the command line, which shouldn’t be necessary. I ran Proxmox in production for a year for a small ecom company, but if I had to do it again, I’d go vSphere. It’s worth it.
- apple4ever 7y ago
  I just read all the release notes and this reduces the biggest issue with networking: > If the package ifupdown2 of the Debian network interface manager is installed, it’s now possible to change the network configuration and reload it in the Proxmox web interface without a reboot.

152 points by tlamponi 7y ago

Proxmox is pretty cool, but it doesn't make sense unless you're running many machines and utilizing ceph. For a home user, Debian (or some other stable distro) is still going to be the best distro for doing hypervisor stuff.

regecks 7y ago
I run it on a single node to run LXC guests for all sorts of non-production workloads. That it takes care of all the networking, storage, OS templates, backups, KVM access and has a comprehensive web UI is all very useful to me. What else is there to take its place? oVirt can't do LXC.
- anon9001 7y ago
  virt-manager can do lxc. I rdesktop to the server and set up things with virt-manager. You can also install pve on a debian system if you want the proxmox gui. If you want to do full disk encryption, that's the recommended approach, because proxmox's installer doesn't support it.
BuildTheRobots 7y ago
I'd disagree entirely. If you want an extremely easy to use gui that gives you access to KVM and LXC virtualisations, along with easy access web remote consoles, snapshots and backups, then Proxmox has been a fantastic go-to for over 6 years. My only objection is that it's a 'full-fat' linux distro that needs to run off a real disk (unlike ESXi), however on every other level it's far superior.
- dngray 7y ago
  > Proxmox is pretty cool, but it doesn't make sense unless you're running many machines and utilizing ceph. For a home user, Debian (or some other stable distro) is still going to be the best distro for doing hypervisor stuff. > My only objection is that it's a 'full-fat' linux distro that needs to run off a real disk (unlike ESXi), however on every other level it's far superior. Why not Alpine Linux? I use that in diskless mode https://wiki.alpinelinux.org/wiki/Installation#diskless_mode https://wiki.alpinelinux.org/wiki/Installation#diskless_mode
  - BuildTheRobots 7y ago
    > If you want an extremely easy to use gui that gives you access to KVM and LXC virtualisations, along with easy access web remote consoles, snapshots and backups My understanding is that alpine is a very light weight linux ditro and isn't a productised, easy to use hypervisor with full control via a gui. It's a bit like me saying "I like music editing using xxx because it has a tonne of value added, makes it easy, but it requires an amount of hardware resource" and someone replying "pah, I just use a reel-to-reel, some sharp scissors and a roll of tape!" - unless I'm missing something, it's not really comparable. edit: as tfigment pointed out along side my comment, I'm slightly annoyed that proxmox is a full-fat linux install for the hypervisor. The comparison I tried to make was with vmware ESXi which boots from an SD card or USB stick and is almost entirely held in memory after boot. ESXi has a whole number of other issues attached to it, but the fact I can throw a £10 usb stick in a server and use that as a boot drive, rather than "wasting" two of my SATA/SAS bays to a pair of real disks with the hypervisor installed on is nice.
  - tfigment 7y ago
    They mean proxmox requires full debian. Pretty sure its not a choice. In VMs you can run anything qemu runs. Also no Docker. The devs are opinionated, not that that is necessarily an issue.
    - roboyoshi 7y ago
      Docker has been on the wishlist for a while but it seems like the developers refuse to integrate it. I just installed docker-ce manually and added portainer as a systemd service. That way I can manage containers with a 2nd webui.
      - regecks 7y ago
        Oh, neat. I've been wanting to do something like that, but I was afraid that the combination of LXC + Docker would blow up due to networking or cgroups or something. Are you running Docker and LXC containers side by side? Any documentation on how you did it?
      - 7y ago
- tfigment 7y ago
  Mostly agree but proxmox still has quite a few quirks. Its cloud-init support is difficult to use unless you just need hostname and single user and single NIC which does cover a lot of cases but not mine. Its web api is non-standard and cannot upload full cloud-init config files but requires files already on disk. Also no Docker, just LxC. I found ESXi free tier better for test lab for somethings like backup and snapshots but no creating vms from scratch via api was an issue which is why I used Proxmox in first place.
F00Fbug 7y ago
But you don't get that great UI. I'm the world's biggest Linux cheerleader; one thing you never hear is that almost every 'modern' Linux has a hypervisor built-in (KVM/QEMU) to it. PVE is basically Debian with a wonderful web-UI wrapped around KVM/QEMU. You don't really need it, but it sure makes life easy!
- reinhardt1053 7y ago
  Unless you need clustering and shared storage. Without Proxmox you are gonna need to setup corosync/ceph yourself.
unixhero 7y ago
I disagree completely. It makes sense to use Proxmox at home or in a business on a lot of planes. Ceph usage is not required, and it comes with excellent ZFS support. There are a lot of other great features which Proxmox facilitates.
Proven 7y ago
Yes. It's nice for LXC and Linux VMs and use cases that fit its feature set, but not nearly as versatile and integrated as commercial hypervisors. Most enterprise users would eventually have to also have - or leave in place - an (enterprise) virtualization platform. For others, it may or may not work.
gmuslera 7y ago
You can see it as Debian, with a web gui for managing virtualized resources. Ceph is not required, but it is something nice to have if you have many hypervisors and no central storage. But it is meant for using the machine mainly as an hypervisor, not as a desktop with the ability to ocasionally run a few virtual machines (in that case kvm+virt_manager or virtualbox are more desktop friendly).

mvip 7y ago

Love Proxmox. It has its quirks but overall a great platform for home usage (labs etc) and for small businesses.

slifin 7y ago

I installed Proxmox at a previous job, doing snapshots in seconds on a database container was nice for backups and the UI makes you feel informed and in control I would recommend it for most businesses honestly

F00Fbug 7y ago

I can't say enough good things about PVE! I started managing VMware in 2003 and ESXi is a wonderful product. I was running ESXi at home on the free license which only gives you eight vCPUs (if I remember correctly). After hitting that limit, I slowly decided to move to PVE. I was able to migrate my machines quite easily and have never looked back. For so many use cases, PVE is just as good as ESXi. If you're looking to virtualize and have relatively straightforward requirements, it's worth a look. Free is great and it's rock solid. I made that switch three years ago and haven't had a single issue.

godzillabrennus 7y ago

Used proxmox in its earliest iterations within a startup and found it fantastic. Recently I tried it again when attempting to install Untangle on a system with EFI. Untangle didn’t support EFI so I needed to virtualize it. I found it too difficult to get software network adapters to work with different physical network cards. Switched over to VMware ESXi and had vswitch configured within minutes. YMMV and I’d still recommend Proxmox for simple virtualization projects.

gboone 7y ago

Can't say enough about how I appreciate Proxmox. I had tried Xenserver but it didn't play well with my setup. Proxmox is robust from GUI and CLI. I run two small businesses from my home and the setup allows me to efficiently host pFsense, a mail server, multiple web and DB servers. But even better for me, the ease and ability to pass through a GPU and run Windows in a VM while using modifiable resources makes it so flexible. I've been a customer for over two years and they provide support with a super reasonable yearly subscription of just $99. The forum is active and the updates are recent and up to speed.

listic 7y ago
What do you mean by "using modifiable resources"?
- gboone 7y ago
  Yes, unclear sorry. I meant modifying the availability of resources, and simply praising the general value of VMs. But also in the cases when the VMs are never running at the same time. Sometimes the "shop" closes and gaming starts...
dangerboysteve 7y ago
The Xenserver fork is really great. https://xcp-ng.org/ https://xcp-ng.org/ very active community and timely updates without all the licensing restrictions.
dan_quixote 7y ago
> Proxmox is robust from GUI and CLI. I'm looking through the CLI docs now and I can't even figure out how to launch a VM. At face value, it doesn't look like a good CLI (well actually they list a bunch of CLI tools for VM management). Am I just looking in the wrong place? I also want the ability to run setup commands (think AWS user-data) at first boot. The docs indicate user-data support though cloud-init, but I see forum discussions that indicate that you can only set up basic things like IP addresses, SSH keys, etc. Is there a programmatic way to inject bash commands at first boot?

deleted 7y ago

[deleted]

mlrtime 7y ago

Proxmox running on a nuc i7 with 64GB of ram and a fast nvme drive. The system is very fast and takes very little space. Just wish I had an extra nic.

moondev 7y ago
You can easily add a USB gigabit nic, or even 10 GbE via TB3 https://medium.com/@moondev/my-adventure-adding-10gbe-networking-to-an-intel-nuc-for-esxi-via-thunderbolt-3-pcie-expansion-1d6a627ffea4 https://medium.com/@moondev/my-adventure-adding-10gbe-networ...
roboyoshi 7y ago
The Hades Canyon NUC actually has 2 RJ45 ports: https://simplynuc.co.uk/hades-canyon/ https://simplynuc.co.uk/hades-canyon/ But it does cost you some serious money. I have a small i3 NUC and would rather plugin a USB3.0 to ethernet adapter.
- oneplane 7y ago
  I'm using a miniPCIe ethernet card and added a hole to the case. Works well, but wouldn't trust it in prod unless some serious testing was done.
- walterbell 7y ago
  There's a dual (Realtek) NIC ASRock Ryzen Embedded for $410, don't know how well it supports Linux, https://www.newegg.com/p/N82E16856158065 https://www.newegg.com/p/N82E16856158065

gbuk2013 7y ago

Proxmox is awesome. :) Back several years ago I worked on a project to build a system for making demos of network solutions. We ran Proxmox inside of an ESX virtual machine and then used it to virtualise various pieces of the demo architecture. The ESX image could simply be coped to an external drive for the consultant to use. Ultimately the company went for a central-lab demo environment instead but it was pretty cool anyway. The demo itself was stored as a bunch of Ansible playbooks and assorted scrips so it was Git-friendly, to be applied to a generic ESX template. We built cool things like a canvas-based Visio-like UI that could be used to define VMs / containers / networks and provision at the touch of a button. Similarly the solution itself would be visualised using a diagram that supported dynamic components like links showing traffic, counters, graphs etc that would show traffic running between the VMs inside the demo. We also had a central server that would store pre-created demo templates using Ceph that could checked-out for use via a web UI. Fun times :)

walterbell 7y ago
> We built cool things like a canvas-based Visio-like UI that could be used to define VMs / containers / networks and provision at the touch of a button. Similarly the solution itself would be visualised using a diagram that supported dynamic components like links showing traffic, counters, graphs etc that would show traffic running between the VMs inside the demo. Nifty! What frontend/backend dev stack did you use? Does anything like that exist in OSS, for any virt platform?
- gbuk2013 7y ago
  No, I built it for this project. The canvas diagram was plain JS and for logos we used SVGs from company official PowerPoint icon pack to make it look consistent. The backend was Node.js that ran Ansible playbooks that provisioned things via Proxmox API and then configured the VMs as needed.

sekh60 7y ago

I had a brief stint with proxmox a few weeks ago and didn't like it much. I didn't like the nag popup when you don't have a subscription (and I had installed the nosubscription package!). I migrated to it to see if it would be easier to manage than OpenStack, but I honestly found OpenStack easier to deal with, probably due to familiarity.

roboyoshi 7y ago
Yeah that nag is my only nitpick with Proxmox. It's a bit too much in the face everytime I login. You can easily disable it with a code snippet, but I think the devs should just advertise the non-subscription model differently.

roboyoshi 7y ago

I've been running proxmox since v3.0 with lots of success. I admit that back then you could break stuff easily if you did not know what you're doing. Over the time I've been running it continuously with every version in my homelab and it just got more awesome. I tried ESXI and Microsoft Server.. But in the end I always went back to Proxmox and the very stable debian that runs it. Now with the latest version ZFS encryption is also native and I have pretty much everything I could ask for. Can only recommend!

zigzaggy 7y ago

Just set up Proxmox on my home lab last week. I’m slowly ramping up the learning curve. Sounds like I made the right choice.

babayega2 7y ago

Last year I had applied for a job where we would have to manage Windows servers. Stuff with Active Directory and storage... I'm Linux guy but have extended experience in WinServer 2012. I just set up a server, boot ProxMox, spun up as many as Win Server 2016 instances I wanted to experiment with all the aspects of the new Win Server and guess what? I was second in the job interview. Thank you ProxMox. You is the best.

cremp 7y ago

Proxmox is good with the large exception of high speed interconnects. If I have Mellanox IB cards in my servers, proxmox fails to handle ipoib without a lot of legwork. Compare that to something like oVirt; that supports it out of the box. There is very little incentive for me to recommend a proxmox subscription to any of my clients because having >= 40 gbit interconnects is far better than using lags on single gbit. High traffic internal applications, (and migration!) benefit so much from those interconnects.

arminiusreturns 7y ago
I have run dual 100gb mellanox in it with no problems without the IPoIB. It seems a pretty specific problem that is probably related to the mellanox cards and not to proxmox, as similar bugs show up with mellanox in oVirt.

mister_hn 7y ago

We use it at work since Proxmox 3. The only thing I miss is a truly compatible Terraform module that lets you create accounts, machines and infrastructure. That would be cool

self_awareness 7y ago

I've used Proxmox on my dedicated host and it was working really well, however one problem I've encountered is that it doesn't really play well with guest OpenBSD (guest randomly was hanging, with no apparent pattern). But Linux and Windows worked perfectly.

wil421 7y ago
I agree, FreeBSD (FreeNAS) doesn’t play well with it and iSCSI did not work well according to a lot of forum postings and blogs. I wanted a robust NAS and doing it through a VM did not provide the peace of mind I was looking. The only upside is now I can build a dedicated visualization machine or grab some old enterprise hardware from Craigslist. My wife is not so amused.

ralala 7y ago

What is your experience with major proxmox and OS updates?

tyingq 7y ago

Probably one of the more notable current applications that still use a lot of Perl. The admin interface is mostly Perl, as are many of the command line utilities.

dan_quixote 7y ago
Zoneminder uses a lot of Perl still IIRC.
- tyingq 7y ago
  Yep. Also, Booking.com is probably the most notable current Perl user.
  - _jal 7y ago
    Craigslist.

cevn 7y ago

I gave proxmox a solid try for virtualizing a mattermost instance. But it was just way too slow compared to bare metal. Chats took seconds to load instead of ms. I'm sure there's some explanation for it but I couldn't be assed to figure it out and reverted.

isodude 7y ago
You were probably missing the virtualization flag if it is that slow.
- cevn 7y ago
  Hmm possible. I will give that a try.

zoobab 7y ago

The company behind it in Vienna tried to recruit developers for 3200eur bruto/month, I found this super low compared to current standards.

deleted 7y ago
[deleted]

MayeulC 7y ago

I'm currently using Yunohost, but I'm finding the system a bit brittle for some use-cases, and some choices rather arbitrary (which was also the point, as I initially chose it to spend less time managing my server). I would like something a bit more fleximble, and was seriously thinking about jumping to Guix or Nix. Does any of you have any input on this? Proxmox is also a candidate, as I could run my old Yunohost side-by-side with Nix, Guix, and others, I guess. I'm just afraid of overcomplexifying my nginx config. Allso, if I run Guix/Nix on Proxmox, I'll lose the declarative config advantage... Is the opposite possible?

hippich 7y ago
I am sure usecases are slightly different for these two. At least for me personally, "plug and play" offer was the key - so I could easily move stuff around and not think about it. And it has browser access so non-tech people can use email/Nextcloud/etc. But perhaps Yunohost can be used to model what you have in mind, and once solidified, use it as a base to build VM with precise setup. Ansible is perhaps the closest you can get to declarative config... See "sovereign" on github - I migrated from it to Yunohost, as things kept breaking and I was looking hands off setup (hence Yunohost), but given that Sovereign is ansible-based - perhaps you can leverage it.

Evidlo 7y ago

One thing I've never understood with Proxmox is why networking can't be configured host-side. It seems you are expected to go into each new VM and manually set network settings. Why not have a DHCP server on the host that hands out prescribed addresses by VM id? If Proxmox could do this in addition to maintaining port forwarding to these VMs I would be happy.

adamfeldman 7y ago
I've set up Proxmox a couple times. I do exactly what you described, using dnsmasq. There are a few blog posts out there, this is one [1] I used for inspiration. I too wish it was a built-in Proxmox feature, or at least officially documented... [1]: https://blog.jenningsga.com/private-network-with-proxmox/ https://blog.jenningsga.com/private-network-with-proxmox/
altmind 7y ago
you can configure the network in the guests in cloud-init tab.

whalesalad 7y ago

I cannot stand the Proxmox software. It’s a total pain in the ass to use. XCP-NG is also a nightmare due to how half baked Xen Orchestra is. VMWare ESXi is the only type 1 hypervisor that I would ever recommend to a fellow hacker. You can get a free license for personal use.

GuyPostington 7y ago
Can you give an example of the half baked-ness please?
- whalesalad 7y ago
  Have you used it? It’s the perfect example of Twitter bootstrap being all the rope to hang yourself.

harryruhr 7y ago

What's wrong with using plain libvirt tools, like virsh? Is there some other advantage to Proxmox besides having a nice WebUI?

tecleandor 7y ago
Nobody said it's wrong. It's perfectly OK. Proxmox is precisely a nice web UI for that and a lot of stuff integrated, such as LXC, CEPH, HA, Live Migration...

tracker1 7y ago

Would be cool to see an addon for NAS services in Proxmox... I'm setting up a home server when I get the chance and just want it to run a VM, a handful of docker containers and provide NAS storage at home. In the end, will probably add Docker support, run the NAS in a container or vm and leave it at that.

antsar 7y ago
Why integrate the NAS? I run a Synology full of spinning rust for backups, archives, and shared file storage. It works reliably and is stupid simple to maintain. Adjacent to that is a server running Proxmox, and I'm currently working to add Kubernetes/Rancher as a layer on top of that. The Proxmox server has a local SSD array so the VMs have fast low-latency storage. VMs that touch larger files (media server, etc) have an NFS mount to the NAS. IMHO the financial/maintenance overhead of a second unit is totally outweighed by the easy opportunity to have two performance and reliability tiers (fast homelab vs reliable slow storage), ability to upgrade/change the two units independently of each other, and ability for cohabitants to use the NAS without having to understand/deal with the virtualization stuff.
- tracker1 7y ago
  I have an older Synology NAS, it's really slow, and TBH need more compute for a few things at home... but don't want to spend on yet another box for this. The main role of the new server is NAS stuff, but if I can do a few other things in the background, so much the better. I'll be running proxmox and VMs off of an nvme drive in a 4x pcie adapter, and the NAS drives using the hotswap bays in the front.. it's a used R710 (iirc), and has 6 3.5" bays (came with 2TB drives, but replacing the raid controller with one that will support larger drives and upgrading to 8TB as time/cost permits). I could totally drop the home server compute requirement before the NAS requirement as my desktop could handle those chores.
  - unethical_ban 7y ago
    TBH FreeNAS is a great NAS/SAN solution. I'm running a 4x4 RAIDZ2 (8TB effective) with a 10 year old quad core AMD and 16GB DDR2 RAM. If you are okay with the limited VM hosting capabilities of Freenas (headless VMs and containers on the same network as the file server) then you can use it for that as well.

apple4ever 7y ago

Proxmox is pretty nice, except it has two glaring issues: Setting up networks is a pain in the butt. Setting up storage is even worse. Both really require the command line, which shouldn’t be necessary. I ran Proxmox in production for a year for a small ecom company, but if I had to do it again, I’d go vSphere. It’s worth it.

apple4ever 7y ago
I just read all the release notes and this reduces the biggest issue with networking: > If the package ifupdown2 of the Debian network interface manager is installed, it’s now possible to change the network configuration and reload it in the Proxmox web interface without a reboot.