26 ms·

Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

245 points by aaronius 3y ago

adr1an 3y ago
For the sake of completeness, xcp-ng is an alternative to migrate VMware ESXi VMs too!
deleted 3y ago
[deleted]
itopaloglu83 3y ago
Very swift move by Proxmox. For context, VMware recently increased their prices as much as 1200% for some customers.
- rwmj 3y ago
  Tons of products like this have existed for years. Virt-v2v (which I wrote) since 2007/8, Platespin, Xen XCP, AWS's tooling, Azure Migrate etc.
  - itopaloglu83 3y ago
    Yes, that’s true. But this is not about the product but about business practices of Broadcom. They tend to do sharp price increases when they purchase a product line. ServeTheHome talked about this a while ago. https://youtu.be/peH4ic7g5yc https://youtu.be/peH4ic7g5yc
whalesalad 3y ago
I did this recently and it was honestly a walk in the park. Was quite pleasantly surprised when all my vm's just booted up and resumed work as normal. The only thing I was worried about was the mac addresses used for dedicated dhcp leases, but all of that "just worked" too!
Denote6737 3y ago
Proxmox striking whilst the iron is still hot. Impressive.
lelandbatey 3y ago
Important because VMWare's been acquired by Broadcom (November 22, 2023) and Broadcom's been turning the screws on customers to get more money. Many folks are looking for alternatives. More context: 2024/02/26 Can confirm a current Broadcom VMware customer went from $8M renewal to $100M https://news.ycombinator.com/item?id=39509506 https://news.ycombinator.com/item?id=39509506 2024/02/13 VMware vSphere ESXi free edition is dead https://news.ycombinator.com/item?id=39359534 https://news.ycombinator.com/item?id=39359534 2024/01/18 VMware End of Availability of perpetual licensing and associated products https://news.ycombinator.com/item?id=39048120 https://news.ycombinator.com/item?id=39048120 2024/01/15 Order/license chaos for VMware products after Broadcom takeover https://news.ycombinator.com/item?id=38998615 https://news.ycombinator.com/item?id=38998615 2023/12/12 VMware transition to subscription, end of sale of perpetual license https://news.ycombinator.com/item?id=38615315 https://news.ycombinator.com/item?id=38615315
blaerk 3y ago
I really hope the crazy prize increase of vmware products will end the use of esxi and the rest of the vsphere suite, it is one of the worst applications and apis i have ever had the displeasure of working with!
- candiddevmike 3y ago
  VMware has a track record of pretty great reliability across a vast array of hardware. Yes, the APIs suck, but they're a case study on tech debt: vSphere is basically the Windows equivalent of datacenter APIs. They chose the best technology at the time (2009, which meant SOAP, Powershell, XML, etc) and had too much inertia to rework it.
  - mianos 3y ago
    Not to mention how flakey it is at scale. There is always some vmware guy who replies to me saying how good it is, but if you have thousands of VMs it is a random crapshoot. Something you just don't see with say AWS and Azure at similar scale. It reaks of old age and hack on hack over many years, and that is saying something when compared to AWS.
- zettabomb 3y ago
  I can't concur. VMware was the leader in virtualization technology for a long time, and honestly nothing is quite as simple to start with as ESXi if you've never used a type 1 hypervisor before. I'm not so familiar with the APIs, so perhaps you're correct in that sense.
  - nolok 3y ago
    > nothing is quite as simple to start with as ESXi if you've never used a type 1 hypervisor before Not sure about where ESXi is at lately on that level, but latest proxmox is really, really simple to start with if you've never used an hypervisor. You boot on the usb drive, press yes a few times, open the ip:port they give you and then you can click "create vm", next next next here is the iso to boot from and that's it. Any tech user who has some vague knowledge about virtual machine or even run virtualbox on his computer could do it, and the more advanced fonctions (from proper backups and snapshot to multi node replication and load balancing) are absurdly simple to figure out in the UI. I can't talk about the performance or quality of one against the other, but in pure difficulty to approach proxmox is doing very very good.
zer00eyz 3y ago
I have been running proxmox at home for a few months now. It has been, to say the least, an adventure. And I have nothing but good things to say about Proxmox at this point. Its running not only my home related items (MQTT, Homeassitant), it also plays host to some of the projects I'm working on (postgres, go apps, etc...) rather than runing some sort of local dev. If you need to exit vmware, proxmox seems like a good way to go.
- rcarmo 3y ago
  I’ve been doing that for almost two years now, including ARM nodes (via a fork). It’s been awesome, and even though I am fully aware Proxmox does not match the entire VMware feature set (I used to migrate VMware stuff to Azure), it has been a game changer in multiple ways. Case in point: just this weekend a drive started to die on one of my hosts (I still use HDDs on older machines). I backed up the VMs on it to my NAS (you can do that by just having a Samba storage entry defined across the entire cluster), replaced the disk, restored, done.
- irusensei 3y ago
  I appreciate projects like Proxmox but it must be also said that you can achieve same functionality sans the UI with tools available on most Linux distributions: libvirt, lx{c,d}, podman etc.
  - zer00eyz 3y ago
    True... And Proxmox is just skin on lxc and quemu/kvm. I will say that as I have just started playing with the lxc api, having the Proxmox UI as a quick and easy visual cross check has been lovely. Podman is an amazing alternative to docker, cant say enough good things about it.
    - tlamponi 3y ago
      > And Proxmox is just skin on lxc and quemu/kvm. Not really, we have a full-blown REST API that provides storage plugins for a dozen of technologies, disk management, system metrics reporting, management of LXC and QEMU (as in full-blown LXD/Incus and libvirt replacement), which alone probably is taking up a third of our code base, to provide replication, live-migration, local-storage (live-)migration, backup management, HA, good integration into our access control management including multifactor authentication, integration in to LDAP/AD or SSO like OpenID Connect, software defined storage and network integrations, our own kernel, qemu and lxc builds, and hundreds of other features. Don't even get me started on the devs required on each project to continue integration and upstream development and provide enterprise support that actually can fix problems. In other words, wrapping QEMU or LXC to provide ones custom VM/CTs might be doable easily, but that isn't even a percent of what Proxmox VE offers you. If a thin UI around LXC/QEMU is all one would need to be competitive with VMWare, then every web dev would be stupid to not create one as a weekend project, but reality is that there's much more required to actually provide the whole ecosystem a modern hyper-visor stack requires to even be considered for any production use case.
shrubble 3y ago
I am researching whether to buy puts on AVGO (Broadcom, owner of VMware) since I believe their Vmware revenue will crater in 12 months or so. They took on 32 billion in debt to buy VMW also, which will tank their stock price, I think.
- candiddevmike 3y ago
  I'd exercise caution, in my experience, it'll take years for companies to transition from VMware to somewhere else. In the interim, their revenue will most likely pop as they're squeezing the shit out of these unlucky souls.
  - mvdwoord 3y ago
    I concur, being close to the fire it will take years for large organizations to move off their VMware stacks. Inertia of large organizations is a thing, but mostly, there are so many custom integrations made with other systems, lots of them tied up in the vSphere stack. SDN is one thing but the amount of effort put in vROPS / vRA / vRO etc is not easily replaced. Workflows integrating with backups, CMDB, IAM, Security and what not have no catch all migration path with some import wizards. Meanwhile, Broadcom will happily litigate where necessary and invoice their way to a higher stock price. $0.02
- bityard 3y ago
  All of AVGO/Broadcom's moves with VMware have been to keep revenue somewhat steady by focusing on their biggest customers locked into their ecosystem, while drastically cutting back everything else to lower expenses. This should produce excellent short-term financial results which the market will very likely reward with a higher stock price over the next year or two. The board and C-suites know what they are doing. Of course, destroying the trust they had with their customers means the long-term prospects of the VMware are not so good.
  - gonzo 3y ago
    so they sell the husk of VMware back to Dell when they're done.
rwmj 3y ago
Does this do the hard bit, ie installing virtio drivers during conversion?
- justinclift 3y ago
  Oh, that would be a smart move. :) If it doesn't, any idea if it's something they could automate easily?
  - rwmj 3y ago
    In recent versions: virt-customize -a disk.img --inject-virtio-win <METHOD> https://libguestfs.org/virt-customize.1.html https://libguestfs.org/virt-customize.1.html However they'll also be missing out on all the other stuff that virt-v2v does.
- bityard 3y ago
  All of the most popular Linux distros tend have the virtio drivers installed by default.
  - rwmj 3y ago
    Not in the initramfs which is rather important if you want them to boot without having to use slow emulated IDE. Then there's Windows guests.
    - prmoustache 3y ago
      It have been years since I used windows VMs, can't you install .inf driver's file beforehand?
      - rwmj 3y ago
        Windows needs at least the boot block device driver to be installed before conversion (else it cannot find the boot disk), and there are many other changes you need to make. Virt-v2v does all this stuff during conversion, and it's hard to get right.
tiberious726 3y ago
Anyone try to replace vsphere with the high-availabilty add-on to RHEL?
- justinclift 3y ago
  If you're open to alternatives, Proxmox does HA. It also has some decent clustering capabilities enabling online VM migration between hosts (equivalent to VMotion), which can go a long way towards solving related use cases. :)
matthew-wegner 3y ago
I'm in game development, and I run ESXi for two reasons: * Unmodified macOS guest VMs can run under ESXi (if you're reading this on macOS, you have an Apple-made VMXNet3 network adapter driver on your system--see /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleVmxnet3Ethernet.kext ) * Accelerated 3D has reasonable guest support, even as pure software. You wouldn't want to work in one of those guest VMs, but for any sort of build agent it should be fine, including opening i.e. Unity editor itself in-VM to diagnose something Does anyone know where either of these things stand with Proxmox today? I imagine macOS VM under Proxmox is basically a hackintosh with i.e. OpenCore as bootloader?
- deleted 3y ago
  [deleted]
- mysteria 3y ago
  The SPICE backend has decent OpenGL 3D support with software rendering, it's slow but it works for simple graphics. It's intended for 2D so the desktop's pretty fast IMO. That only works for Linux and Windows guests though, not Apple ones. MacOS VMs do work in Proxmox with a Hackintosh setup but you pretty much have to passthrough a GPU to the VM if you're using the GUI. Otherwise you're stuck with the Apple VNC remote desktop and that's unbearably slow.
  - zozbot234 3y ago
    For paravirtualized hardware rendering you can use virtio-gpu. In addition to Linux, a Windows guest driver is available but it's highly experimental still and not very easy to get working.
    - zbrozek 3y ago
      What's the best solution for remote viewing with virtio-gpu? I vaguely recall that it had some incompatibility with spice. I have a bunch of qemu/kvm virtual machines for various tasks. For two of them I decided that graphics performance needed to be at least OK and ended up buying and passing through old Quadro cards. It'd be lovely to not have to do that.
deleted 3y ago
[deleted]
bluedino 3y ago
It's the ecosystem. Sure, your organization is spending another million dollars on VMware this year, but what are the options? * Your outsourced VMware-certified experts don't actually know that much about virtualization (somehow) * Your backup software provider is just now researching adding Proxmox support (https://www.theregister.com/2024/01/22/veeam_proxmox_oracle_support/ https://www.theregister.com/2024/01/22/veeam_proxmox_oracle_...) * A few years ago you 'reduced storage cost and complexity' by moving to VMware vSAN, now you have a SAN purchase and data migration on your task list * The hybrid cloud solution that was implemented isn't compatible with Proxmox * The ServiceNow integration for VMware works great and is saving you tons of time and money. You want to give that up? * Can you live without logging, reporting, and dashboards until your team gets some free time?
- zozbot234 3y ago
  With a million dollar per year to play with, you should ultimately be able to replace all of these. Especially since it's not like Proxmox is lacking its own third-party support options (but it being built on FLOSS tech still leaves you with a lot more flexibility).
- mavhc 3y ago
  Your outsourced experts are actually just people with google. Proxmox on zfs means zfs snapshot send/receive, simple. I made my own immutable zfs backup system for £5
- oneplane 3y ago
  All of those points would also assume: * You are big enough to need that and actually implement it * You have the budget to do so * You actually have the need to do that in-house If you are at that scale but you don't have the internal knowledge, you were going to get bitten anyway. If you are not at that scale, you were already bitten and you shouldn't have been doing it anyway.
- technion 3y ago
  I'll agree in theory but to play devil's advocate: * I did the VCP4 and 5 courses. It's entirely a sales certification. I mean it's a technical certification, but I've never run into anyone who certified for the purpose of running an organisation's tech. Rather, you certify for the purpose of your company being able to sell the product. Note also much of VMware's training focus lately has been on things outside their main virtualisation, like Horizon or their MDM product. * Accurate. But I don't think it'll be far off. * Proxmox does Ceph out of the box. I'll also add that it's very easy to manage, unlike vSAN. I'll further add that none of the VMware training and certifications I've ever done covered vSAN, all the courses assume someone bought a SAN. * All the "hybrid cloud" pushed at least by Microsoft completely assumes you're in Hyper-V and is irrelevant to VMware * I've consulted to an awful lot of VMware organisations and I've never seen servicenow integration in place. I'm sure it's relevant for some peopel.
Helmut10001 3y ago
I use Proxmox since 5 years at home. Mostly docker nested in unprivileged LXCs on ZFS for performance reasons. I love the reliability. Proxmox has never lost me. They churn out constant progress without making too much noise. No buzzwords, no advertising, just a good reliable product that keeps getting better.
- dusanh 3y ago
  Unprivileged LXCs? Interesting, I thought containers would require privileged LXC. At least, that it my takeaway from trying to run Podman in a nesting enabled, but unprivileged LXC under non-root user. I kept running into > newuidmap: write to uid_map failed: Operation not permitted I tried googling it, tried some of the solutions, but reached the conclusion that it's happening because the LXC is not privileged.
  - LilBytes 3y ago
    Have to map GUID and UIDs from the Proxmox host to the LxC to allow bind mappings work as an example. Proxmox doco for unprivileged LxC is here: https://pve.proxmox.com/wiki/Unprivileged_LXC_containers https://pve.proxmox.com/wiki/Unprivileged_LXC_containers
  - Helmut10001 3y ago
    I once wrote a post about Docker in unprivileged LXC on ZFS [1]. The post is a little bit outdated, as it is much simpler today with ZFS 2.2.0, which is natively supported. There's also a more recent post that shows how to run rootless docker [2], with updated uid-mappings. Both may be helpful, have a look. The advantage of using LXC for me is resource consumption and separation of concerns. I have about 35 Docker containers spread over 10 LXCs. The average CPU use is 1-3% and I only need about 10GB of memory (even with running bigger containers like Nextcloud, Gitlab, mailcow-dockerized etc.). With docker-compose.yml's, automatic updates are easy and robust. [1]: https://du.nkel.dev/blog/2021-03-25_proxmox_docker/ https://du.nkel.dev/blog/2021-03-25_proxmox_docker/ [2]: https://du.nkel.dev/blog/2023-12-12_mastodon-docker-rootless/ https://du.nkel.dev/blog/2023-12-12_mastodon-docker-rootless...
F00Fbug 3y ago
I spent 15 years managing a VMware-centric data center. I ran the free version at home for at least 5 years. When I ran out of vCPUs on my free license I switched to Proxmox and the migration was almost painless. This new tool should help even more. For most vanilla hosting, you could get away with Proxmox and be just fine. I've been running it for at least 5 years in my basement and haven't had a single hiccup. I bet a lot of VMware customers will be jumping ship when their licenses expire.
moondev 3y ago
If proxmox supported OVA and OVF it would be huge. It seems technically possible as there is a new experimental kvm backend for virtual box which supports OVA. At the end of the day OVA is just machine metadata packaged as XML with all required VM artifacts, there is also some cool things like supporting launch variables. Leveraging the format would bring a bunch of momentum considering all the existing OVAs in the wild
- the_swd 3y ago
  Proxmox documentation does mention OVF support https://pve.proxmox.com/wiki/Migrate_to_Proxmox_VE#Import_OVF https://pve.proxmox.com/wiki/Migrate_to_Proxmox_VE#Import_OV... Seems a bit barebones, as in no support for a nice OVF properties launch UI, but one should be able to extract an OVA to an OVF and VMDK an manually edit the OVF with appropriate properties. I actually had plans this week to try exactly that...
  - Underphil 3y ago
    I've set up a ton of virtual appliances that way. It's just a regular ZIP file with the config and vmdk(s).
  - moondev 3y ago
    Interesting thanks for sharing. Surfacing this in the UI would be great if it works well for sure. Another handy feature is the ContentLibrary for organizing and launching OVA/OVF, as well as launching OVA directly from a URL without needing to download it next to the cli. This makes me think there could be an opportunity in "PhotoPea (kvm gui) for vCenter" - in the same manner photopea is a clean room implementation of the photoshop UI/UX
    - tlamponi 3y ago
      > Interesting thanks for sharing. Surfacing this in the UI would be great if it works well for sure. That's on the roadmap, from the original forum post linked here: > Q: Will other import sources be supported in the future? > A: We plan to integrate our OVF/OVA import tools into this new stack in the future. > Currently, integrating additional import sources is not on our roadmap, but will be re-evaluated periodically.
rafaelturk 3y ago
Proxmox is great! I just wish that they had a better initial plan, plans start at €1020.
- subract 3y ago
  I see plans with access to the Enterprise repos starting at €110/yr, and plans with 3 support tickets starting at €340. €1020 is the starting price for a plan with a 2hr SLA. https://shop.proxmox.com/index.php?rp=/store/proxmox-ve-community https://shop.proxmox.com/index.php?rp=/store/proxmox-ve-comm...
- justinclift 3y ago
  For non production purposes, you're probably fine to use the the "No subscription" repositories. They're an official thing, although their website seems to go out of it's way to not mention it. You don't actually need a subscription to run Proxmox, it's FOSS software after all.
luzer7 3y ago
Does anyone have a good _basic_ guide on LVM/LVM Thin? I'm having a hard time wrapping my head around LVM and moving the vmdk to it. Mainly a Window admin with some Linux experience. I understand that LVM holds data in it but when I make a Windows VM in proxmox it stores the data in a LVM partition(?) as opposed to ESXi or Hyper-V making a VHD or VMDK. Kinda confusing .
- abbbi 3y ago
  proxmox is using LVM for direct attached raw volumes. LVM is just a logical volume manager for linux, which gives you more features than using old fashioned disk partitioning. I guess they chose this path for windows virtual machine migration because windows running on vmware before, does usually not have the required virtio drivers installed to support the qemu hypervisors virtio solution for disk bus virtualization out of the box. It would mean the hypervisor has to simulate IDE or SCSI bus which comes with great overhead perfomance wise (in the case of migration) So an direct attached lvm volume is the best solution performance wise. In the vmware world this would be an direct attached raw device either from local disk or SAN. For fresh install on proxmox its better to chose qcow as disk image format with virtio-scsi bus (comparable to vhdx, vmdk, qemus disk format) and add virtio drivers during windows setup.
- m463 3y ago
  i ran into the same sort of documentation desert. off the top of my head: - keep in mind there is LVM and LVM2, and proxmox now uses lvm2 - I don't understand the thinpool allocation. You don't have to use lvm-thin if you don't want to deal with oversubscribed volumes, or don't care about snapshots or cloning storage. - get to know "pvesm". A lot of things you can do in the gui - when making linux VMs, I found it easier to use separate devices for the efi partition and the linux partition, such as: efidisk0: local-lvm:vm-205-disk-0,size=4M virtio0: local-lvm:vm-205-disk-1,iothread=1,size=1G virtio1: local-lvm:vm-205-disk-2,cache=writeback,discard=on,iothread=1,size=32G (virtio0 = efi, virtio1 = /) and I can mount/expand/resize /dev/mapper/big2-vm--205--disk--2 without having to deal with disk partitions
fulafel 3y ago
No mentions yet in the comments re the widespread VMware breakin/ransomware epidemic of recent times as the reason to move. I hope there are many people motivated by that and not just the price increases.

245 points by aaronius 3y ago

adr1an 3y ago

For the sake of completeness, xcp-ng is an alternative to migrate VMware ESXi VMs too!

deleted 3y ago

[deleted]

itopaloglu83 3y ago

Very swift move by Proxmox. For context, VMware recently increased their prices as much as 1200% for some customers.

rwmj 3y ago
Tons of products like this have existed for years. Virt-v2v (which I wrote) since 2007/8, Platespin, Xen XCP, AWS's tooling, Azure Migrate etc.
- itopaloglu83 3y ago
  Yes, that’s true. But this is not about the product but about business practices of Broadcom. They tend to do sharp price increases when they purchase a product line. ServeTheHome talked about this a while ago. https://youtu.be/peH4ic7g5yc https://youtu.be/peH4ic7g5yc

whalesalad 3y ago

I did this recently and it was honestly a walk in the park. Was quite pleasantly surprised when all my vm's just booted up and resumed work as normal. The only thing I was worried about was the mac addresses used for dedicated dhcp leases, but all of that "just worked" too!

Denote6737 3y ago

Proxmox striking whilst the iron is still hot. Impressive.

lelandbatey 3y ago

Important because VMWare's been acquired by Broadcom (November 22, 2023) and Broadcom's been turning the screws on customers to get more money. Many folks are looking for alternatives. More context: 2024/02/26 Can confirm a current Broadcom VMware customer went from $8M renewal to $100M https://news.ycombinator.com/item?id=39509506 https://news.ycombinator.com/item?id=39509506 2024/02/13 VMware vSphere ESXi free edition is dead https://news.ycombinator.com/item?id=39359534 https://news.ycombinator.com/item?id=39359534 2024/01/18 VMware End of Availability of perpetual licensing and associated products https://news.ycombinator.com/item?id=39048120 https://news.ycombinator.com/item?id=39048120 2024/01/15 Order/license chaos for VMware products after Broadcom takeover https://news.ycombinator.com/item?id=38998615 https://news.ycombinator.com/item?id=38998615 2023/12/12 VMware transition to subscription, end of sale of perpetual license https://news.ycombinator.com/item?id=38615315 https://news.ycombinator.com/item?id=38615315

blaerk 3y ago

I really hope the crazy prize increase of vmware products will end the use of esxi and the rest of the vsphere suite, it is one of the worst applications and apis i have ever had the displeasure of working with!

candiddevmike 3y ago
VMware has a track record of pretty great reliability across a vast array of hardware. Yes, the APIs suck, but they're a case study on tech debt: vSphere is basically the Windows equivalent of datacenter APIs. They chose the best technology at the time (2009, which meant SOAP, Powershell, XML, etc) and had too much inertia to rework it.
- mianos 3y ago
  Not to mention how flakey it is at scale. There is always some vmware guy who replies to me saying how good it is, but if you have thousands of VMs it is a random crapshoot. Something you just don't see with say AWS and Azure at similar scale. It reaks of old age and hack on hack over many years, and that is saying something when compared to AWS.
zettabomb 3y ago
I can't concur. VMware was the leader in virtualization technology for a long time, and honestly nothing is quite as simple to start with as ESXi if you've never used a type 1 hypervisor before. I'm not so familiar with the APIs, so perhaps you're correct in that sense.
- nolok 3y ago
  > nothing is quite as simple to start with as ESXi if you've never used a type 1 hypervisor before Not sure about where ESXi is at lately on that level, but latest proxmox is really, really simple to start with if you've never used an hypervisor. You boot on the usb drive, press yes a few times, open the ip:port they give you and then you can click "create vm", next next next here is the iso to boot from and that's it. Any tech user who has some vague knowledge about virtual machine or even run virtualbox on his computer could do it, and the more advanced fonctions (from proper backups and snapshot to multi node replication and load balancing) are absurdly simple to figure out in the UI. I can't talk about the performance or quality of one against the other, but in pure difficulty to approach proxmox is doing very very good.

zer00eyz 3y ago

I have been running proxmox at home for a few months now. It has been, to say the least, an adventure. And I have nothing but good things to say about Proxmox at this point. Its running not only my home related items (MQTT, Homeassitant), it also plays host to some of the projects I'm working on (postgres, go apps, etc...) rather than runing some sort of local dev. If you need to exit vmware, proxmox seems like a good way to go.

rcarmo 3y ago
I’ve been doing that for almost two years now, including ARM nodes (via a fork). It’s been awesome, and even though I am fully aware Proxmox does not match the entire VMware feature set (I used to migrate VMware stuff to Azure), it has been a game changer in multiple ways. Case in point: just this weekend a drive started to die on one of my hosts (I still use HDDs on older machines). I backed up the VMs on it to my NAS (you can do that by just having a Samba storage entry defined across the entire cluster), replaced the disk, restored, done.
irusensei 3y ago
I appreciate projects like Proxmox but it must be also said that you can achieve same functionality sans the UI with tools available on most Linux distributions: libvirt, lx{c,d}, podman etc.
- zer00eyz 3y ago
  True... And Proxmox is just skin on lxc and quemu/kvm. I will say that as I have just started playing with the lxc api, having the Proxmox UI as a quick and easy visual cross check has been lovely. Podman is an amazing alternative to docker, cant say enough good things about it.
  - tlamponi 3y ago
    > And Proxmox is just skin on lxc and quemu/kvm. Not really, we have a full-blown REST API that provides storage plugins for a dozen of technologies, disk management, system metrics reporting, management of LXC and QEMU (as in full-blown LXD/Incus and libvirt replacement), which alone probably is taking up a third of our code base, to provide replication, live-migration, local-storage (live-)migration, backup management, HA, good integration into our access control management including multifactor authentication, integration in to LDAP/AD or SSO like OpenID Connect, software defined storage and network integrations, our own kernel, qemu and lxc builds, and hundreds of other features. Don't even get me started on the devs required on each project to continue integration and upstream development and provide enterprise support that actually can fix problems. In other words, wrapping QEMU or LXC to provide ones custom VM/CTs might be doable easily, but that isn't even a percent of what Proxmox VE offers you. If a thin UI around LXC/QEMU is all one would need to be competitive with VMWare, then every web dev would be stupid to not create one as a weekend project, but reality is that there's much more required to actually provide the whole ecosystem a modern hyper-visor stack requires to even be considered for any production use case.

shrubble 3y ago

I am researching whether to buy puts on AVGO (Broadcom, owner of VMware) since I believe their Vmware revenue will crater in 12 months or so. They took on 32 billion in debt to buy VMW also, which will tank their stock price, I think.

candiddevmike 3y ago
I'd exercise caution, in my experience, it'll take years for companies to transition from VMware to somewhere else. In the interim, their revenue will most likely pop as they're squeezing the shit out of these unlucky souls.
- mvdwoord 3y ago
  I concur, being close to the fire it will take years for large organizations to move off their VMware stacks. Inertia of large organizations is a thing, but mostly, there are so many custom integrations made with other systems, lots of them tied up in the vSphere stack. SDN is one thing but the amount of effort put in vROPS / vRA / vRO etc is not easily replaced. Workflows integrating with backups, CMDB, IAM, Security and what not have no catch all migration path with some import wizards. Meanwhile, Broadcom will happily litigate where necessary and invoice their way to a higher stock price. $0.02
bityard 3y ago
All of AVGO/Broadcom's moves with VMware have been to keep revenue somewhat steady by focusing on their biggest customers locked into their ecosystem, while drastically cutting back everything else to lower expenses. This should produce excellent short-term financial results which the market will very likely reward with a higher stock price over the next year or two. The board and C-suites know what they are doing. Of course, destroying the trust they had with their customers means the long-term prospects of the VMware are not so good.
- gonzo 3y ago
  so they sell the husk of VMware back to Dell when they're done.

rwmj 3y ago

Does this do the hard bit, ie installing virtio drivers during conversion?

justinclift 3y ago
Oh, that would be a smart move. :) If it doesn't, any idea if it's something they could automate easily?
- rwmj 3y ago
  In recent versions: virt-customize -a disk.img --inject-virtio-win <METHOD> https://libguestfs.org/virt-customize.1.html https://libguestfs.org/virt-customize.1.html However they'll also be missing out on all the other stuff that virt-v2v does.
bityard 3y ago
All of the most popular Linux distros tend have the virtio drivers installed by default.
- rwmj 3y ago
  Not in the initramfs which is rather important if you want them to boot without having to use slow emulated IDE. Then there's Windows guests.
  - prmoustache 3y ago
    It have been years since I used windows VMs, can't you install .inf driver's file beforehand?
    - rwmj 3y ago
      Windows needs at least the boot block device driver to be installed before conversion (else it cannot find the boot disk), and there are many other changes you need to make. Virt-v2v does all this stuff during conversion, and it's hard to get right.

tiberious726 3y ago

Anyone try to replace vsphere with the high-availabilty add-on to RHEL?

justinclift 3y ago
If you're open to alternatives, Proxmox does HA. It also has some decent clustering capabilities enabling online VM migration between hosts (equivalent to VMotion), which can go a long way towards solving related use cases. :)

matthew-wegner 3y ago

I'm in game development, and I run ESXi for two reasons: * Unmodified macOS guest VMs can run under ESXi (if you're reading this on macOS, you have an Apple-made VMXNet3 network adapter driver on your system--see /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleVmxnet3Ethernet.kext ) * Accelerated 3D has reasonable guest support, even as pure software. You wouldn't want to work in one of those guest VMs, but for any sort of build agent it should be fine, including opening i.e. Unity editor itself in-VM to diagnose something Does anyone know where either of these things stand with Proxmox today? I imagine macOS VM under Proxmox is basically a hackintosh with i.e. OpenCore as bootloader?

deleted 3y ago
[deleted]
mysteria 3y ago
The SPICE backend has decent OpenGL 3D support with software rendering, it's slow but it works for simple graphics. It's intended for 2D so the desktop's pretty fast IMO. That only works for Linux and Windows guests though, not Apple ones. MacOS VMs do work in Proxmox with a Hackintosh setup but you pretty much have to passthrough a GPU to the VM if you're using the GUI. Otherwise you're stuck with the Apple VNC remote desktop and that's unbearably slow.
- zozbot234 3y ago
  For paravirtualized hardware rendering you can use virtio-gpu. In addition to Linux, a Windows guest driver is available but it's highly experimental still and not very easy to get working.
  - zbrozek 3y ago
    What's the best solution for remote viewing with virtio-gpu? I vaguely recall that it had some incompatibility with spice. I have a bunch of qemu/kvm virtual machines for various tasks. For two of them I decided that graphics performance needed to be at least OK and ended up buying and passing through old Quadro cards. It'd be lovely to not have to do that.

deleted 3y ago

[deleted]

bluedino 3y ago

It's the ecosystem. Sure, your organization is spending another million dollars on VMware this year, but what are the options? * Your outsourced VMware-certified experts don't actually know that much about virtualization (somehow) * Your backup software provider is just now researching adding Proxmox support (https://www.theregister.com/2024/01/22/veeam_proxmox_oracle_support/ https://www.theregister.com/2024/01/22/veeam_proxmox_oracle_...) * A few years ago you 'reduced storage cost and complexity' by moving to VMware vSAN, now you have a SAN purchase and data migration on your task list * The hybrid cloud solution that was implemented isn't compatible with Proxmox * The ServiceNow integration for VMware works great and is saving you tons of time and money. You want to give that up? * Can you live without logging, reporting, and dashboards until your team gets some free time?

zozbot234 3y ago
With a million dollar per year to play with, you should ultimately be able to replace all of these. Especially since it's not like Proxmox is lacking its own third-party support options (but it being built on FLOSS tech still leaves you with a lot more flexibility).
mavhc 3y ago
Your outsourced experts are actually just people with google. Proxmox on zfs means zfs snapshot send/receive, simple. I made my own immutable zfs backup system for £5
oneplane 3y ago
All of those points would also assume: * You are big enough to need that and actually implement it * You have the budget to do so * You actually have the need to do that in-house If you are at that scale but you don't have the internal knowledge, you were going to get bitten anyway. If you are not at that scale, you were already bitten and you shouldn't have been doing it anyway.
technion 3y ago
I'll agree in theory but to play devil's advocate: * I did the VCP4 and 5 courses. It's entirely a sales certification. I mean it's a technical certification, but I've never run into anyone who certified for the purpose of running an organisation's tech. Rather, you certify for the purpose of your company being able to sell the product. Note also much of VMware's training focus lately has been on things outside their main virtualisation, like Horizon or their MDM product. * Accurate. But I don't think it'll be far off. * Proxmox does Ceph out of the box. I'll also add that it's very easy to manage, unlike vSAN. I'll further add that none of the VMware training and certifications I've ever done covered vSAN, all the courses assume someone bought a SAN. * All the "hybrid cloud" pushed at least by Microsoft completely assumes you're in Hyper-V and is irrelevant to VMware * I've consulted to an awful lot of VMware organisations and I've never seen servicenow integration in place. I'm sure it's relevant for some peopel.

Helmut10001 3y ago

I use Proxmox since 5 years at home. Mostly docker nested in unprivileged LXCs on ZFS for performance reasons. I love the reliability. Proxmox has never lost me. They churn out constant progress without making too much noise. No buzzwords, no advertising, just a good reliable product that keeps getting better.

dusanh 3y ago
Unprivileged LXCs? Interesting, I thought containers would require privileged LXC. At least, that it my takeaway from trying to run Podman in a nesting enabled, but unprivileged LXC under non-root user. I kept running into > newuidmap: write to uid_map failed: Operation not permitted I tried googling it, tried some of the solutions, but reached the conclusion that it's happening because the LXC is not privileged.
- LilBytes 3y ago
  Have to map GUID and UIDs from the Proxmox host to the LxC to allow bind mappings work as an example. Proxmox doco for unprivileged LxC is here: https://pve.proxmox.com/wiki/Unprivileged_LXC_containers https://pve.proxmox.com/wiki/Unprivileged_LXC_containers
- Helmut10001 3y ago
  I once wrote a post about Docker in unprivileged LXC on ZFS [1]. The post is a little bit outdated, as it is much simpler today with ZFS 2.2.0, which is natively supported. There's also a more recent post that shows how to run rootless docker [2], with updated uid-mappings. Both may be helpful, have a look. The advantage of using LXC for me is resource consumption and separation of concerns. I have about 35 Docker containers spread over 10 LXCs. The average CPU use is 1-3% and I only need about 10GB of memory (even with running bigger containers like Nextcloud, Gitlab, mailcow-dockerized etc.). With docker-compose.yml's, automatic updates are easy and robust. [1]: https://du.nkel.dev/blog/2021-03-25_proxmox_docker/ https://du.nkel.dev/blog/2021-03-25_proxmox_docker/ [2]: https://du.nkel.dev/blog/2023-12-12_mastodon-docker-rootless/ https://du.nkel.dev/blog/2023-12-12_mastodon-docker-rootless...

F00Fbug 3y ago

I spent 15 years managing a VMware-centric data center. I ran the free version at home for at least 5 years. When I ran out of vCPUs on my free license I switched to Proxmox and the migration was almost painless. This new tool should help even more. For most vanilla hosting, you could get away with Proxmox and be just fine. I've been running it for at least 5 years in my basement and haven't had a single hiccup. I bet a lot of VMware customers will be jumping ship when their licenses expire.

moondev 3y ago

If proxmox supported OVA and OVF it would be huge. It seems technically possible as there is a new experimental kvm backend for virtual box which supports OVA. At the end of the day OVA is just machine metadata packaged as XML with all required VM artifacts, there is also some cool things like supporting launch variables. Leveraging the format would bring a bunch of momentum considering all the existing OVAs in the wild

the_swd 3y ago
Proxmox documentation does mention OVF support https://pve.proxmox.com/wiki/Migrate_to_Proxmox_VE#Import_OVF https://pve.proxmox.com/wiki/Migrate_to_Proxmox_VE#Import_OV... Seems a bit barebones, as in no support for a nice OVF properties launch UI, but one should be able to extract an OVA to an OVF and VMDK an manually edit the OVF with appropriate properties. I actually had plans this week to try exactly that...
- Underphil 3y ago
  I've set up a ton of virtual appliances that way. It's just a regular ZIP file with the config and vmdk(s).
- moondev 3y ago
  Interesting thanks for sharing. Surfacing this in the UI would be great if it works well for sure. Another handy feature is the ContentLibrary for organizing and launching OVA/OVF, as well as launching OVA directly from a URL without needing to download it next to the cli. This makes me think there could be an opportunity in "PhotoPea (kvm gui) for vCenter" - in the same manner photopea is a clean room implementation of the photoshop UI/UX
  - tlamponi 3y ago
    > Interesting thanks for sharing. Surfacing this in the UI would be great if it works well for sure. That's on the roadmap, from the original forum post linked here: > Q: Will other import sources be supported in the future? > A: We plan to integrate our OVF/OVA import tools into this new stack in the future. > Currently, integrating additional import sources is not on our roadmap, but will be re-evaluated periodically.

rafaelturk 3y ago

Proxmox is great! I just wish that they had a better initial plan, plans start at €1020.

subract 3y ago
I see plans with access to the Enterprise repos starting at €110/yr, and plans with 3 support tickets starting at €340. €1020 is the starting price for a plan with a 2hr SLA. https://shop.proxmox.com/index.php?rp=/store/proxmox-ve-community https://shop.proxmox.com/index.php?rp=/store/proxmox-ve-comm...
justinclift 3y ago
For non production purposes, you're probably fine to use the the "No subscription" repositories. They're an official thing, although their website seems to go out of it's way to not mention it. You don't actually need a subscription to run Proxmox, it's FOSS software after all.

luzer7 3y ago

Does anyone have a good _basic_ guide on LVM/LVM Thin? I'm having a hard time wrapping my head around LVM and moving the vmdk to it. Mainly a Window admin with some Linux experience. I understand that LVM holds data in it but when I make a Windows VM in proxmox it stores the data in a LVM partition(?) as opposed to ESXi or Hyper-V making a VHD or VMDK. Kinda confusing .

abbbi 3y ago
proxmox is using LVM for direct attached raw volumes. LVM is just a logical volume manager for linux, which gives you more features than using old fashioned disk partitioning. I guess they chose this path for windows virtual machine migration because windows running on vmware before, does usually not have the required virtio drivers installed to support the qemu hypervisors virtio solution for disk bus virtualization out of the box. It would mean the hypervisor has to simulate IDE or SCSI bus which comes with great overhead perfomance wise (in the case of migration) So an direct attached lvm volume is the best solution performance wise. In the vmware world this would be an direct attached raw device either from local disk or SAN. For fresh install on proxmox its better to chose qcow as disk image format with virtio-scsi bus (comparable to vhdx, vmdk, qemus disk format) and add virtio drivers during windows setup.
m463 3y ago
i ran into the same sort of documentation desert. off the top of my head: - keep in mind there is LVM and LVM2, and proxmox now uses lvm2 - I don't understand the thinpool allocation. You don't have to use lvm-thin if you don't want to deal with oversubscribed volumes, or don't care about snapshots or cloning storage. - get to know "pvesm". A lot of things you can do in the gui - when making linux VMs, I found it easier to use separate devices for the efi partition and the linux partition, such as: efidisk0: local-lvm:vm-205-disk-0,size=4M virtio0: local-lvm:vm-205-disk-1,iothread=1,size=1G virtio1: local-lvm:vm-205-disk-2,cache=writeback,discard=on,iothread=1,size=32G (virtio0 = efi, virtio1 = /) and I can mount/expand/resize /dev/mapper/big2-vm--205--disk--2 without having to deal with disk partitions

fulafel 3y ago

No mentions yet in the comments re the widespread VMware breakin/ransomware epidemic of recent times as the reason to move. I hope there are many people motivated by that and not just the price increases.