6 ms·

My heart goes out to all of the institutions that are about to get hammered by this change, but I would by lying if I said I wasn't excited about a potential in

My heart goes out to all of the institutions that are about to get hammered by this change, but I would by lying if I said I wasn't excited about a potential increase in investment in open source cloud virtualization infrastructure. Proxmox seems poised to benefit from this; is Openstack in a situation where it could be reasonably rolled out as a substitute for vSphere without requiring significant staff increases?

thedougd 3y ago
OpenStack provides a different axis of offering than vCenter. It's more aligned for multi-tenant, API driven use cases, ala cloud. Of course, you could open access to only the virtualization team, but they'd likely find the console experience frustrating. VMware even has an OpenStack product that is built on vSphere virtualization and NSX-T software defined networking. The challenge is that companies architect their services and applications around what they have, and they may be very operationally dependent on specific features of vCenter and how they've configured those features.
- NBJack 3y ago
  Yeah, OpenStack is very much not a great competitor. If you can afford an expert to set it up for you, great! Otherwise, prepare to invest in other ways. I'm not going to say VMware has a super intuitive system, but I've gotten surprisingly far at times without a lot of deep knowledge. Well, until the *nix-like kernel in the hypervisor misbehaves....
  - znpy 3y ago
    If you have any significant workload on vmware you likely already have an expert on payroll.
  - Klasiaster 3y ago
    The setup is not too complicated with something like https://opendev.org/openstack/devstack https://opendev.org/openstack/devstack
    - prometheanfire 3y ago
      devstack is for dev work, expect nothing to persist. If you want a deployment choose something like openstack-ansible.
  - throw0101c 3y ago
    > Yeah, OpenStack is very much not a great competitor. If you can afford an expert to set it up for you, great! If you know (say) Ubuntu and Ansible, then there's pretty good automation with openstack-ansible and kolla-ansible. There are a number of moving parts, but an intermediate-level Linux sysadmin can get things going without too much difficulty. There are vendor-provided solutions (e.g., from RH) as well if you don't want to do a fully DIY, or want commercial support. * https://www.openstack.org/marketplace/distros/ https://www.openstack.org/marketplace/distros/ At a previous job we had a team of 4-5, and 1.5 FTEs split their time between our HPC cluster and our OpenStack cluster (20+ hypervisors, ~10 Ceph servers; >350 VMs/instances).
- mitjam 3y ago
  There is also Rancher Harvester (https://harvesterhci.io/ https://harvesterhci.io/) which I think is a breadth of fresh air compared to VMware and OpenStack and then there is the Windows Server based Azure Stack HCI, too. In my experience, VMware‘s biggest moat is the vast support from software vendors. It basically is the default supported platform for almost every business application. This comes with well known best practices, many experienced admins, out of the box solutions from hardware vendors and highly standardized managed services. Most apps leave VMware for the cloud or Kubernetes but not another on-prem VM platform.
  - hhh 3y ago
    Harvester has been great for me, except when I wanted to start using a real SAN with it. It's KubeVirt under the hood if I recall correctly, which is also a great project to check out.
  - nolist_policy 3y ago
    Or go vendor independent with pacemaker + drbd + libvirt. For that stack you can get support from SUSE, Redhat and Linbit and maybe even Ubuntu. Or just go with Debian if you don't need support.
- throw0101c 3y ago
  > VMware even has an OpenStack product that is built on vSphere virtualization and NSX-T software defined networking. * https://www.vmware.com/ca/topics/glossary/content/openstack-api.html https://www.vmware.com/ca/topics/glossary/content/openstack-... * https://www.vmware.com/ca/products/openstack.html https://www.vmware.com/ca/products/openstack.html
  - thedougd 3y ago
    It's a pretty good product, IME, especially if you're using it with NSX-T.
- KaiserPro 3y ago
  unless something has changed recently, openstack is a horror to setup and administer. the point of vmware is that is fairly trivial to go from a single server with some VMs, to a massive cluster with live zero-downtime migrations. (and have funky storage, get working snapshots and backups.) Sure openstack can do that. but its non trivial to setup.
  - I_Am_Nous 3y ago
    We used Openstack for a while, but overall it was a nightmare. I'm sure it's better than it used to be, but it is not simple to spin up or maintain, and it's not designed to baby VMs -- if one acts up they expect you to delete and recreate it, which works OK for microservices but not for huge Windows terminal servers.
  - freeopinion 3y ago
    I see companies hire a team of experts who plan and architect for six months to a year before they are ready to deploy a new VMWare cluster. I've seen a two or three rack cluster cost a million dollars before the architecture team requisitioned a single piece of hardware or software. I've seen a very small outfit spend six months with all hired consultants to set up a 3-server VMWare cluster. VMWare doesn't seem that trivial.
    - bogantech 3y ago
      > I see companies hire a team of experts who plan and architect for six months to a year before they are ready to deploy a new VMWare cluster. Then those people are woefully incompetent and will be regardless of the software they're using.
    - KaiserPro 3y ago
      I mean yeah, but thats the cost of hiring consultants. VMware (wasnt) that hard. How many VMs do you need? work out the storage requirements, triple it, plumb it in, get the right licences, and go.
natbennett 3y ago
Has OpenStack as an organization solved the problem where there’s not really such a thing as OpenStack? In terms of e.g. what APIs will be available and so on? That’s basically what stopped it back in ‘14 or so. It was too hard to provide software that could run on an OpenStack because it was such a choose-your-own-adventure.
pierrelf 3y ago
I was at the Apache Cloudstack conference just a few weeks ago and it seems most of the integrators are pivoting to moving VMware customers to KVM + CloudStack, and have even upstreamed an auto conversion tool. https://cloudstack.apache.org/ https://cloudstack.apache.org/
- jabl 3y ago
  People at a cloudstack conference think cloudstack is great and want their customers to use it? What's next, a survey of people attending the annual dentistry conference shows that they think dental health is important?
- rwmj 3y ago
  Are they using virt-v2v for conversions from VMware to KVM or something else?
  - Nux 3y ago
    Virt-v2v indeed and thanks for that btw.
csydas 3y ago
While I am excited to see that quite a decent number of past clients I've had are considering proxmox and other virtualization stacks, I am not confident there will be a fast answer for this new change in the virtualization realm. Large and Enterprise companies won't really be bothered by the change except the price, and unless other platforms include all the "Enterprise Check-boxes" they think they want, I'm not sure how fast adoption will be if at all for stacks like Proxmox. I am not defending the business model or the tech offerings of Vmware, Microsoft, etc -- far from it. They're extremely bloated, buggy, and unless you happen to be one of VMware or Microsoft's most expensive customers, the support might as well just be a sin tax. Even for the "biggest" clients, they were not able to get Vmware or Microsoft to budge on some pretty obvious and complete blocker bugs that would drop production environments entirely. Enterprise is weird -- I would be extremely happy if not a bit exhausted supporting a large OSS virtualization infrastructure for clients, but I just do not see it happening without a lot of "carrots" for such companies from the virtualization vendor. Enterprise business is basically about shifting responsibility and liability for your tech-stack to someone else -- yes, it requires strong tech skills to use these tech-stacks, but the more maintenance/liability that can be shifted to the vendor, the more preferable it is for most companies. I don't know what the businesses will choose -- I'm hopeful medium and smaller businesses will shift to more open solutions and from this business, the open solutions will be able to grow and become more attractive choices for enterprise, I'm just not sure that such solutions will avoid the tech industry's obsession with subscription licensing and feature bloat to check boxes.
- depereo 3y ago
  With all the 'investment' in ISO27001:* validation of the existing infrastructure choices and the like, I'd be astonished if any move in a large company was able to happen within five years. From what I've seen none of them have taken this issue seriously, either, and won't until they get squeezed hard enough, at which point they'll have to... spend five years trying to move!
- phpisthebest 3y ago
  We would have to see a 2x minimum YoY cost increase just to justify a poliot program to consider moving to something else Keep in mind it is more than just shifting all of the VM workloads to a new platform something not easy in the first place But you also have to factor admin training, backup and recovery systems, monitoriting systems, automation tooling, etc. We have been using vmware since version 3 or 4, our entire team is almost exclusively training on vmware, with limited knowledge on hyperv and virtually none on anything else Something like ProxMox that has no third party support, no veeam support, limited automation tooling, limited support for commerical monitoring (like PRTG) etc etc is probally a non-starter unless those things support it. It would cost us far far far far more than 3x licensing increases to git rid vmware. Broadcom knows this. Honestly VMWare has not been increasing its prices as aggressievly as other vendors anyway, of my software licensing fees I pay annually vmware is one of the cheaper vendors I have, not even breaking into the top 10, I pay more for Veeam, Windows Server, and many many many other things than i do vmware
eddyg 3y ago
Don't forget about the open-source XCP-ng[1] to which you can add Xen Orchestra (which is like vCenter; a key reason people use ESXi) which has a flat-rate (no socket, host or VM limits) $2600/year Enterprise license. And XCP-ng even supports migrating from VMware[3]. [1] https://xcp-ng.org https://xcp-ng.org [2] https://xen-orchestra.com/#!/xo-features/webinterface https://xen-orchestra.com/#!/xo-features/webinterface [3] https://xcp-ng.org/blog/2022/10/19/migrate-from-vmware-to-xcp-ng/ https://xcp-ng.org/blog/2022/10/19/migrate-from-vmware-to-xc... and https://xcp-ng.org/blog/2022/04/13/how-bedrock-streaming-migrated-from-vmware-to-xcp-ng/ https://xcp-ng.org/blog/2022/04/13/how-bedrock-streaming-mig...
bushbaba 3y ago
Suspect it'll be the biggest gift to the public cloud providers. Firms who use VMWare rarely want to self-manage their virtualization tech.
ThinkBeat 3y ago
proxmox is presently not an alternative to the VMWare stack. Not even close. Converting from the VMWare stack⭢ proxmox would be a giant shitshow. I have been a paying user of WMware personally since the beginning and I have worked on their enterprise stack for several years. With the changes being made I cant see myself wanting to spend any personal money on it I and I would not recommend someone to start fresh with it, but for a lot of enterprises, they will, short term, have little choice than to pay the fees. I have no idea about the state of Microsoft HyperV. The little I have worked with it, and that was quite a few years ago now it seemed severely lacking in the tools department. Oracle owns VirtualBox, and even if they started to build it out (or have they?) going to Oracle to avoid high fees is not going to be a good idea.
- fuzzylightbulb 3y ago
  Microsoft is ending mainstream support of Hyper-V Server 2019 on January 9, 2024 and extended support will end on January 9, 2029. Hyper-V Server 2019 will be the last version of this product and Microsoft is encouraging customers to transition to Azure Stack HCI.
  - EvanAnderson 3y ago
    Hyper-V as a role you can install on Windows Server is not being discontinued. They're discontinuing the no cost SKU but the feature isn't going away.
    - sebazzz 3y ago
      Exactly, but the Windows license you need for a proper Hyper-V server is quite high
- tlamponi 3y ago
  We have many happy customers successfully converting from VMWare to Proxmox VE just fine, what's missimg for your usecase?
- pashadee 3y ago

thedougd 3y ago

OpenStack provides a different axis of offering than vCenter. It's more aligned for multi-tenant, API driven use cases, ala cloud. Of course, you could open access to only the virtualization team, but they'd likely find the console experience frustrating. VMware even has an OpenStack product that is built on vSphere virtualization and NSX-T software defined networking. The challenge is that companies architect their services and applications around what they have, and they may be very operationally dependent on specific features of vCenter and how they've configured those features.

NBJack 3y ago
Yeah, OpenStack is very much not a great competitor. If you can afford an expert to set it up for you, great! Otherwise, prepare to invest in other ways. I'm not going to say VMware has a super intuitive system, but I've gotten surprisingly far at times without a lot of deep knowledge. Well, until the *nix-like kernel in the hypervisor misbehaves....
- znpy 3y ago
  If you have any significant workload on vmware you likely already have an expert on payroll.
- Klasiaster 3y ago
  The setup is not too complicated with something like https://opendev.org/openstack/devstack https://opendev.org/openstack/devstack
  - prometheanfire 3y ago
    devstack is for dev work, expect nothing to persist. If you want a deployment choose something like openstack-ansible.
- throw0101c 3y ago
  > Yeah, OpenStack is very much not a great competitor. If you can afford an expert to set it up for you, great! If you know (say) Ubuntu and Ansible, then there's pretty good automation with openstack-ansible and kolla-ansible. There are a number of moving parts, but an intermediate-level Linux sysadmin can get things going without too much difficulty. There are vendor-provided solutions (e.g., from RH) as well if you don't want to do a fully DIY, or want commercial support. * https://www.openstack.org/marketplace/distros/ https://www.openstack.org/marketplace/distros/ At a previous job we had a team of 4-5, and 1.5 FTEs split their time between our HPC cluster and our OpenStack cluster (20+ hypervisors, ~10 Ceph servers; >350 VMs/instances).
mitjam 3y ago
There is also Rancher Harvester (https://harvesterhci.io/ https://harvesterhci.io/) which I think is a breadth of fresh air compared to VMware and OpenStack and then there is the Windows Server based Azure Stack HCI, too. In my experience, VMware‘s biggest moat is the vast support from software vendors. It basically is the default supported platform for almost every business application. This comes with well known best practices, many experienced admins, out of the box solutions from hardware vendors and highly standardized managed services. Most apps leave VMware for the cloud or Kubernetes but not another on-prem VM platform.
- hhh 3y ago
  Harvester has been great for me, except when I wanted to start using a real SAN with it. It's KubeVirt under the hood if I recall correctly, which is also a great project to check out.
- nolist_policy 3y ago
  Or go vendor independent with pacemaker + drbd + libvirt. For that stack you can get support from SUSE, Redhat and Linbit and maybe even Ubuntu. Or just go with Debian if you don't need support.
throw0101c 3y ago
> VMware even has an OpenStack product that is built on vSphere virtualization and NSX-T software defined networking. * https://www.vmware.com/ca/topics/glossary/content/openstack-api.html https://www.vmware.com/ca/topics/glossary/content/openstack-... * https://www.vmware.com/ca/products/openstack.html https://www.vmware.com/ca/products/openstack.html
- thedougd 3y ago
  It's a pretty good product, IME, especially if you're using it with NSX-T.
KaiserPro 3y ago
unless something has changed recently, openstack is a horror to setup and administer. the point of vmware is that is fairly trivial to go from a single server with some VMs, to a massive cluster with live zero-downtime migrations. (and have funky storage, get working snapshots and backups.) Sure openstack can do that. but its non trivial to setup.
- I_Am_Nous 3y ago
  We used Openstack for a while, but overall it was a nightmare. I'm sure it's better than it used to be, but it is not simple to spin up or maintain, and it's not designed to baby VMs -- if one acts up they expect you to delete and recreate it, which works OK for microservices but not for huge Windows terminal servers.
- freeopinion 3y ago
  I see companies hire a team of experts who plan and architect for six months to a year before they are ready to deploy a new VMWare cluster. I've seen a two or three rack cluster cost a million dollars before the architecture team requisitioned a single piece of hardware or software. I've seen a very small outfit spend six months with all hired consultants to set up a 3-server VMWare cluster. VMWare doesn't seem that trivial.
  - bogantech 3y ago
    > I see companies hire a team of experts who plan and architect for six months to a year before they are ready to deploy a new VMWare cluster. Then those people are woefully incompetent and will be regardless of the software they're using.
  - KaiserPro 3y ago
    I mean yeah, but thats the cost of hiring consultants. VMware (wasnt) that hard. How many VMs do you need? work out the storage requirements, triple it, plumb it in, get the right licences, and go.

natbennett 3y ago

Has OpenStack as an organization solved the problem where there’s not really such a thing as OpenStack? In terms of e.g. what APIs will be available and so on? That’s basically what stopped it back in ‘14 or so. It was too hard to provide software that could run on an OpenStack because it was such a choose-your-own-adventure.

pierrelf 3y ago

I was at the Apache Cloudstack conference just a few weeks ago and it seems most of the integrators are pivoting to moving VMware customers to KVM + CloudStack, and have even upstreamed an auto conversion tool. https://cloudstack.apache.org/ https://cloudstack.apache.org/

jabl 3y ago
People at a cloudstack conference think cloudstack is great and want their customers to use it? What's next, a survey of people attending the annual dentistry conference shows that they think dental health is important?
rwmj 3y ago
Are they using virt-v2v for conversions from VMware to KVM or something else?
- Nux 3y ago
  Virt-v2v indeed and thanks for that btw.

csydas 3y ago

While I am excited to see that quite a decent number of past clients I've had are considering proxmox and other virtualization stacks, I am not confident there will be a fast answer for this new change in the virtualization realm. Large and Enterprise companies won't really be bothered by the change except the price, and unless other platforms include all the "Enterprise Check-boxes" they think they want, I'm not sure how fast adoption will be if at all for stacks like Proxmox. I am not defending the business model or the tech offerings of Vmware, Microsoft, etc -- far from it. They're extremely bloated, buggy, and unless you happen to be one of VMware or Microsoft's most expensive customers, the support might as well just be a sin tax. Even for the "biggest" clients, they were not able to get Vmware or Microsoft to budge on some pretty obvious and complete blocker bugs that would drop production environments entirely. Enterprise is weird -- I would be extremely happy if not a bit exhausted supporting a large OSS virtualization infrastructure for clients, but I just do not see it happening without a lot of "carrots" for such companies from the virtualization vendor. Enterprise business is basically about shifting responsibility and liability for your tech-stack to someone else -- yes, it requires strong tech skills to use these tech-stacks, but the more maintenance/liability that can be shifted to the vendor, the more preferable it is for most companies. I don't know what the businesses will choose -- I'm hopeful medium and smaller businesses will shift to more open solutions and from this business, the open solutions will be able to grow and become more attractive choices for enterprise, I'm just not sure that such solutions will avoid the tech industry's obsession with subscription licensing and feature bloat to check boxes.

depereo 3y ago
With all the 'investment' in ISO27001:* validation of the existing infrastructure choices and the like, I'd be astonished if any move in a large company was able to happen within five years. From what I've seen none of them have taken this issue seriously, either, and won't until they get squeezed hard enough, at which point they'll have to... spend five years trying to move!
phpisthebest 3y ago
We would have to see a 2x minimum YoY cost increase just to justify a poliot program to consider moving to something else Keep in mind it is more than just shifting all of the VM workloads to a new platform something not easy in the first place But you also have to factor admin training, backup and recovery systems, monitoriting systems, automation tooling, etc. We have been using vmware since version 3 or 4, our entire team is almost exclusively training on vmware, with limited knowledge on hyperv and virtually none on anything else Something like ProxMox that has no third party support, no veeam support, limited automation tooling, limited support for commerical monitoring (like PRTG) etc etc is probally a non-starter unless those things support it. It would cost us far far far far more than 3x licensing increases to git rid vmware. Broadcom knows this. Honestly VMWare has not been increasing its prices as aggressievly as other vendors anyway, of my software licensing fees I pay annually vmware is one of the cheaper vendors I have, not even breaking into the top 10, I pay more for Veeam, Windows Server, and many many many other things than i do vmware

eddyg 3y ago

Don't forget about the open-source XCP-ng[1] to which you can add Xen Orchestra (which is like vCenter; a key reason people use ESXi) which has a flat-rate (no socket, host or VM limits) $2600/year Enterprise license. And XCP-ng even supports migrating from VMware[3]. [1] https://xcp-ng.org https://xcp-ng.org [2] https://xen-orchestra.com/#!/xo-features/webinterface https://xen-orchestra.com/#!/xo-features/webinterface [3] https://xcp-ng.org/blog/2022/10/19/migrate-from-vmware-to-xcp-ng/ https://xcp-ng.org/blog/2022/10/19/migrate-from-vmware-to-xc... and https://xcp-ng.org/blog/2022/04/13/how-bedrock-streaming-migrated-from-vmware-to-xcp-ng/ https://xcp-ng.org/blog/2022/04/13/how-bedrock-streaming-mig...

bushbaba 3y ago

Suspect it'll be the biggest gift to the public cloud providers. Firms who use VMWare rarely want to self-manage their virtualization tech.

ThinkBeat 3y ago

proxmox is presently not an alternative to the VMWare stack. Not even close. Converting from the VMWare stack⭢ proxmox would be a giant shitshow. I have been a paying user of WMware personally since the beginning and I have worked on their enterprise stack for several years. With the changes being made I cant see myself wanting to spend any personal money on it I and I would not recommend someone to start fresh with it, but for a lot of enterprises, they will, short term, have little choice than to pay the fees. I have no idea about the state of Microsoft HyperV. The little I have worked with it, and that was quite a few years ago now it seemed severely lacking in the tools department. Oracle owns VirtualBox, and even if they started to build it out (or have they?) going to Oracle to avoid high fees is not going to be a good idea.

fuzzylightbulb 3y ago
Microsoft is ending mainstream support of Hyper-V Server 2019 on January 9, 2024 and extended support will end on January 9, 2029. Hyper-V Server 2019 will be the last version of this product and Microsoft is encouraging customers to transition to Azure Stack HCI.
- EvanAnderson 3y ago
  Hyper-V as a role you can install on Windows Server is not being discontinued. They're discontinuing the no cost SKU but the feature isn't going away.
  - sebazzz 3y ago
    Exactly, but the Windows license you need for a proper Hyper-V server is quite high
tlamponi 3y ago
We have many happy customers successfully converting from VMWare to Proxmox VE just fine, what's missimg for your usecase?
pashadee 3y ago