5 ms·

OpenStack started out with AWS EC2 API compatibility. I believe you still have the option to run that. Also the terraform provider is actually quite good, so I

OpenStack started out with AWS EC2 API compatibility. I believe you still have the option to run that. Also the terraform provider is actually quite good, so I don't really think those are the reasons its failed. Here are some reasons it lost IMO: - It doesn't follow the "normal" OSS process of submitting pull requests. Their CI/Repos/Bug tracking systems are different enough that a lot of people probably don't bother trying to contribute - The docs are not great and sometimes do not get updated. - You basically have to run the control plan in containers to make it realistic to deploy. Kolla-Ansible and Openstack Helm are pretty good here - The APIs are pretty slow since they're all in python - You also have to deploy Ceph to get storage, which is itself an entire beast

mschuster91 4y ago
> The APIs are pretty slow since they're all in python That and that everything is so goddamn brittle. It's incredibly hard to debug Openstack because there are soooooooo many different components involved in simply hosting a couple KVM-based VMs, and the fact that I tried that right during the Python 2->3 conversion didn't help either. Also, everyone of the myriad subcomponents needs their own database, their own user accounts, there are no shared configuration files. I get why it is so highly modularized (because of high-availability demand for each component), but FFS they could provide something like "kubeadm" that does all the tiny steps involved into setting up a cluster and joining a new node for you. Once you get it running, it is pretty much smooth sailing until upgrade time, at which it gets really nasty yet again. > The docs are not great and sometimes do not get updated. One cannot help but feel that the docs sometimes haven't even been tested in real life.
- jetbalsa 4y ago
  So the main methods of deploying openstack right now are Openstack-Ansible and Openstack Kolla, Both use Ansible to deploy everything and get the cluster up, but you /really/ have to dig into how things are setup if you are going to get a complicated install online. adding a new node into Kolla isn't too bad but can lead to the stack exploding at times. Upgrading is a huge PITA and I do opt to just wipe and replace most of the time.
- neverartful 4y ago
  "and the fact that I tried that right during the Python 2->3 conversion didn't help either" The huge and painful Python 2->3 transition could not have come at a worse time for OpenStack adoption. I believe it's one of the major reasons why OpenStack was found to be difficult to use, brittle, and documentation not up-to-date.
bogota 4y ago
I have run multiple datacenter automation teams. Essentially providing the plumbing for provisioning bare metal. We looked at open stack several times and never had a team large enough to justify trying to run it. It’s just way too complex and requires too many machines to get the full stack running. We could manage 5k plus machines with just 5 relatively small hosts and keeping the stack extremely simple. Now you are going to say that we don’t provide the same features and that is true but the few cases where it would have been nice didn’t justify it. I can write a datacenter automation system from scratch that i know won’t be causing me alerts at night faster than I can get a resilient openstack setup running. Unless that changed why would I use it.
- throwyo234a65 4y ago
  I am interested by your expertise as we'll soon be building data center automation and would appreciate having a consultant being able to advise us on what we'd need to plan for. Could you please contact me at this address: 7vmq5a5fd@mozmail.com
- mattpallissard 4y ago
  Large bare metal installations are one of those few places where it often makes sense to roll your own for large swaths of the stack.
  - bogota 4y ago
    That is true and why i tried to stay working in that area so long. AWS eventually ate my lunch so I have switched to a pure SWE role now and just happen to setup a lot of the cloud infra as well. I’m hoping some time I can work on DC automation again though. Brings me a lot of joy to get to figure out why the brand new NICs are eating LLDP packets before they get to the hosts and other fun problems you never get to see in the cloud.
neverartful 4y ago
"You also have to deploy Ceph to get storage, which is itself an entire beast" You do not have to use Ceph, although it is the most popular storage backend for OpenStack. Cinder is the OpenStack project that's used to provision block storage for OpenStack. Look at the list of Cinder drivers [0] to see what the storage options are. I do agree with this part: "is itself an entire beast"! [0] https://docs.openstack.org/cinder/latest/reference/support-matrix.html#driver-support-matrix https://docs.openstack.org/cinder/latest/reference/support-m...
- wmf 4y ago
  Many of those drivers either don't work or are shims for very expensive proprietary storage. If you want to run an actual open stack, Ceph was pretty much the option for a long time.

mschuster91 4y ago

> The APIs are pretty slow since they're all in python That and that everything is so goddamn brittle. It's incredibly hard to debug Openstack because there are soooooooo many different components involved in simply hosting a couple KVM-based VMs, and the fact that I tried that right during the Python 2->3 conversion didn't help either. Also, everyone of the myriad subcomponents needs their own database, their own user accounts, there are no shared configuration files. I get why it is so highly modularized (because of high-availability demand for each component), but FFS they could provide something like "kubeadm" that does all the tiny steps involved into setting up a cluster and joining a new node for you. Once you get it running, it is pretty much smooth sailing until upgrade time, at which it gets really nasty yet again. > The docs are not great and sometimes do not get updated. One cannot help but feel that the docs sometimes haven't even been tested in real life.

jetbalsa 4y ago
So the main methods of deploying openstack right now are Openstack-Ansible and Openstack Kolla, Both use Ansible to deploy everything and get the cluster up, but you /really/ have to dig into how things are setup if you are going to get a complicated install online. adding a new node into Kolla isn't too bad but can lead to the stack exploding at times. Upgrading is a huge PITA and I do opt to just wipe and replace most of the time.
neverartful 4y ago
"and the fact that I tried that right during the Python 2->3 conversion didn't help either" The huge and painful Python 2->3 transition could not have come at a worse time for OpenStack adoption. I believe it's one of the major reasons why OpenStack was found to be difficult to use, brittle, and documentation not up-to-date.

bogota 4y ago

I have run multiple datacenter automation teams. Essentially providing the plumbing for provisioning bare metal. We looked at open stack several times and never had a team large enough to justify trying to run it. It’s just way too complex and requires too many machines to get the full stack running. We could manage 5k plus machines with just 5 relatively small hosts and keeping the stack extremely simple. Now you are going to say that we don’t provide the same features and that is true but the few cases where it would have been nice didn’t justify it. I can write a datacenter automation system from scratch that i know won’t be causing me alerts at night faster than I can get a resilient openstack setup running. Unless that changed why would I use it.

throwyo234a65 4y ago
I am interested by your expertise as we'll soon be building data center automation and would appreciate having a consultant being able to advise us on what we'd need to plan for. Could you please contact me at this address: 7vmq5a5fd@mozmail.com
mattpallissard 4y ago
Large bare metal installations are one of those few places where it often makes sense to roll your own for large swaths of the stack.
- bogota 4y ago
  That is true and why i tried to stay working in that area so long. AWS eventually ate my lunch so I have switched to a pure SWE role now and just happen to setup a lot of the cloud infra as well. I’m hoping some time I can work on DC automation again though. Brings me a lot of joy to get to figure out why the brand new NICs are eating LLDP packets before they get to the hosts and other fun problems you never get to see in the cloud.

neverartful 4y ago

"You also have to deploy Ceph to get storage, which is itself an entire beast" You do not have to use Ceph, although it is the most popular storage backend for OpenStack. Cinder is the OpenStack project that's used to provision block storage for OpenStack. Look at the list of Cinder drivers [0] to see what the storage options are. I do agree with this part: "is itself an entire beast"! [0] https://docs.openstack.org/cinder/latest/reference/support-matrix.html#driver-support-matrix https://docs.openstack.org/cinder/latest/reference/support-m...

wmf 4y ago
Many of those drivers either don't work or are shims for very expensive proprietary storage. If you want to run an actual open stack, Ceph was pretty much the option for a long time.