8 ms·

Is OpenStack fighting a lost battle?

63 points by __warlord__ 4y ago

YUNG_BEEF 4y ago
more_corn 4y ago
Open stack lost because it doesn’t do the job. I’ve used it at two clients and it lacks critical features. At least with k8s you can bolt on the pieces you need.
- jetbalsa 4y ago
  I see Openstack as a replacement for running your own Digital Ocean / Linode, I've yet to find a product that allows me to run fat VMs, with any kind of networking I really want (I use the crap out of OVN and duplicate network blocks / isolation) and I can just yeet a project at a student and not worry about them going over quota with it.
- rrdharan 4y ago
  Exactly this.. Kubernetes provided some realizable value from the very first release. OpenStack has been around for 10+ years and mostly just provided slide-ware value and maybe some strategic negotiating leverage and insurance of various forms. It’s actually interesting to me to hear from the sibling comment, that someone actually got to 1000s of VMs under management. I’m still not convinced it’s necessary or adding value in such situations vs “the field”.
  - jetbalsa 4y ago
    Overall.. I am just running core services on openstack, I do like its project isolation. This allows me to give a account to a student group and not worry about them. this comes almost free with openstack. vSphere and Proxmox really don't do this well. the easisted to use deployment of openstack is "Kolla" and it really isn't maintaining anything outside of core services. its a real crapshoot if any of the "cloud services" will deploy at all or just explode your stack.
  - mschuster91 4y ago
    > OpenStack has been around for 10+ years and mostly just provided slide-ware value and maybe some strategic negotiating leverage and insurance of various forms. There isn't much competition in the market and most of it is closed source, proprietary stuff - OpenStack in contrast is open source and made to fit the needs of CERN (they run 320.000 nodes). That maybe also explains the choice of Python and the sorry state of its documentation - scientists have ample time to deal with bullshit, corporate does not. [1] https://www.openstack.org/videos/summits/berlin-2018/towards-fully-automated-cern-private-cloud https://www.openstack.org/videos/summits/berlin-2018/towards...
    - ashinclouds 4y ago
      So, CERN doesn't run that many machines. They do have that many cores across their OpenStack fleet. I actually think it is even more than that now since they've tweeted graphs[1] from adding 800+ new physical nodes to their clusters. Anyway, it is not all about CERN, but they are an active community contributor. They are an operator who speaks publicly of their work, challenges, and successes. They have championed fixing some issues they have encountered at their scale, and provided feedback to the various projects on aspects which would help them solve their infrastructure automation issues. The same has occurred with the telecoms and other verticals with specific needs that cannot use public clouds. As for docs, they vary from project to project, but one aspect I've encountered as a frustration and heard large scale operators echo, is that often search index results seem to favor some older specific release as opposed to the latest release, which creates a lot of confusion if your not aware of it and don't see the warning on the rendered webpage. Some other popular open source projects have encountered the same exact challenge, and it is a hard need to balance. [1] https://twitter.com/ArneWiebalck/status/1478778715005984772 https://twitter.com/ArneWiebalck/status/1478778715005984772
      - hyperionplays 4y ago
        China Mobile Cloud is built on openstack and has 1M+ cpu's at last count.
  - deleted 4y ago
    [deleted]
deleted 4y ago
[deleted]
donedealomg 4y ago
jetbalsa 4y ago
As someone who is running a rather large Openstack install (3500~ VMs) I agree with the statements on what needs to change, Making it easier to install would be nice, Kolla / Openstack-Anisble are pretty OK, just when they explode its really hard to figure out why it exploded, This on top of lack luster community support (Their IRC channels are /very/ dead when it comes to support) what people mean when its "consulting-ware" is that there are no real documentation on the failure states, tunables, and intermixed services. it has rather shit logs (90% of the time its just a python stacktrace shown to the end user) so you have to hit up a consultant that has had to deal with all the BS that openstack has.
- candiddevmike 4y ago
  IMO vendors made OpenStack that way to ensure you always need some kind of third party to manage it. Kubernetes is going the same route.
  - jetbalsa 4y ago
    and oh boy do they charge for managing a Openstack, a 8 Node support contract from Redhat was north of 80k/yr on top of telling me that Openstack is dead, have you tried Openshift :V
    - neverartful 4y ago
      Did you mean to say 'charge' instead of 'change'?
      - jetbalsa 4y ago
        Yes, My comment has been updated.
- OpenStackN00b 4y ago
  Regarding the installation / maintenance: Are you aware about Yaook (https://yaook.cloud/ https://yaook.cloud/)?
rgoulter 4y ago
Kubernetes as an abstraction layer above clouds, whereas OpenStack is a cloud computing environment. Though, I heartily agree with "it's easy to get started with Kubernetes, and difficult to get started with AWS". When I tried looking at deploying an application to OpenStack (as an alternative to deploying to AWS), I was barely able to setup an OpenStack cloud environment on my computer. The most practical option was to use a public cloud which provided OpenStack APIs. Even with public clouds with OpenStack APIs.. my experience trying "compute VM with an internet-facing load-balancer in front" was that different public clouds required different setup for this. -- TBH, this surprised me. I understand that you can't just copy-paste Terraform code for AWS and have it work for GCP; but I had expected to be able to copy-paste Terraform code for one OpenStack cloud and have it work with another OpenStack cloud.
- jetbalsa 4y ago
  For your "local" deployment of openstack, look into "Devstack" -- For the cross openstack terraform, the only main issue I ran into was images for the VMs you are going to use are going to differ a bunch, same with networking. as everyone has their own take on how networking should work in their openstack.
- nicolaslem 4y ago
  The reason the network setups do not really match between public OpenStack cloud providers is that most of them significantly change that part of OpenStack. My knowledge is very outdated but I remember that the networking part had significant scalability issues that pushed every provider to introduce their secret sauce.
  - jetbalsa 4y ago
    Yep, a "stock" openstack install will tend to use OVN or VLANs -- Most public clouds throw that out completely since they are going to use their existing methods for networking. its a shame, I've not found a public cloud that supports something like OVN.
    - 4y ago
funnym0nk3y 4y ago
I was very interested in Openstack for a homelab. I tried to wrap my head around it several times. It was just too difficult. Too difficult to install. Too difficult to understand the concept with the given documentation and resources. With K8s you have all those opinionated distributions like K3s, minikube etc. which lower the bar considerably. There are countless guides on YouTube.
samcat116 4y ago
OpenStack started out with AWS EC2 API compatibility. I believe you still have the option to run that. Also the terraform provider is actually quite good, so I don't really think those are the reasons its failed. Here are some reasons it lost IMO: - It doesn't follow the "normal" OSS process of submitting pull requests. Their CI/Repos/Bug tracking systems are different enough that a lot of people probably don't bother trying to contribute - The docs are not great and sometimes do not get updated. - You basically have to run the control plan in containers to make it realistic to deploy. Kolla-Ansible and Openstack Helm are pretty good here - The APIs are pretty slow since they're all in python - You also have to deploy Ceph to get storage, which is itself an entire beast
- mschuster91 4y ago
  > The APIs are pretty slow since they're all in python That and that everything is so goddamn brittle. It's incredibly hard to debug Openstack because there are soooooooo many different components involved in simply hosting a couple KVM-based VMs, and the fact that I tried that right during the Python 2->3 conversion didn't help either. Also, everyone of the myriad subcomponents needs their own database, their own user accounts, there are no shared configuration files. I get why it is so highly modularized (because of high-availability demand for each component), but FFS they could provide something like "kubeadm" that does all the tiny steps involved into setting up a cluster and joining a new node for you. Once you get it running, it is pretty much smooth sailing until upgrade time, at which it gets really nasty yet again. > The docs are not great and sometimes do not get updated. One cannot help but feel that the docs sometimes haven't even been tested in real life.
  - jetbalsa 4y ago
    So the main methods of deploying openstack right now are Openstack-Ansible and Openstack Kolla, Both use Ansible to deploy everything and get the cluster up, but you /really/ have to dig into how things are setup if you are going to get a complicated install online. adding a new node into Kolla isn't too bad but can lead to the stack exploding at times. Upgrading is a huge PITA and I do opt to just wipe and replace most of the time.
stephen_g 4y ago
I was looking at it for a small VMWare cluster replacement (3 servers or so, couple of dozen VMs, but should grow larger over time) for internal use at our small electronics company, but it sounds too complex. Proxmox looks easier (and probably more suited to this tiny scale?) so I’ll probably try that.
- jetbalsa 4y ago
  Yes, if you are just trying to replace vSphere/ESXi Proxmox is pretty good on that scale. Openstack is great if you are trying to run a internal Digital Ocean style service.
- guerby 4y ago
  Proxmox has also web UI for creating a ceph cluster, very good upgrade documentation (went from PVE 6 ceph 15 to PVE 7 ceph 16 without issue), and paying support if you need some. And don't forget to add Proxmox Backup Server to your list, live restore is a fantastic feature of it :)
- dcpit 4y ago
  We used oVirt since 7 years. About 550 VMs (Linux and Windows Server). Works really fine.
jbotdev 4y ago
I think this misses a much larger issue, which is that running your own datacenters (or just your own hardware) is becoming very unpopular. OpenStack could have all the features in the world, but at the end of the day you need a team to run it. Having worked with large footprints on both OpenStack and AWS, it’s also clear that there are just inherent difficulties with running your own hardware, especially in your own data center. Even if you make the investment in a good infra team, it’s cost prohibitive to get anywhere near the experience of something like EC2 in terms of hardware availability and hourly billing. Not to mention they’re literally inventing their own hardware for things like high-performance storage and ARM servers.
- hotpotamus 4y ago
  Do you worry that given those trends, at some point AWS (or at least a tech oligopoly) will own all significant internet infrastructure? I do.
  - Sevii 4y ago
    Could be a concern if hosting becomes impractical for normal people much like how email is basically impossible to host yourself.
  - scott_w 4y ago
    Yes but when you're trying to build and maintain a product, ownership of the internet's infrastructure is low down on your list of problems. It's similar to the fact Microsoft owned desktop computing for many years. Yes, it was a problem, but the people purchasing Windows and Office weren't interested in solving it. They just want to do their jobs.
    - busterarm 4y ago
      Ma Bell used to be the only phone company and arguably did a better job than since the phone industry was deregulated...except maybe for introducing new products. (Slightly biased former Telco employee opinion)
  - 4y ago
brentcetinich 4y ago
Does anyone remember eucalyptus ? what happened to them?
- mdaniel 4y ago
  They rebranded: https://github.com/corymbia/eucalyptus#readme https://github.com/corymbia/eucalyptus#readme but development is either happening "offline" or they've stalled development because https://github.com/Corymbia/eucalyptus-blog/blob/master/content/post/eucalyptus-51-release.md https://github.com/Corymbia/eucalyptus-blog/blob/master/cont... is the most recent blog update It's also a beast to even compile and package, to say nothing of actually deploy
sc68cal 4y ago
The author is asking for OpenStack to "Implement transparent APIs from AWS, Azure, GCP for OpenStack" There _was_ an EC2 API that OpenStack nova had back in the day, that became unmaintained and was eventually removed http://cloudscaling.com/blog/openstack/the-future-of-openstacks-ec2-apis/ http://cloudscaling.com/blog/openstack/the-future-of-opensta... This blog was from 2015, and the end result was that nobody was willing to step up and do the hard work of creating these API compatibility layers. Frankly, there were barely enough resources being put in by large companies, to keep the OpenStack APIs themselves well maintained. So, I think the issue of OpenStack was that everyone suddenly decided to shift to containers, instead of VMs for their deployments, and OpenStack no longer was required.
- candiddevmike 4y ago
  OpenStack tried to become a container scheduler (Magnum), and then shifted to making kubernetes a first class citizen, it wasn't well received. Oddly enough some of the kubernetes bare metal stuff is borrowing from OpenStack (Metal3 is built on ironic)
  - samcat116 4y ago
    The ironic stuff is some of the best services within OpenStack. Its a hard problem and it works surprisingly well.
    - neverartful 4y ago
      'Ironic' is the OpenStack project that deals with bare-metal provisioning. And there is indeed some very cool stuff there!
    - jayofdoom 4y ago
      Thanks! Ironic (ironicbaremetal.org) can be operated standalone outside of Ironic, and we do pay attention to non-OpenStack use cases (e.g. https://metal3.io/ https://metal3.io/). If there's anything you're trying to do with Ironic that you're having trouble with, feel free to come chat. We're on IRC/Matrix on irc.oftc.net, #openstack-ironic. -Jay, Ironic PTL
BirAdam 4y ago
I have deployed OpenStack into production environments and it always takes a considerable amount of time and effort. Once up and running, however, it is awesome. I think its lack of success in gaining more market share is that documentation is not good, and the project lacks evangelists. Never underestimate marketing.
rwmj 4y ago
> Is it because is “difficult” to install? Yes ... but somehow Kubernetes has an even more confusing and difficult installation story (and don't get me started on OpenShift).
- ojhughes 4y ago
  I wouldn’t say that “gcloud container clusters create” is too complicated. One command and you’re up and running. I agree OpenShift is an entirely different kettle of fish
  - rwmj 4y ago
    I mean installing it, not running it in someone's cloud.
justinsb 4y ago
I also started (non-trivially) contributing to OSS with OpenStack, and later switched to contributing to Kubernetes. I whole-heartedly agree with the points made here; these same observations really shaped how I chose to contribute to Kubernetes. Trying to get all the AWS early adopters to abandon their investment and move to something else seemed like it was a huge barrier to adoption, so I started off contributing to the AWS support for kubernetes and making sure that kubernetes worked well there. But the pains of installing OpenStack from upstream were fresh in my mind, so I also contributed to make sure that kube-up worked well and then (as we outgrew that architecture) started the kOps project. My goal is that you should always be able to run OSS kubernetes from upstream, even if you just treat that as an insurance policy that gives you the confidence to use a managed service. I think the miracle of kubernetes is that I was just one of a large number of people here, each of us bringing our past experiences to make kubernetes better in some way. And the community continues to grow with people each addressing their own painpoints, which does create a lot of churn/progress (depending on your perspective!) I'd say that the people that organized the community were the unsung heroes here, and I suspect they learned a lot from OpenStack as well.
- yrgulation 4y ago
  > I started off contributing to the AWS support for kubernetes and making sure that kubernetes worked well there In effect providing free work to amazon.
  - stubish 4y ago
    It is kind of the point with open software that, when you scratch an itch, others also benefit.
tristor 4y ago
OpenStack was done almost a decade ago. I was involved in the early days, it was abandoned by its founding entities, and IBM ruined it further. I didn't realize anyone was still talking about it.
swyx 4y ago
lol i'm not sure it was ever really a debate
didip 4y ago
The complexity of running OpenStack is so large that running your own Hypervisor looked simple in comparison.
cmsj 4y ago
Compare https://www.stackalytics.io/?metric=commits&release=folsom https://www.stackalytics.io/?metric=commits&release=folsom from 10 years ago with https://www.stackalytics.io/?metric=commits&release=zed https://www.stackalytics.io/?metric=commits&release=zed from this month. Notice that 2012's Folsom release was mostly made by OpenStack providers while 2022's Zed release was mostly made by Red Hat and OpenStack consumers. (might help to expand to "All"). OpenStack did find some markets, enough to sustain a couple of vendors, but it didn't replace AWS, but it's also not trying to replace AWS anymore, it's trying to continue to serve the users it did find. Shifting more towards my personal opinion: Containers are, in most cases, a more sensible unit of software deployment than VMs are, so OpenStack was always going to "lose" to something built around containers. Shifting more towards other peoples' opinions: Severless is, in most cases, a more sensible unit of software deployment than containers are, so k8s will either be supplanted or become uninteresting infrastructure at some point.
- z33k 4y ago
  What is a serverless deployment unit? We already have Fargate, so no need to choose between serverless and k8s.
  - vbezhenar 4y ago
    I don't understand why containers are not serverless deployment unit. Run container, send http request to it, shutdown if necessary.
    - cmsj 4y ago
      I'll start by repeating that the serverless part of my comment was me extending the things that other people are predicting for the future. I'm not 100% I agree, but I'll continue their position: containers are only really better than VMs because they use fewer resources. They are not better than VMs in that you still have a whole software stack inside them that you have to care about. Anyone building their own containers should also be maintaining a whole pipeline for updating all of the software in the containers that isn't their application. Narrowing the unit of software deployment down to the application itself, removes a lot of that (or rather abstracts it to whomever is responsible for maintaining the serverless runtime containers you use).
pphysch 4y ago
OpenStack feels like a promising prototype that accidently made it into production. I think HashiStack is a much more promising "v1" but not necessarily the "Linux of Cloud OS" yet.
bvrmn 4y ago
Rewrite it in Rust. /s Implement transparent APIs from AWS, Azure, GCP for OpenStack, so we can reuse Pulumi and Terraform. Make it easier to install it. Make it easier to work with Prometheus stacks, Service mesh and other cloud native tools. Make it less consulting-ware… whatever that means. Uff. How about to make OpenStack manageable by small infra team without paying to vendors? I guess it's the last point.
SirMonkey 4y ago
A small trip down to memory lane: My first OpenStack install was on a bunch of VMs. Got that to run following the official documentation. Got enough people interested in it, that I was "allowed" to salvage whatever hardware I could find and put in a Rack. Getting Neutron to work was a head first intro to networks and in the end I had to learn how to configure actual network hardware. After adding Ceph on top of old spinning disk, watching them die one after the other and the cluster was still up with performance not worst than the IBM-Stuff we were paying for gave me warm-fuzzy feelings. Thanks to Triple-O, Fuel and most of the other "installers" at that time I learned that - the "big guys" don't know what they are doing either ;-) At the end of the day, it took the "mystical" out of the cloud, and for Junior me that was good - it also saved me from the path management wanted me to go: Sharepoint developer/admin.
nwilkens 4y ago
We went down the Openstack path long ago.. but ended up choosing Triton DataCenter. The simplicity in operational support is just amazing. We’ve since acquired the commercial support business from Joyent this year for Triton. Triton is much simpler to operate, manage and support - and we’ve never looked back!
tianzhou 4y ago
~5 years ago, I had a lunch interview with a candidate from IBM working on OpenStack. He complained it took forever to pass all integration tests for any changes. The ecosystems were so fragmented, and several large vendors were participating, and each had its interests. I knew it was doomed.

63 points by __warlord__ 4y ago

Open stack lost because it doesn’t do the job. I’ve used it at two clients and it lacks critical features. At least with k8s you can bolt on the pieces you need.

jetbalsa 4y ago
I see Openstack as a replacement for running your own Digital Ocean / Linode, I've yet to find a product that allows me to run fat VMs, with any kind of networking I really want (I use the crap out of OVN and duplicate network blocks / isolation) and I can just yeet a project at a student and not worry about them going over quota with it.
rrdharan 4y ago
Exactly this.. Kubernetes provided some realizable value from the very first release. OpenStack has been around for 10+ years and mostly just provided slide-ware value and maybe some strategic negotiating leverage and insurance of various forms. It’s actually interesting to me to hear from the sibling comment, that someone actually got to 1000s of VMs under management. I’m still not convinced it’s necessary or adding value in such situations vs “the field”.
- jetbalsa 4y ago
  Overall.. I am just running core services on openstack, I do like its project isolation. This allows me to give a account to a student group and not worry about them. this comes almost free with openstack. vSphere and Proxmox really don't do this well. the easisted to use deployment of openstack is "Kolla" and it really isn't maintaining anything outside of core services. its a real crapshoot if any of the "cloud services" will deploy at all or just explode your stack.
- mschuster91 4y ago
  > OpenStack has been around for 10+ years and mostly just provided slide-ware value and maybe some strategic negotiating leverage and insurance of various forms. There isn't much competition in the market and most of it is closed source, proprietary stuff - OpenStack in contrast is open source and made to fit the needs of CERN (they run 320.000 nodes). That maybe also explains the choice of Python and the sorry state of its documentation - scientists have ample time to deal with bullshit, corporate does not. [1] https://www.openstack.org/videos/summits/berlin-2018/towards-fully-automated-cern-private-cloud https://www.openstack.org/videos/summits/berlin-2018/towards...
  - ashinclouds 4y ago
    So, CERN doesn't run that many machines. They do have that many cores across their OpenStack fleet. I actually think it is even more than that now since they've tweeted graphs[1] from adding 800+ new physical nodes to their clusters. Anyway, it is not all about CERN, but they are an active community contributor. They are an operator who speaks publicly of their work, challenges, and successes. They have championed fixing some issues they have encountered at their scale, and provided feedback to the various projects on aspects which would help them solve their infrastructure automation issues. The same has occurred with the telecoms and other verticals with specific needs that cannot use public clouds. As for docs, they vary from project to project, but one aspect I've encountered as a frustration and heard large scale operators echo, is that often search index results seem to favor some older specific release as opposed to the latest release, which creates a lot of confusion if your not aware of it and don't see the warning on the rendered webpage. Some other popular open source projects have encountered the same exact challenge, and it is a hard need to balance. [1] https://twitter.com/ArneWiebalck/status/1478778715005984772 https://twitter.com/ArneWiebalck/status/1478778715005984772
    - hyperionplays 4y ago
      China Mobile Cloud is built on openstack and has 1M+ cpu's at last count.
- deleted 4y ago
  [deleted]

deleted 4y ago

[deleted]

donedealomg 4y ago

jetbalsa 4y ago

As someone who is running a rather large Openstack install (3500~ VMs) I agree with the statements on what needs to change, Making it easier to install would be nice, Kolla / Openstack-Anisble are pretty OK, just when they explode its really hard to figure out why it exploded, This on top of lack luster community support (Their IRC channels are /very/ dead when it comes to support) what people mean when its "consulting-ware" is that there are no real documentation on the failure states, tunables, and intermixed services. it has rather shit logs (90% of the time its just a python stacktrace shown to the end user) so you have to hit up a consultant that has had to deal with all the BS that openstack has.

candiddevmike 4y ago
IMO vendors made OpenStack that way to ensure you always need some kind of third party to manage it. Kubernetes is going the same route.
- jetbalsa 4y ago
  and oh boy do they charge for managing a Openstack, a 8 Node support contract from Redhat was north of 80k/yr on top of telling me that Openstack is dead, have you tried Openshift :V
  - neverartful 4y ago
    Did you mean to say 'charge' instead of 'change'?
    - jetbalsa 4y ago
      Yes, My comment has been updated.
OpenStackN00b 4y ago
Regarding the installation / maintenance: Are you aware about Yaook (https://yaook.cloud/ https://yaook.cloud/)?

rgoulter 4y ago

Kubernetes as an abstraction layer above clouds, whereas OpenStack is a cloud computing environment. Though, I heartily agree with "it's easy to get started with Kubernetes, and difficult to get started with AWS". When I tried looking at deploying an application to OpenStack (as an alternative to deploying to AWS), I was barely able to setup an OpenStack cloud environment on my computer. The most practical option was to use a public cloud which provided OpenStack APIs. Even with public clouds with OpenStack APIs.. my experience trying "compute VM with an internet-facing load-balancer in front" was that different public clouds required different setup for this. -- TBH, this surprised me. I understand that you can't just copy-paste Terraform code for AWS and have it work for GCP; but I had expected to be able to copy-paste Terraform code for one OpenStack cloud and have it work with another OpenStack cloud.

jetbalsa 4y ago
For your "local" deployment of openstack, look into "Devstack" -- For the cross openstack terraform, the only main issue I ran into was images for the VMs you are going to use are going to differ a bunch, same with networking. as everyone has their own take on how networking should work in their openstack.
nicolaslem 4y ago
The reason the network setups do not really match between public OpenStack cloud providers is that most of them significantly change that part of OpenStack. My knowledge is very outdated but I remember that the networking part had significant scalability issues that pushed every provider to introduce their secret sauce.
- jetbalsa 4y ago
  Yep, a "stock" openstack install will tend to use OVN or VLANs -- Most public clouds throw that out completely since they are going to use their existing methods for networking. its a shame, I've not found a public cloud that supports something like OVN.
  - 4y ago

funnym0nk3y 4y ago

I was very interested in Openstack for a homelab. I tried to wrap my head around it several times. It was just too difficult. Too difficult to install. Too difficult to understand the concept with the given documentation and resources. With K8s you have all those opinionated distributions like K3s, minikube etc. which lower the bar considerably. There are countless guides on YouTube.

samcat116 4y ago

OpenStack started out with AWS EC2 API compatibility. I believe you still have the option to run that. Also the terraform provider is actually quite good, so I don't really think those are the reasons its failed. Here are some reasons it lost IMO: - It doesn't follow the "normal" OSS process of submitting pull requests. Their CI/Repos/Bug tracking systems are different enough that a lot of people probably don't bother trying to contribute - The docs are not great and sometimes do not get updated. - You basically have to run the control plan in containers to make it realistic to deploy. Kolla-Ansible and Openstack Helm are pretty good here - The APIs are pretty slow since they're all in python - You also have to deploy Ceph to get storage, which is itself an entire beast

mschuster91 4y ago
> The APIs are pretty slow since they're all in python That and that everything is so goddamn brittle. It's incredibly hard to debug Openstack because there are soooooooo many different components involved in simply hosting a couple KVM-based VMs, and the fact that I tried that right during the Python 2->3 conversion didn't help either. Also, everyone of the myriad subcomponents needs their own database, their own user accounts, there are no shared configuration files. I get why it is so highly modularized (because of high-availability demand for each component), but FFS they could provide something like "kubeadm" that does all the tiny steps involved into setting up a cluster and joining a new node for you. Once you get it running, it is pretty much smooth sailing until upgrade time, at which it gets really nasty yet again. > The docs are not great and sometimes do not get updated. One cannot help but feel that the docs sometimes haven't even been tested in real life.
- jetbalsa 4y ago
  So the main methods of deploying openstack right now are Openstack-Ansible and Openstack Kolla, Both use Ansible to deploy everything and get the cluster up, but you /really/ have to dig into how things are setup if you are going to get a complicated install online. adding a new node into Kolla isn't too bad but can lead to the stack exploding at times. Upgrading is a huge PITA and I do opt to just wipe and replace most of the time.

stephen_g 4y ago

I was looking at it for a small VMWare cluster replacement (3 servers or so, couple of dozen VMs, but should grow larger over time) for internal use at our small electronics company, but it sounds too complex. Proxmox looks easier (and probably more suited to this tiny scale?) so I’ll probably try that.

jetbalsa 4y ago
Yes, if you are just trying to replace vSphere/ESXi Proxmox is pretty good on that scale. Openstack is great if you are trying to run a internal Digital Ocean style service.
guerby 4y ago
Proxmox has also web UI for creating a ceph cluster, very good upgrade documentation (went from PVE 6 ceph 15 to PVE 7 ceph 16 without issue), and paying support if you need some. And don't forget to add Proxmox Backup Server to your list, live restore is a fantastic feature of it :)
dcpit 4y ago
We used oVirt since 7 years. About 550 VMs (Linux and Windows Server). Works really fine.

jbotdev 4y ago

I think this misses a much larger issue, which is that running your own datacenters (or just your own hardware) is becoming very unpopular. OpenStack could have all the features in the world, but at the end of the day you need a team to run it. Having worked with large footprints on both OpenStack and AWS, it’s also clear that there are just inherent difficulties with running your own hardware, especially in your own data center. Even if you make the investment in a good infra team, it’s cost prohibitive to get anywhere near the experience of something like EC2 in terms of hardware availability and hourly billing. Not to mention they’re literally inventing their own hardware for things like high-performance storage and ARM servers.

hotpotamus 4y ago
Do you worry that given those trends, at some point AWS (or at least a tech oligopoly) will own all significant internet infrastructure? I do.
- Sevii 4y ago
  Could be a concern if hosting becomes impractical for normal people much like how email is basically impossible to host yourself.
- scott_w 4y ago
  Yes but when you're trying to build and maintain a product, ownership of the internet's infrastructure is low down on your list of problems. It's similar to the fact Microsoft owned desktop computing for many years. Yes, it was a problem, but the people purchasing Windows and Office weren't interested in solving it. They just want to do their jobs.
  - busterarm 4y ago
    Ma Bell used to be the only phone company and arguably did a better job than since the phone industry was deregulated...except maybe for introducing new products. (Slightly biased former Telco employee opinion)
- 4y ago

brentcetinich 4y ago

Does anyone remember eucalyptus ? what happened to them?

mdaniel 4y ago
They rebranded: https://github.com/corymbia/eucalyptus#readme https://github.com/corymbia/eucalyptus#readme but development is either happening "offline" or they've stalled development because https://github.com/Corymbia/eucalyptus-blog/blob/master/content/post/eucalyptus-51-release.md https://github.com/Corymbia/eucalyptus-blog/blob/master/cont... is the most recent blog update It's also a beast to even compile and package, to say nothing of actually deploy

sc68cal 4y ago

The author is asking for OpenStack to "Implement transparent APIs from AWS, Azure, GCP for OpenStack" There _was_ an EC2 API that OpenStack nova had back in the day, that became unmaintained and was eventually removed http://cloudscaling.com/blog/openstack/the-future-of-openstacks-ec2-apis/ http://cloudscaling.com/blog/openstack/the-future-of-opensta... This blog was from 2015, and the end result was that nobody was willing to step up and do the hard work of creating these API compatibility layers. Frankly, there were barely enough resources being put in by large companies, to keep the OpenStack APIs themselves well maintained. So, I think the issue of OpenStack was that everyone suddenly decided to shift to containers, instead of VMs for their deployments, and OpenStack no longer was required.

candiddevmike 4y ago
OpenStack tried to become a container scheduler (Magnum), and then shifted to making kubernetes a first class citizen, it wasn't well received. Oddly enough some of the kubernetes bare metal stuff is borrowing from OpenStack (Metal3 is built on ironic)
- samcat116 4y ago
  The ironic stuff is some of the best services within OpenStack. Its a hard problem and it works surprisingly well.
  - neverartful 4y ago
    'Ironic' is the OpenStack project that deals with bare-metal provisioning. And there is indeed some very cool stuff there!
  - jayofdoom 4y ago
    Thanks! Ironic (ironicbaremetal.org) can be operated standalone outside of Ironic, and we do pay attention to non-OpenStack use cases (e.g. https://metal3.io/ https://metal3.io/). If there's anything you're trying to do with Ironic that you're having trouble with, feel free to come chat. We're on IRC/Matrix on irc.oftc.net, #openstack-ironic. -Jay, Ironic PTL

BirAdam 4y ago

I have deployed OpenStack into production environments and it always takes a considerable amount of time and effort. Once up and running, however, it is awesome. I think its lack of success in gaining more market share is that documentation is not good, and the project lacks evangelists. Never underestimate marketing.

rwmj 4y ago

> Is it because is “difficult” to install? Yes ... but somehow Kubernetes has an even more confusing and difficult installation story (and don't get me started on OpenShift).

ojhughes 4y ago
I wouldn’t say that “gcloud container clusters create” is too complicated. One command and you’re up and running. I agree OpenShift is an entirely different kettle of fish
- rwmj 4y ago
  I mean installing it, not running it in someone's cloud.

justinsb 4y ago

I also started (non-trivially) contributing to OSS with OpenStack, and later switched to contributing to Kubernetes. I whole-heartedly agree with the points made here; these same observations really shaped how I chose to contribute to Kubernetes. Trying to get all the AWS early adopters to abandon their investment and move to something else seemed like it was a huge barrier to adoption, so I started off contributing to the AWS support for kubernetes and making sure that kubernetes worked well there. But the pains of installing OpenStack from upstream were fresh in my mind, so I also contributed to make sure that kube-up worked well and then (as we outgrew that architecture) started the kOps project. My goal is that you should always be able to run OSS kubernetes from upstream, even if you just treat that as an insurance policy that gives you the confidence to use a managed service. I think the miracle of kubernetes is that I was just one of a large number of people here, each of us bringing our past experiences to make kubernetes better in some way. And the community continues to grow with people each addressing their own painpoints, which does create a lot of churn/progress (depending on your perspective!) I'd say that the people that organized the community were the unsung heroes here, and I suspect they learned a lot from OpenStack as well.

yrgulation 4y ago
> I started off contributing to the AWS support for kubernetes and making sure that kubernetes worked well there In effect providing free work to amazon.
- stubish 4y ago
  It is kind of the point with open software that, when you scratch an itch, others also benefit.

tristor 4y ago

OpenStack was done almost a decade ago. I was involved in the early days, it was abandoned by its founding entities, and IBM ruined it further. I didn't realize anyone was still talking about it.

swyx 4y ago

lol i'm not sure it was ever really a debate

didip 4y ago

The complexity of running OpenStack is so large that running your own Hypervisor looked simple in comparison.

cmsj 4y ago

Compare https://www.stackalytics.io/?metric=commits&release=folsom https://www.stackalytics.io/?metric=commits&release=folsom from 10 years ago with https://www.stackalytics.io/?metric=commits&release=zed https://www.stackalytics.io/?metric=commits&release=zed from this month. Notice that 2012's Folsom release was mostly made by OpenStack providers while 2022's Zed release was mostly made by Red Hat and OpenStack consumers. (might help to expand to "All"). OpenStack did find some markets, enough to sustain a couple of vendors, but it didn't replace AWS, but it's also not trying to replace AWS anymore, it's trying to continue to serve the users it did find. Shifting more towards my personal opinion: Containers are, in most cases, a more sensible unit of software deployment than VMs are, so OpenStack was always going to "lose" to something built around containers. Shifting more towards other peoples' opinions: Severless is, in most cases, a more sensible unit of software deployment than containers are, so k8s will either be supplanted or become uninteresting infrastructure at some point.

z33k 4y ago
What is a serverless deployment unit? We already have Fargate, so no need to choose between serverless and k8s.
- vbezhenar 4y ago
  I don't understand why containers are not serverless deployment unit. Run container, send http request to it, shutdown if necessary.
  - cmsj 4y ago
    I'll start by repeating that the serverless part of my comment was me extending the things that other people are predicting for the future. I'm not 100% I agree, but I'll continue their position: containers are only really better than VMs because they use fewer resources. They are not better than VMs in that you still have a whole software stack inside them that you have to care about. Anyone building their own containers should also be maintaining a whole pipeline for updating all of the software in the containers that isn't their application. Narrowing the unit of software deployment down to the application itself, removes a lot of that (or rather abstracts it to whomever is responsible for maintaining the serverless runtime containers you use).

pphysch 4y ago

OpenStack feels like a promising prototype that accidently made it into production. I think HashiStack is a much more promising "v1" but not necessarily the "Linux of Cloud OS" yet.

bvrmn 4y ago

Rewrite it in Rust. /s Implement transparent APIs from AWS, Azure, GCP for OpenStack, so we can reuse Pulumi and Terraform. Make it easier to install it. Make it easier to work with Prometheus stacks, Service mesh and other cloud native tools. Make it less consulting-ware… whatever that means. Uff. How about to make OpenStack manageable by small infra team without paying to vendors? I guess it's the last point.

SirMonkey 4y ago

A small trip down to memory lane: My first OpenStack install was on a bunch of VMs. Got that to run following the official documentation. Got enough people interested in it, that I was "allowed" to salvage whatever hardware I could find and put in a Rack. Getting Neutron to work was a head first intro to networks and in the end I had to learn how to configure actual network hardware. After adding Ceph on top of old spinning disk, watching them die one after the other and the cluster was still up with performance not worst than the IBM-Stuff we were paying for gave me warm-fuzzy feelings. Thanks to Triple-O, Fuel and most of the other "installers" at that time I learned that - the "big guys" don't know what they are doing either ;-) At the end of the day, it took the "mystical" out of the cloud, and for Junior me that was good - it also saved me from the path management wanted me to go: Sharepoint developer/admin.

nwilkens 4y ago

We went down the Openstack path long ago.. but ended up choosing Triton DataCenter. The simplicity in operational support is just amazing. We’ve since acquired the commercial support business from Joyent this year for Triton. Triton is much simpler to operate, manage and support - and we’ve never looked back!

tianzhou 4y ago

~5 years ago, I had a lunch interview with a candidate from IBM working on OpenStack. He complained it took forever to pass all integration tests for any changes. The ecosystems were so fragmented, and several large vendors were participating, and each had its interests. I knew it was doomed.