9 ms·

What Happened to OpenStack?

239 points by kragniz 8y ago

enriquto 8y ago
This way of writing (like a journalist) is so annoying. I prefer much the scientific style of writing where the main idea is given in the first sentence, and suspense is avoided as much as possible.
- zrail 8y ago
  Cool. The author chose to write a first person narrative on their personal blog. What you prefer doesn’t really matter, except in so far as you can choose not read the piece.
  - yjftsjthsd-h 8y ago
    We're in a comment thread. Disliking the article, with reasons, is a valuable contribution. Besides... Someone chose to write a comment. What you prefer doesn’t really matter, except in so far as you can choose not read the comment;)
    - indigodaddy 8y ago
      I see what you did there with the missing grammar
- arvidkahl 8y ago
  The main idea is (literally) given in the first sentence: "The project is still quite healthy and is following the usual hype curve[...]".
- mnm1 8y ago
  Ironically, what you describe is the way journalists used to write before the internet. Nowadays, everyone wants to bore their readers with badly written narratives and nonsensical tangents thinking it's good writing. I hate it too and wish journalists and others would take a journalism class.
- ternaryoperator 8y ago
  The very first thing on the page tells you that it's a personal blog consisting of "online ramblings," so I think it's a trifle unfair to complain about the style.
notatoad 8y ago
>Enterprise customers. That’s a nice way of saying well-known brands who spend a lot of money on each other every year i'm definitely going to be stealing that line
justinsaccount 8y ago
Openstack got good enough for people who wanted to run their own EC2 like internal cloud, right about the time that people stopped wanting to run VMs.
privateSFacct 8y ago
My own impression. If you develop on AWS you get a supported experience for a long LONG time (see simpleDB which I used and still works even though they don't seem to market it). Same thing with old instance types. S3 etc etc. With openstack at least a year or two ago - who can seriously stay on top of what is going on there. You could develop something 3-4 years ago and getting it going on the latest open stack = total pain. What exactly open stack was also muddy - lots of ifs/buts/this 5 year old code that ran on vendor X openstack doesn't seem to run today on vendor Y. Didn't spend much time on open stack though - and I know the hype train was / is huge - (AWS killer etc). My own sense - a lot of folks freaked out about AWS and all WANTED openstack to work so they had some big gun to blow up AWS with - but they didn't seem to spend much time talking to actual customers / developers, while AWS certainly did.
- 013a 8y ago
  > see simpleDB which I used and still works even though they don't seem to market it They don't market it, and if you create an AWS account after it was deprecated in favor of Dynamo you'd basically never know it existed except for some footnotes in the Dynamo documentation referencing its predecessor. Which is fine; hats off to AWS for maintaining it for customers for so long.
kodablah 8y ago
My opinion: it grew to a gross, large set of python and apis that, when combined with multiple implementations, extensions, and company-specific customizations, made it an unmaintanable mess that was difficult to deploy and code around. So, although the author compares it with the growing k8s project out there now, at least k8s more clearly stewarded, more developer oriented instead of only for ops (with code quality to match), and doesn't feel as hamstrung by environments and dependencies (just try to run a little openstack setup on your laptop for development... very annoying for a project of such age with so many company's hands in the pot).
- gboudrias 8y ago
  I wanted to like OpenStack but at the time I felt like it didn't... Let me? It did everything complicated in a relatively straightforward manner, but I didn't want dev to be complicated. I wanted my dev simple, and I felt like they just weren't really interested in that (strangely enough). Could be wrong though. Never found a way to justify adoption to the team.
- m0zg 8y ago
  >> k8s more clearly stewarded Not clearly enough, IMO. Not very "clean" to begin with, it's accumulating cruft at an alarming pace, and doesn't drop much legacy over time (a painful, but necessary step in fighting code entropy). It seems to have inherited Google's internal modus operandi: launch new shit and then let it rot.
  - dilyevsky 8y ago
    You’re either kidding or simply not aware of how much refactoring and breaking changes are introduced each release.
  - p_l 8y ago
    It has a very clear API stability process, preventing the typical arguments about google dropping things willy-nilly. I rarely have to change anything in my manifests to get them to run after upgrading k8s
hodgesrm 8y ago
> I am publishing this now in the hope that it can serve as a warning to everyone out there who is investing in Kubernetes. You’ll never be able to run it as effectively, at the same scale, as GKE does – unless you also invest in a holistic change to your organization. This is a meaningless argument. I don't have to run Kubernetes at the same scale as GKE to develop--I just run minikube, which runs very well on Linux hosts. When I get ready to deploy there is a pick of environments to host on because Kubernetes apps are largely portable. OpenStack has never achieved this level of accessibility.
- atarian 8y ago
  I don't think that statement was directed at end-users of Kubernetes; it was directed at the people who operate Kubernetes as a service like AWS, Oracle, Cisco, or RedHat.
- mfer 8y ago
  Two things occurred to me about this... 1. I had a local OpenStack environment. Most of what I needed for app dev I could do there. 2. A lot of app devs aren't happy with Kubernetes and talk about it. Sometimes in the comments right here on Hacker News. Kubernetes has a lot of parallels when you drill down and look at it. Aeva isn't the only one talking about it.
  - yebyen 8y ago
    There weren't ever 101 different ways[1] for you to get your OpenStack environment provisioned. 101 products from 81 certified vendors. 33 completely separate, independent, certified, hosted environments, and every other different entry on the CNCF listing. With no less than 12 different ways to install it for yourself, on resources that you own one way or another. I think we're past Landmark status already, OpenStack never did all this. As a developer, I feel I am ready to go with this approach. It's my ops teams that can't cope with that degree of choice – they're apprehensive to choose, knowing that with 90+ options and almost all of them acceptable to me and my team, there's non-zero risk that we're going to choose the wrong one! We'll have to switch. And who knows why? We'll find out, if we settle on one. The operational expense for us to set up Kubernetes is already great enough. The prospect of ultimately learning that maybe we picked the wrong one, then needing to switch to another one, for them, seems too large, I think. Why not wait for the market to die down a little bit, or for that list to get just a little bit shorter first? Seems like I'll be waiting forever. If I narrow it down to only options that have been certified since K8S v1.9, maybe the choices will look a little bit more constrained. I really want to convince them. [1]: https://docs.google.com/spreadsheets/d/1LxSqBzjOxfGx3cmtZ4EbB_BGCxT_wlxW_xgHVVa23es/edit#gid=0 https://docs.google.com/spreadsheets/d/1LxSqBzjOxfGx3cmtZ4Eb...
ec109685 8y ago
“Even then, if you’re using Kubernetes, you probably won’t succeed, because it isn’t in Google’s best interest to let anyone else actually compete with GKE.” Google succeeds if it is anybody but AWS proprietary solutions. If they can groom a healthy ecosystem of open source and commercial solutions that target Kubernetes, then the tremendous advantage of AWS being a one stop shop for any service you can imagine starts to dwindle. As of now, amazon offers a solid compute environment and services galore, which is hard to compete with. The author didn’t do much at all to tie what happened to OpenStack with Kubernetes. K8s is deployed at scale by all the cloud providers. Both Google and Microsoft solely run containers on it in their public cloud (while AWS still has their own orchestrator). That never happened at scale with OpenStack. And regardless of how you feel about Google, Azure has a very strong vested interest in K8s success.
- BinaryIdiot 8y ago
  > Google succeeds if it is anybody but AWS proprietary solutions. Maybe? But don't forget Azure is bigger than Google's cloud services so they would also benefit from people not using AWS proprietary solutions and there are many smaller companies also in this space. I don't think Kubernetes necessarily helps good at all in the market place itself. Maybe with talent and admiration but almost everyone offers Kubernetes deployments now.
  - jjeaff 8y ago
    Is azure really bigger? Because if you ask azure, they claim to be bigger than AWS as well. They are obviously lumping office 360 and all of their cloud software.
    - rapsey 8y ago
      It is by revenue
    - 013a 8y ago
      And Amazon lumps WorkDocs and WorkMail and Chime into their AWS line item as well. Its the same thing. Except Amazon's just sucks, so they don't generate nearly as much revenue.
musicale 8y ago
When I used to use OpenStack, it frustrated me because the software seemed badly designed, unreliable, poorly documented, and hard to use. Perhaps having hundreds or thousands of contributors is more of a problem than a solution. Or maybe it just needed better technical oversight.
- mirceal 8y ago
  I believe that’s a bit unfair. Different Openstack components had different level of quality. The biggest problem I saw was that little to no thought was put into what the experience of an operator would be. It looked more like a playground / place to experiment and learn than something you would bet the farm on. If someone would have cared enough to holistically drive this across the whole platform I think this could have gone in a different direction.
godzillabrennus 8y ago
NephoScale was an open stack vendor I followed before they vanished. Seems like it never really got easy enough for average developers to find it accessible.
- gtirloni 8y ago
  I remember getting to work and finding out the Nebula folks had disappeared without warning [0]. On April 1st of all days. Led by Chris Kemp (@kemp). Next virtualization cluster was KVM/libvirt-based and automated with Ansible. That particular company didn't want to hear about OpenStack anymore. 0 - https://blogs.dxc.technology/2015/04/02/nebula-openstack-cloud-vendor-just-shut-down-with-no-notice/ https://blogs.dxc.technology/2015/04/02/nebula-openstack-clo...
0xbadcafebee 8y ago
I've never seen an openstack implementation that wasn't horrible. The reason more people use k8s is it's actually less useful than openstack, which makes it more opinionated, which makes its implementations more uniform. Plus, most people deploy it either on top of an openstack or other cloud platform. You should not build your own cloud platform; that much is obvious. It's less obvious that you should not build your own k8s, because it seems simpler and more useful.
FooHentai 8y ago
>When you’re looking at other cloud products, think about similar conflicts of interest that might be affecting your favorite spokespersons today… (I’m looking at you, kubernetes) See also: Banks investing in cryptocurrency R&D
- bdcravens 8y ago
  See also: Blockchain startups discussing database solutions.
SteveNuts 8y ago
OpenStack was an extremely ambitious project, which requires so much cooperation and interoperability between so many companies that I believe it was doomed from the start. In 2015 my company purchased "Flexpod" which is a solution that's certified by VMware, Cisco, and NetApp to work together. The result is nothing but a bunch of back and forth finger pointing with support, and even a critical vulnerability will take 6+ months to get patched and certified between all the different vendors. I personally like the Ansible approach where each Storage/Computer/Network vendor provides APIs for management of their devices, and Ansible is just the glue between them. TL;DR getting major tech vendors to play nice together is hard.
- ec109685 8y ago
  K8s is going in that direction. Providing an API for storage providers to implement and letting them drive the implementation versus trying to offer a monolithic all batteries included solution.
- webmaven 8y ago
  > OpenStack was an extremely ambitious project, which requires so much cooperation and interoperability between so many companies that I believe it was doomed from the start. OpenStack was doomed from the start, but the reasons were subtly different than the difficulties of integrating software from multiple vendors. However difficult OpenStack was to get (and keep) running, it would have been worth it if, once you (or anyone else) got an OpenStack instance up and running, a developer could migrate their app (from or to any other OpenStack cloud) with no code or configuration changes. That was never really possible, since every OpenStack-based cloud provider insisted on adding their own special sauce to the developer experience. That was, for most of them, the whole point of participating in OpenStack: Sharing the cost of developing the code necessary for building a public or private cloud, but locking in their customers just as firmly as AWS was doing. As a result, none of those individual cloud providers ever got big enough to give AWS serious competition, and there wasn't never a realistic portability story that could give them collective weight in the market. The simplest way to ensure developed and deployed applications and services were truly (and trivially) portable between the different OpenStack cloud providers would have been to commit OpenStack to API-compatibility with AWS (which would also have given AWS customers a clear migration path to OpenStack), but this suggestion was rejected outright at the outset of the project. Having real portability between different OpenStack-based cloud providers and AWS would mean that competition would largely have been on price (and quality/reliability) rather than features, both lowering AWS' margins and growing the market faster than Amazon could capture, as well as enabling higher-level businesses such as marketplaces with dynamic spot-pricing and rapid migration of jobs between providers. Unfortunately, that isn't how it shook out. None of the cloud companies behind OpenStack wanted to compete with AWS on price instead of features, which meant they stuck their heads in the sand, kept their attractive margins, but effectively ceded the bulk of the market to Amazon, keeping prices high enough that the market grew only about as fast as Amazon/MS/Google could collectively add capacity (roughly maintaining AWS marketshare), rather than the hockey-stick growth that would have happened if everyone could have got in on the act (like web-hosting did in the 90s).
vkat 8y ago
I think the battle for running virtual machines in the enterprise was already won by vmware and so OpenStack became a niche for service providers wanting to do Nfvi workloads and had resources to afford dedicated teams to running OpenStack. Kubernetes on the other hand capitalized on the need for running and orchestrating containers. Kubernetes also got a few things that right such as well documented and prescriptive set of tools one could use to get dev and production cluster up. On a separate note, having worked on OpenStack I can also attest that the code was gross, not so much in kubernetes.
- dfox 8y ago
  For me the reason why VMWare won is outlined in the article. Case in point: running OpenStack with shared storage on SAN is major PITA while for VMWare it is the recommended way to run things.
djsumdog 8y ago
I think the question is not "What happened to OpenStack" but rather, "Is OpenStack still total garbage?" I've been at two companies that attempted to go down the OpenStack route. One wanted to start a cloud offering to their clients and hemorrhaged tons of money trying to just keep OpenStack stable. We couldn't even run our basic Logstash offerings on our OpenStack cluster without them having bizarre performance issues. We had a really good manager too who had accounts on every other provider (Rackspace, RedHat, Canonical .. all the big ones) and time and time again he was like, "What is this? How are they doing this.." and we just figured they used a ton of specialized proprietary plugins they just weren't open sourcing or a ton of special patch sets. Second shop had tried moving onto an OpenStack cluster to save on AWS prices. It could never run anything reliably and they scrapped the entire project and re-purposed all the servers for DC/OS, which was super nice and reliable and every team migrate hundreds of services onto.
- acje 8y ago
  To me the problem with openstack is a strategic one. The Project tries to automate status qou putting the compatibility burden on the automation. PaaS alternative like Kubernetes on the other hand turns this on its head and puts the burden on the komponents of the applications. This is a much better long term strategy for keeping komplexity curbed. Also openstack was open for fake vendor openness. Where vendors could make compatible api with extensions. This doesn’t help the system integrator in the long run.
- naikrovek 8y ago
  Interesting that you cite stability concerns. I am not sure that's the case anymore. My employer runs 5-6 complete openstack environments and those things have never had an unplanned outage that I'm aware of. My stuff hasn't ever gone down, I know that.
  - ps 8y ago
    Similar experience here. Back in 2013 we had to evaluate existing cloud/VM platforms in order to replace the plain KVM/libvirt and support and enable the growth. oVirt was garbage (missing installation ISOs, randomly broken install process, cluster nodes not communicating etc.), OpenNebula buggy, OpenStack seemed to be quite hard to grasp, Hyper-V Windows only and VMWare expensive as hell (even now the TCO Calculator gives us 4000+ EUR/VM - this must be joke). We run several VMs with docker and our apps, manage dedicated servers and their networks (VLANs as provider networks in OpenStack), provide IPSec VPNs to for tenants, run Kubernetes clusters on OpenStack. We also manage several dedicated servers that are not managed by OpenStack for historical reasons and hopefully will migrate them to the cloud. If OpenStack makes our heads hurt it is due to lack of documented design patterns. After the years, documentation is good for the initial deployment and IMHO for developers (either API consumers or contributors), but no so much for network engineers or system architects. Some design choices are pretty crucial upfront and you will pay the price to change the design. We ended up modifying database records several times and then slowly rolled the changes to the compute nodes. Recently the OpenVSwitch flow tables were populated undeterministically after some network changes and we had to inspect sources and even then did not understand, why do we experience the issues. But never did we encounter the stability issues, that were not caused by wild actions of an administrator.
RyanShook 8y ago
As an outsider, the concept of openstack was always really appealing but it felt like RackSpace’s answer to AWS. Never really learned more about it than what was on their website but that was always the impression I got.
yingw787 8y ago
I worked a little bit with OpenStack about four years ago, and my impression was that it was very design by committee. Design by committee doesn't work too well in software: https://sourcemaking.com/antipatterns/design-by-committee https://sourcemaking.com/antipatterns/design-by-committee I think a lot of the enterprise companies supporting OpenStack, like Mirantis (https://en.wikipedia.org/wiki/Mirantis https://en.wikipedia.org/wiki/Mirantis), realized this one way or another, got themselves acquired, and then used the new funding to pivot to Kubernetes or another open-source IaaS offering: https://www.mirantis.com/ https://www.mirantis.com/ Without any promise of enterprise support, there's really no way for the large companies targeted by OpenStack to adopt it and make that adoption sticky. So that's how it died.
- pas 8y ago
  RedHat and Canonical and RackSpace all offer(ed) enterprise support, no?
  - p_l 8y ago
    The practical end result was that you didn't buy "OpenStack", you bought "Mirantis OpenStack" or "Juniper Openstack" (... that one was so broken...) and so on, and there wasn't much portability between them.
    - pas 8y ago
      The basics worked across all distributions, as far as I know. (The openstack-cli which was built on the HTTP API, which was shared.) Mostly I had problems with the classic deployment, debug, develop cycle. Reporting bugs is like throwing time out of the window, debugging through overlay networks, über verbose python daemon logs and RabbitMQ madness was also more of a surreally dark exercise in futility, than rewarding experience. Most of the problems I experienced were problems due to the fundamental trade offs taken during the deveopment of OpenStack. And these are slowly addressed, but ... it was too little too late - at least in our case.
nwrk 8y ago
Luckily there is also OpenStack running on top of Docker... https://github.com/openstack/kolla https://github.com/openstack/kolla
- Sylamore 8y ago
  That's basically how AT&T is making openstack work: https://www.sdxcentral.com/articles/news/att-5g-airship-plans-powered-by-mirantis/2019/02/ https://www.sdxcentral.com/articles/news/att-5g-airship-plan...
- sekh60 8y ago
  Just a home labber here. I use Kolla for my openstack environment and it's been really easy to install and manage.
- p_l 8y ago
  Running openstack inside docker using k8s is how SAP managed to waive many pains of openstack installation/maintenance
mschuster91 8y ago
Funny enough I'm deploying OpenStack right now for us as an internal playground. It's decent - but hell, the learning curve is nasty and the documentation is incomplete. Many things I could only get working after asking on IRC and waiting hours for a reply. But still, it's better than having to manage KVM by hand and cheaper than buying VMware.
timeu 8y ago
We are working on a project where we plan to manage our new HPC system with OpenStack. This will replace 3 legacy HPC systems that were manged with propriatory management systems. Due to shifting requirements by our customers (scientist) we decided to move to a cloud framework where probably still the majority of resources is dedidcated to a batch scheduling system but it would allow us to also provide more cloud like services (jupyerhub, rstudio, databases, etc) It's quite an ambitious project and we (4 engineers) basically spent the last year understanding the in and outs of OpenStack. We also went full in with integrating all kinds of datacenter components into OS (NetApp, SDN, DNS, etc) Some lessons learned so far: - OpenStack is very complex - It's less of a product and more a framework and you need a dedicated engineering team with cross cutting skillset - You definately need a dev/staging environment to test upgrades and customizations - Some of the reference implementations of OS servies (SDN) are fine for small deployments but if can replace them with dedicated hardware/appliances you should do that.
7ewis 8y ago
I've been using AWS for years, and had never heard of OpenStack until this year. The only reason I'm aware of it, is because I'm studying for a degree part time - and OpenStack is taught in one of the modules. It's a shame really that they only mention AWS and advise against using it in case you accidentally spend money.

239 points by kragniz 8y ago

This way of writing (like a journalist) is so annoying. I prefer much the scientific style of writing where the main idea is given in the first sentence, and suspense is avoided as much as possible.

zrail 8y ago
Cool. The author chose to write a first person narrative on their personal blog. What you prefer doesn’t really matter, except in so far as you can choose not read the piece.
- yjftsjthsd-h 8y ago
  We're in a comment thread. Disliking the article, with reasons, is a valuable contribution. Besides... Someone chose to write a comment. What you prefer doesn’t really matter, except in so far as you can choose not read the comment;)
  - indigodaddy 8y ago
    I see what you did there with the missing grammar
arvidkahl 8y ago
The main idea is (literally) given in the first sentence: "The project is still quite healthy and is following the usual hype curve[...]".
mnm1 8y ago
Ironically, what you describe is the way journalists used to write before the internet. Nowadays, everyone wants to bore their readers with badly written narratives and nonsensical tangents thinking it's good writing. I hate it too and wish journalists and others would take a journalism class.
ternaryoperator 8y ago
The very first thing on the page tells you that it's a personal blog consisting of "online ramblings," so I think it's a trifle unfair to complain about the style.

notatoad 8y ago

>Enterprise customers. That’s a nice way of saying well-known brands who spend a lot of money on each other every year i'm definitely going to be stealing that line

justinsaccount 8y ago

Openstack got good enough for people who wanted to run their own EC2 like internal cloud, right about the time that people stopped wanting to run VMs.

privateSFacct 8y ago

My own impression. If you develop on AWS you get a supported experience for a long LONG time (see simpleDB which I used and still works even though they don't seem to market it). Same thing with old instance types. S3 etc etc. With openstack at least a year or two ago - who can seriously stay on top of what is going on there. You could develop something 3-4 years ago and getting it going on the latest open stack = total pain. What exactly open stack was also muddy - lots of ifs/buts/this 5 year old code that ran on vendor X openstack doesn't seem to run today on vendor Y. Didn't spend much time on open stack though - and I know the hype train was / is huge - (AWS killer etc). My own sense - a lot of folks freaked out about AWS and all WANTED openstack to work so they had some big gun to blow up AWS with - but they didn't seem to spend much time talking to actual customers / developers, while AWS certainly did.

013a 8y ago
> see simpleDB which I used and still works even though they don't seem to market it They don't market it, and if you create an AWS account after it was deprecated in favor of Dynamo you'd basically never know it existed except for some footnotes in the Dynamo documentation referencing its predecessor. Which is fine; hats off to AWS for maintaining it for customers for so long.

kodablah 8y ago

My opinion: it grew to a gross, large set of python and apis that, when combined with multiple implementations, extensions, and company-specific customizations, made it an unmaintanable mess that was difficult to deploy and code around. So, although the author compares it with the growing k8s project out there now, at least k8s more clearly stewarded, more developer oriented instead of only for ops (with code quality to match), and doesn't feel as hamstrung by environments and dependencies (just try to run a little openstack setup on your laptop for development... very annoying for a project of such age with so many company's hands in the pot).

gboudrias 8y ago
I wanted to like OpenStack but at the time I felt like it didn't... Let me? It did everything complicated in a relatively straightforward manner, but I didn't want dev to be complicated. I wanted my dev simple, and I felt like they just weren't really interested in that (strangely enough). Could be wrong though. Never found a way to justify adoption to the team.
m0zg 8y ago
>> k8s more clearly stewarded Not clearly enough, IMO. Not very "clean" to begin with, it's accumulating cruft at an alarming pace, and doesn't drop much legacy over time (a painful, but necessary step in fighting code entropy). It seems to have inherited Google's internal modus operandi: launch new shit and then let it rot.
- dilyevsky 8y ago
  You’re either kidding or simply not aware of how much refactoring and breaking changes are introduced each release.
- p_l 8y ago
  It has a very clear API stability process, preventing the typical arguments about google dropping things willy-nilly. I rarely have to change anything in my manifests to get them to run after upgrading k8s

hodgesrm 8y ago

> I am publishing this now in the hope that it can serve as a warning to everyone out there who is investing in Kubernetes. You’ll never be able to run it as effectively, at the same scale, as GKE does – unless you also invest in a holistic change to your organization. This is a meaningless argument. I don't have to run Kubernetes at the same scale as GKE to develop--I just run minikube, which runs very well on Linux hosts. When I get ready to deploy there is a pick of environments to host on because Kubernetes apps are largely portable. OpenStack has never achieved this level of accessibility.

atarian 8y ago
I don't think that statement was directed at end-users of Kubernetes; it was directed at the people who operate Kubernetes as a service like AWS, Oracle, Cisco, or RedHat.
mfer 8y ago
Two things occurred to me about this... 1. I had a local OpenStack environment. Most of what I needed for app dev I could do there. 2. A lot of app devs aren't happy with Kubernetes and talk about it. Sometimes in the comments right here on Hacker News. Kubernetes has a lot of parallels when you drill down and look at it. Aeva isn't the only one talking about it.
- yebyen 8y ago
  There weren't ever 101 different ways[1] for you to get your OpenStack environment provisioned. 101 products from 81 certified vendors. 33 completely separate, independent, certified, hosted environments, and every other different entry on the CNCF listing. With no less than 12 different ways to install it for yourself, on resources that you own one way or another. I think we're past Landmark status already, OpenStack never did all this. As a developer, I feel I am ready to go with this approach. It's my ops teams that can't cope with that degree of choice – they're apprehensive to choose, knowing that with 90+ options and almost all of them acceptable to me and my team, there's non-zero risk that we're going to choose the wrong one! We'll have to switch. And who knows why? We'll find out, if we settle on one. The operational expense for us to set up Kubernetes is already great enough. The prospect of ultimately learning that maybe we picked the wrong one, then needing to switch to another one, for them, seems too large, I think. Why not wait for the market to die down a little bit, or for that list to get just a little bit shorter first? Seems like I'll be waiting forever. If I narrow it down to only options that have been certified since K8S v1.9, maybe the choices will look a little bit more constrained. I really want to convince them. [1]: https://docs.google.com/spreadsheets/d/1LxSqBzjOxfGx3cmtZ4EbB_BGCxT_wlxW_xgHVVa23es/edit#gid=0 https://docs.google.com/spreadsheets/d/1LxSqBzjOxfGx3cmtZ4Eb...

ec109685 8y ago

“Even then, if you’re using Kubernetes, you probably won’t succeed, because it isn’t in Google’s best interest to let anyone else actually compete with GKE.” Google succeeds if it is anybody but AWS proprietary solutions. If they can groom a healthy ecosystem of open source and commercial solutions that target Kubernetes, then the tremendous advantage of AWS being a one stop shop for any service you can imagine starts to dwindle. As of now, amazon offers a solid compute environment and services galore, which is hard to compete with. The author didn’t do much at all to tie what happened to OpenStack with Kubernetes. K8s is deployed at scale by all the cloud providers. Both Google and Microsoft solely run containers on it in their public cloud (while AWS still has their own orchestrator). That never happened at scale with OpenStack. And regardless of how you feel about Google, Azure has a very strong vested interest in K8s success.

BinaryIdiot 8y ago
> Google succeeds if it is anybody but AWS proprietary solutions. Maybe? But don't forget Azure is bigger than Google's cloud services so they would also benefit from people not using AWS proprietary solutions and there are many smaller companies also in this space. I don't think Kubernetes necessarily helps good at all in the market place itself. Maybe with talent and admiration but almost everyone offers Kubernetes deployments now.
- jjeaff 8y ago
  Is azure really bigger? Because if you ask azure, they claim to be bigger than AWS as well. They are obviously lumping office 360 and all of their cloud software.
  - rapsey 8y ago
    It is by revenue
  - 013a 8y ago
    And Amazon lumps WorkDocs and WorkMail and Chime into their AWS line item as well. Its the same thing. Except Amazon's just sucks, so they don't generate nearly as much revenue.

musicale 8y ago

When I used to use OpenStack, it frustrated me because the software seemed badly designed, unreliable, poorly documented, and hard to use. Perhaps having hundreds or thousands of contributors is more of a problem than a solution. Or maybe it just needed better technical oversight.

mirceal 8y ago
I believe that’s a bit unfair. Different Openstack components had different level of quality. The biggest problem I saw was that little to no thought was put into what the experience of an operator would be. It looked more like a playground / place to experiment and learn than something you would bet the farm on. If someone would have cared enough to holistically drive this across the whole platform I think this could have gone in a different direction.

godzillabrennus 8y ago

NephoScale was an open stack vendor I followed before they vanished. Seems like it never really got easy enough for average developers to find it accessible.

gtirloni 8y ago
I remember getting to work and finding out the Nebula folks had disappeared without warning [0]. On April 1st of all days. Led by Chris Kemp (@kemp). Next virtualization cluster was KVM/libvirt-based and automated with Ansible. That particular company didn't want to hear about OpenStack anymore. 0 - https://blogs.dxc.technology/2015/04/02/nebula-openstack-cloud-vendor-just-shut-down-with-no-notice/ https://blogs.dxc.technology/2015/04/02/nebula-openstack-clo...

0xbadcafebee 8y ago

I've never seen an openstack implementation that wasn't horrible. The reason more people use k8s is it's actually less useful than openstack, which makes it more opinionated, which makes its implementations more uniform. Plus, most people deploy it either on top of an openstack or other cloud platform. You should not build your own cloud platform; that much is obvious. It's less obvious that you should not build your own k8s, because it seems simpler and more useful.

FooHentai 8y ago

>When you’re looking at other cloud products, think about similar conflicts of interest that might be affecting your favorite spokespersons today… (I’m looking at you, kubernetes) See also: Banks investing in cryptocurrency R&D

bdcravens 8y ago
See also: Blockchain startups discussing database solutions.

SteveNuts 8y ago

OpenStack was an extremely ambitious project, which requires so much cooperation and interoperability between so many companies that I believe it was doomed from the start. In 2015 my company purchased "Flexpod" which is a solution that's certified by VMware, Cisco, and NetApp to work together. The result is nothing but a bunch of back and forth finger pointing with support, and even a critical vulnerability will take 6+ months to get patched and certified between all the different vendors. I personally like the Ansible approach where each Storage/Computer/Network vendor provides APIs for management of their devices, and Ansible is just the glue between them. TL;DR getting major tech vendors to play nice together is hard.

ec109685 8y ago
K8s is going in that direction. Providing an API for storage providers to implement and letting them drive the implementation versus trying to offer a monolithic all batteries included solution.
webmaven 8y ago
> OpenStack was an extremely ambitious project, which requires so much cooperation and interoperability between so many companies that I believe it was doomed from the start. OpenStack was doomed from the start, but the reasons were subtly different than the difficulties of integrating software from multiple vendors. However difficult OpenStack was to get (and keep) running, it would have been worth it if, once you (or anyone else) got an OpenStack instance up and running, a developer could migrate their app (from or to any other OpenStack cloud) with no code or configuration changes. That was never really possible, since every OpenStack-based cloud provider insisted on adding their own special sauce to the developer experience. That was, for most of them, the whole point of participating in OpenStack: Sharing the cost of developing the code necessary for building a public or private cloud, but locking in their customers just as firmly as AWS was doing. As a result, none of those individual cloud providers ever got big enough to give AWS serious competition, and there wasn't never a realistic portability story that could give them collective weight in the market. The simplest way to ensure developed and deployed applications and services were truly (and trivially) portable between the different OpenStack cloud providers would have been to commit OpenStack to API-compatibility with AWS (which would also have given AWS customers a clear migration path to OpenStack), but this suggestion was rejected outright at the outset of the project. Having real portability between different OpenStack-based cloud providers and AWS would mean that competition would largely have been on price (and quality/reliability) rather than features, both lowering AWS' margins and growing the market faster than Amazon could capture, as well as enabling higher-level businesses such as marketplaces with dynamic spot-pricing and rapid migration of jobs between providers. Unfortunately, that isn't how it shook out. None of the cloud companies behind OpenStack wanted to compete with AWS on price instead of features, which meant they stuck their heads in the sand, kept their attractive margins, but effectively ceded the bulk of the market to Amazon, keeping prices high enough that the market grew only about as fast as Amazon/MS/Google could collectively add capacity (roughly maintaining AWS marketshare), rather than the hockey-stick growth that would have happened if everyone could have got in on the act (like web-hosting did in the 90s).

vkat 8y ago

I think the battle for running virtual machines in the enterprise was already won by vmware and so OpenStack became a niche for service providers wanting to do Nfvi workloads and had resources to afford dedicated teams to running OpenStack. Kubernetes on the other hand capitalized on the need for running and orchestrating containers. Kubernetes also got a few things that right such as well documented and prescriptive set of tools one could use to get dev and production cluster up. On a separate note, having worked on OpenStack I can also attest that the code was gross, not so much in kubernetes.

dfox 8y ago
For me the reason why VMWare won is outlined in the article. Case in point: running OpenStack with shared storage on SAN is major PITA while for VMWare it is the recommended way to run things.

djsumdog 8y ago

I think the question is not "What happened to OpenStack" but rather, "Is OpenStack still total garbage?" I've been at two companies that attempted to go down the OpenStack route. One wanted to start a cloud offering to their clients and hemorrhaged tons of money trying to just keep OpenStack stable. We couldn't even run our basic Logstash offerings on our OpenStack cluster without them having bizarre performance issues. We had a really good manager too who had accounts on every other provider (Rackspace, RedHat, Canonical .. all the big ones) and time and time again he was like, "What is this? How are they doing this.." and we just figured they used a ton of specialized proprietary plugins they just weren't open sourcing or a ton of special patch sets. Second shop had tried moving onto an OpenStack cluster to save on AWS prices. It could never run anything reliably and they scrapped the entire project and re-purposed all the servers for DC/OS, which was super nice and reliable and every team migrate hundreds of services onto.

acje 8y ago
To me the problem with openstack is a strategic one. The Project tries to automate status qou putting the compatibility burden on the automation. PaaS alternative like Kubernetes on the other hand turns this on its head and puts the burden on the komponents of the applications. This is a much better long term strategy for keeping komplexity curbed. Also openstack was open for fake vendor openness. Where vendors could make compatible api with extensions. This doesn’t help the system integrator in the long run.
naikrovek 8y ago
Interesting that you cite stability concerns. I am not sure that's the case anymore. My employer runs 5-6 complete openstack environments and those things have never had an unplanned outage that I'm aware of. My stuff hasn't ever gone down, I know that.
- ps 8y ago
  Similar experience here. Back in 2013 we had to evaluate existing cloud/VM platforms in order to replace the plain KVM/libvirt and support and enable the growth. oVirt was garbage (missing installation ISOs, randomly broken install process, cluster nodes not communicating etc.), OpenNebula buggy, OpenStack seemed to be quite hard to grasp, Hyper-V Windows only and VMWare expensive as hell (even now the TCO Calculator gives us 4000+ EUR/VM - this must be joke). We run several VMs with docker and our apps, manage dedicated servers and their networks (VLANs as provider networks in OpenStack), provide IPSec VPNs to for tenants, run Kubernetes clusters on OpenStack. We also manage several dedicated servers that are not managed by OpenStack for historical reasons and hopefully will migrate them to the cloud. If OpenStack makes our heads hurt it is due to lack of documented design patterns. After the years, documentation is good for the initial deployment and IMHO for developers (either API consumers or contributors), but no so much for network engineers or system architects. Some design choices are pretty crucial upfront and you will pay the price to change the design. We ended up modifying database records several times and then slowly rolled the changes to the compute nodes. Recently the OpenVSwitch flow tables were populated undeterministically after some network changes and we had to inspect sources and even then did not understand, why do we experience the issues. But never did we encounter the stability issues, that were not caused by wild actions of an administrator.

RyanShook 8y ago

As an outsider, the concept of openstack was always really appealing but it felt like RackSpace’s answer to AWS. Never really learned more about it than what was on their website but that was always the impression I got.

yingw787 8y ago

I worked a little bit with OpenStack about four years ago, and my impression was that it was very design by committee. Design by committee doesn't work too well in software: https://sourcemaking.com/antipatterns/design-by-committee https://sourcemaking.com/antipatterns/design-by-committee I think a lot of the enterprise companies supporting OpenStack, like Mirantis (https://en.wikipedia.org/wiki/Mirantis https://en.wikipedia.org/wiki/Mirantis), realized this one way or another, got themselves acquired, and then used the new funding to pivot to Kubernetes or another open-source IaaS offering: https://www.mirantis.com/ https://www.mirantis.com/ Without any promise of enterprise support, there's really no way for the large companies targeted by OpenStack to adopt it and make that adoption sticky. So that's how it died.

pas 8y ago
RedHat and Canonical and RackSpace all offer(ed) enterprise support, no?
- p_l 8y ago
  The practical end result was that you didn't buy "OpenStack", you bought "Mirantis OpenStack" or "Juniper Openstack" (... that one was so broken...) and so on, and there wasn't much portability between them.
  - pas 8y ago
    The basics worked across all distributions, as far as I know. (The openstack-cli which was built on the HTTP API, which was shared.) Mostly I had problems with the classic deployment, debug, develop cycle. Reporting bugs is like throwing time out of the window, debugging through overlay networks, über verbose python daemon logs and RabbitMQ madness was also more of a surreally dark exercise in futility, than rewarding experience. Most of the problems I experienced were problems due to the fundamental trade offs taken during the deveopment of OpenStack. And these are slowly addressed, but ... it was too little too late - at least in our case.

nwrk 8y ago

Luckily there is also OpenStack running on top of Docker... https://github.com/openstack/kolla https://github.com/openstack/kolla

Sylamore 8y ago
That's basically how AT&T is making openstack work: https://www.sdxcentral.com/articles/news/att-5g-airship-plans-powered-by-mirantis/2019/02/ https://www.sdxcentral.com/articles/news/att-5g-airship-plan...
sekh60 8y ago
Just a home labber here. I use Kolla for my openstack environment and it's been really easy to install and manage.
p_l 8y ago
Running openstack inside docker using k8s is how SAP managed to waive many pains of openstack installation/maintenance

mschuster91 8y ago

Funny enough I'm deploying OpenStack right now for us as an internal playground. It's decent - but hell, the learning curve is nasty and the documentation is incomplete. Many things I could only get working after asking on IRC and waiting hours for a reply. But still, it's better than having to manage KVM by hand and cheaper than buying VMware.

timeu 8y ago

We are working on a project where we plan to manage our new HPC system with OpenStack. This will replace 3 legacy HPC systems that were manged with propriatory management systems. Due to shifting requirements by our customers (scientist) we decided to move to a cloud framework where probably still the majority of resources is dedidcated to a batch scheduling system but it would allow us to also provide more cloud like services (jupyerhub, rstudio, databases, etc) It's quite an ambitious project and we (4 engineers) basically spent the last year understanding the in and outs of OpenStack. We also went full in with integrating all kinds of datacenter components into OS (NetApp, SDN, DNS, etc) Some lessons learned so far: - OpenStack is very complex - It's less of a product and more a framework and you need a dedicated engineering team with cross cutting skillset - You definately need a dev/staging environment to test upgrades and customizations - Some of the reference implementations of OS servies (SDN) are fine for small deployments but if can replace them with dedicated hardware/appliances you should do that.

7ewis 8y ago

I've been using AWS for years, and had never heard of OpenStack until this year. The only reason I'm aware of it, is because I'm studying for a degree part time - and OpenStack is taught in one of the modules. It's a shame really that they only mention AWS and advise against using it in case you accidentally spend money.