As a small competitor to Racksapce in 2010, designing a replacement for our (then) 8-year old VM structure, Bytemark bet against Openstack from the 1st announc

As a small competitor to Racksapce in 2010, designing a replacement for our (then) 8-year old VM structure, Bytemark bet against Openstack from the 1st announcement and spent years building something the-same-but-different. It wasn't _just_ that Openstack was unreleased and talking about foundations and making grand promises without any production-grade code. It was just that we could never work out which version Rackspace was actually running for their actual customers, and the details of how to actually _deploy_ it and make it work seemed enormously fiddly compared to VMWare, which was where they were taking aim. I expected them to say "now running on Openstack, 100%!" Also the design seemed to mirror AWS, as if the only answer to their dominance was to ... copy every facility they were producing exactly? We had a more definite vision of where VMs should go, and thought it was a bad plan to aim at "Amazon, but smaller!". In particular we really really wanted live migrations to be a part of our platform - where we really really cared about uptime of individual VMs, and wanted people to be able to upgrade them on the fly. Plus, y'know, for a hosting company, being in control of, and having opinions of our hosting platform was what people paid us for! So we designed our in-house platform BigV instead (now Bytemark Cloud Servers) -> https://blog.bytemark.co.uk/wp-content/uploads/2012/12/DesignAndImplementationOfBigV.pdf https://blog.bytemark.co.uk/wp-content/uploads/2012/12/Desig... [pdf] And even though we (re)invented an NBD server to make all the live migration stuff work https://github.com/BytemarkHosting/flexnbd-c https://github.com/BytemarkHosting/flexnbd-c I believe we've ended up with something that does a small number of things far better. I see people asking whether Openstack is actually running anywhere very public, and have read some pained war stories. So it still feels like the right decision, even before this announcement. Plus it seems like live migration in Openstack is still an exotic and difficult option, whereas that's been our standard practice for years. There are plenty of problems running your own hosting stack, but I like that we can solve our own problems on our own terms, and tie software decisions to hardware or data centre decisions that we make at the same time. I eye up Kubernetes & Ganeti every few months, but they keep reminding me that if you've got the expertise, and a specific purpose in mind, you can usually build something that fits your purpose more closely. (though the second half of that sentence will probably be written on my self-carved tombstone)