4 ms·
That's wishful thinking. If there's state in that virtual machine, it's probably either stored on the physical host or in a SAN. If it's in the physical host,
by olau 7y ago
That's wishful thinking.
If there's state in that virtual machine, it's probably either stored on the physical host or in a SAN. If it's in the physical host, it has to be fished out of that machine or restored from a backup. If it's a SAN, you can lose your virtual machine if the SAN goes down.
I've seen both happen.
Actually, a single machine with RAID is surprisingly stable. A provider like Hetzner can switch out a faulty disk in less than five minutes, or switch a faulty motherboard/power supply in less than half an hour.
Virtualization on top of this does not increase stability, in my experience.
Now, some cloud providers do have more sophisticated distributed systems that do not have single points of failure, and that's a completely different story.
Of course, the software itself is a source of correlated failures, so even there you should never rely on a single cloud vendor.
There's a poster here on this site who commented some months ago that he had for years rented three servers, each on a different continent, each from a different provider, and never had downtime. That's engineering.
- treis 7y ago>If there's state in that virtual machine, it's probably either stored on the physical host or in a SAN I think the point is to write your application in a way that there's not state in the VM. My VMs are disposable and in fact the way I do deployments is to spin up a new VM in DO and then assign it the floating IP for production. If things go haywire I can easily swap back the IP address to the known good VM.
- dijit 7y agoAre you using managed databases then?
- treis 7y agoI am. Mostly for the backup and other DB administration that comes with it. Also use Spaces (DO's S3 equivalent) for any assets that aren't packaged with the code like user uploads.
- aitchnyu 7y agoFuture wish: all VPSes have managed databases, object hosting and centralized logs as standard so as not to rely on local disk at all. There is a streaming backup protocol, which allows any database and queue vendors to stream to any location, and acts as a peer for fully synchronous replication. If your DO Postgres, Kafka and Mongo instances are killed, you have a Bakstream cluster in Linode which lost no data and allows you to restore in Vultr.
- brianwawok 7y agoThats not wishful thinking. I did it from day one with my startup with GKE. You can blow up any of my servers, and a new one gets spun up and dockers deployed while the glitch is load balanced around. All for only a few hours of work. GKE has given me the infrastructure of a 10 million dollar company, with me as the IT guy doing 1h or less a week of IT work. I would not be able to do this on a $35 hertzer machine.
- dijit 7y agoGKE (and GCP in general) is a special beast when it comes to this. They support online migrations of instances and it's nearly seamless. In fact, it's very commonly happening to instances, probably ones you have. But if you're running on AWS with their EC2 instances for your application, then it's not more reliable than a single chunk of hardware in my experience (there are special cases, some hardware is just crappy)
- thaumaturgy 7y agoI receive on average about one notification from DigitalOcean each month that there's a problem with the host for one of my VMs and there may be some downtime while they migrate it to a new host. Usually the downtime is about the same as a reboot, sometimes there's no downtime at all. This works in part because both Xen and KVM hypervisors (and possibly others) support live migrations, so it's altogether false that virtualization does not increase stability. Both DigitalOcean and Linode use KVM behind the scenes. So, at lower cost than rented hardware, customers get staff whose job it is to constantly monitor systems for hardware failures and deal with them proactively in a way that minimizes downtime.