4 ms·
The reason for having most instances use network storage is that it makes possible migrating instances to other hosts. If the host fails, the network storage ca
by ianburrell 3y ago
The reason for having most instances use network storage is that it makes possible migrating instances to other hosts. If the host fails, the network storage can be pointed at the new host with a reboot. AWS sends out notices regularly when they are going to reboot or migrate instances.
Their probably should be more local instance storage types for using with instances that can be recreated without loss. But it is simple for them to have a single way of doing things.
At work, someone used fast NVMe instance storage for Clickhouse which is a database. It was a huge hassle to copy data when instances were going to be restarted because the data would be lost.
- mike_hearn 3y agoSure, I understand that, but this user is claiming that on GCP even local SSDs aren't really local, which raises the question of why not. I suspect the answer is something to do with their manufacturing processes/rack designs. When I worked there (pre GCP) machines had only a tiny disk used for booting and they wanted to get rid of that. Storage was handled by "diskful" machines that had dedicated trays of HDDs connected to their motherboards. If your datacenters and manufacturing processes are optimized for building machines that are either compute or storage but not both, perhaps the more normal cloud model is hard to support and that pushes you towards trying to aggregate storage even for "local" SSD or something.
- deadmutex 3y agoThe GCE claim is unverified. OP seems to be referring to PD-SSD and not LocalSSD
- rwiggins 3y agoGCE local SSDs absolutely are on the same host as the VM. The docs [0] are pretty clear on this, I think: > Local SSD disks are physically attached to the server that hosts your VM. Disclosure: I work on GCE. [0] https://cloud.google.com/compute/docs/disks/local-ssd https://cloud.google.com/compute/docs/disks/local-ssd
- flaminHotSpeedo 3y agoThey're claiming so, but they're wrong.
- youngtaff 3y ago> At work, someone used fast NVMe instance storage for Clickhouse which is a database. It was a huge hassle to copy data when instances were going to be restarted because the data would be lost. This post on how Discord RAIDed local NVMe volumes with slower remote volumes might be on interest https://discord.com/blog/how-discord-supercharges-network-disks-for-extreme-low-latency https://discord.com/blog/how-discord-supercharges-network-di...
- ianburrell 3y agoWe moved to running Clickhouse on EKS with EBS volumes for storage. It can better survive instances going down. I didn't work on it so don't how much slower it is. Lowering the management burden was big priority.
- wiredfool 3y agoAre you saying that a reboot wipes the ephemeral disks? Or a stop the instance and start the instance from AWS console/api?
- ianburrell 3y agoReboot keeps the instance storage volumes. Restarting wipes them. Starting frequently migrates to new host. And the "restart" notices AWS sends are likely cause the host has a problem and need to migrate it.