4 ms·
Now I really want to know though.
by xandrius 1y ago
Now I really want to know though.
- cperciva 1y agoMy understanding is that EBS has some heuristics for deciding whether to keep data cached; an AMI which has a cached snapshot as its root disk will boot much faster than an AMI where all the data needs to be pulled from S3.
- tedunangst 1y agoSome huge customer chunked their data into 5GB pieces so now there's a "if size == 5GB" in the cache code.
- cperciva 1y agoMaybe, but I don't think that would explain 8 GB also being fast while 6 GB is slow?
- 0x457 1y agoCustomer started using 8GB chunks /s
- deleted 1y ago[deleted]
- MobiusHorizons 1y agoYeah, I found that pretty unintuitive when I read it. How did you find 8GB worked? Trial and error?
- deleted 1y ago[deleted]
- JoshTriplett 1y agoWhat's the smallest size for which those heuristics keep the snapshot cached? (I'm currently using 1GB snapshots, because my actual disk image is a tiny fraction of that size. But if bumping that to 2GB or 4GB would make it faster, that's a small price to pay.)
- cperciva 1y agoI believe 1 GB is also fast.
- JoshTriplett 1y agoThanks, that helps to hear! Do you have any other wisdom regarding mysterious reasons for fast or slow booting? EC2's boot process is deeply opaque, and any insight at all is better than nothing.
- cperciva 1y agoNothing comes to mind, but if you want to drop me an email I can walk you through some benchmarking.
- richardwhiuk 1y agoAt a guess, powers of 2 are fast?
- avidphantasm 1y agoYeah, I am constantly curious about how the sausage that is cloud services like AWS is made. It seems generally slick on the surface, but what’s holding it all together? I imagine it as a tangled ball of tools like Puppet, Chef, etc. and custom glue.
- akdev1l 1y agoAt Amazon scale mostly everything is custom Less puppet/chef
- avidphantasm 1y agoYeah, I would imagine they maybe started with off-the-shelf tools that were then gradually replaced as the system grew and matured.
- akdev1l 1y agoKind of the opposite, I think AWS was the first hyper scaler so tooling did not exist for many of these problems back then Like they have their own custom clustering software where you would probably use k8s if you were to rebuild things today Repeat this over a million different tools, etc This article is interesting if you want to take a peek behind the curtain: https://www.allthingsdistributed.com/2014/11/apollo-amazon-deployment-engine.html https://www.allthingsdistributed.com/2014/11/apollo-amazon-d...
- arcfour 1y agoA lot of AWS services are built on other AWS services. Like Lambda, SQS, and other such "core services" are used by others under the hood.