5 ms·
A couple of days ago, I tried to search on how AWS operates RDS behind the scenes, since it is a managed stateful service I was wondering whether it runs in a t
by abarrak 5y ago
A couple of days ago, I tried to search on how AWS operates RDS behind the scenes, since it is a managed stateful service I was wondering whether it runs in a traditional way VM-based or in a fully containerized environment? .. Unfortunately, a simple search will lead you to the consumer/customer resources out there only.
- rorykoehler 5y agoBased on how they bill it, it looks like it's running on VMs
- StratusBen 5y agoAgreed. AWS RDS instance types are just EC2 instance types prefixed with "db." and you're choosing either single-AZ or multi-AZ deployments so presumably AWS is just spinning up 1 to 3 EC2 instances with some preconfigured software on them.
- navaati 5y agoFrom what I know there is a secret sauce beyond a mere AMI and a control plane, based on some EBS volumes magic. I may be mixing things up with Aurora though.
- fierro 5y agothis is very, very safe to assume. There is probably a hundred engineer's worth of "secret sauce" for an entire managed DB product line.
- aeyes 5y agoThere were some comments in the early days that the Multi-AZ magic for classic RDS was just drbd on top of EBS. Aurora is a completely different approach where the RDBMS code is modified to directly interface with EBS instead of going through a traditional OS filesystem layer.
- rrdharan 5y agoAmazon published a paper describing how Aurora works: https://www.allthingsdistributed.com/files/p1041-verbitski.pdf https://www.allthingsdistributed.com/files/p1041-verbitski.p...
- redis_mlc 5y ago> There were some comments in the early days that the Multi-AZ magic for classic RDS was just drbd on top of EBS. If this was true, then you could support multiple database vendors in a generic way, and gradually add the supported products, without knowing the intricacies of each product's replication details. Hmm. :)
- ec109685 5y agoThis is a good paper that talks about Aurora and provides some insight into how RDS operates: https://www.allthingsdistributed.com/files/p1041-verbitski.pdf https://www.allthingsdistributed.com/files/p1041-verbitski.p... It’s nice that AWS builds their own higher level abstractions on the same primitives outside developers use. Feels like they eat their own dogfood much more than Google where they bypass GCP and instead utilize underlying Borg primitives for many services.
- abarrak 5y agoNice. I wonder if the stateful merits provided and marketed by containers orchestrates (e.g. K8S) is something they will consider in the future? ..
- rejectedandsad 5y agoTo build a new service at Amazon, the general path of least resistance these days is to use Lambda. If not Lambda, then ECS. If not ECS (or if it requires bare metal) then EC2.
- valenterry 5y agoYeah, except for the lambda time limit...
- rejectedandsad 5y agoGiven most things are frontended by API Gateway anyway this isn’t too big of a concern since there’s already an effective 30 second timeout there. For ETL like stuff you can use Stepfunctions too.
- valenterry 5y agoIt's still sad, because then you need a lambda for just these one or two things that might run long (or maybe even forever) and now you have to mix lambda and e.g. ECS.