4 ms·

Not sure what do you mean about scaling storage layer separately from the compute layer. It's how it's usually done currently, with the disk being the storage l

• actuator 8y ago
  > Not sure what do you mean about scaling storage layer separately from the compute layer. It's how it's usually done currently, with the disk being the storage layer. Yeah, I meant that only. DynamoDB and Aurora Serverless are examples where they are decoupled and can be scaled independently. So, I wanted to know about more products like them but ones that scale with load really well. > AWS now has the Aurora Serverless, though I don't have any real-life experience with it's latency yet (VPC requirement is a downer for now). I have tried Aurora Serverless and I like it but it also plagued by the provisioning problems for now. The lack of options to customize database parameters was kind of a downer. Also for some weird reason they only support MySQL in t type instances. > (VPC requirement is a downer for now) BTW, you have an use case where you need to use DB from outside VPC and you can't do peering?
  • anothergoogler 8y ago
    I don't understand what you're looking for here. A network file system? Every database decouples storage from "compute."
    • actuator 8y ago
      Yeah, but not all of them allow you to scale them independently without a downtime. For example in traditional MySQL setups to scale up the compute capacity your server has to be unavailable for some time.
      • rhizome 8y ago
        I think people way overestimate the business effects of maintenance windows.
        • etaioinshrdlu 8y ago
          Disagree completely. Services that you can mostly depend on to never go down, and scale as far as you will ever want, are a huge relief. Like S3. We should ask for the same thing with rdbms's.
          • rhizome 8y ago
            The overestimation is in whether a company should be using s3 as an operational standard.
        • actuator 8y ago
          If you take them once a quarter/year, they are fine. But if you have load patterns where you want to pay for what you use and not over provision the DB significantly then you can't rely on maintenance windows. I think we will get there eventually with even RDBMS getting solutions like Aurora Serverless but right now most of them don't work well.
      • nemothekid 8y ago
        Google Cloud's BigTable is like this. I don't remember how long scaling up took but instead of telling it how many "IOPS" to provision, you tell BigTable how many compute nodes you want. Once you hit ~80% on a node, the latency would rise, until you added another compute node. Since storage/compute was decoupled you wouldn't have to "wait" for a rebalance either.
  • bni 8y ago
    > BTW, you have an use case where you need to use DB from outside VPC and you can't do peering? Lambda in VPC have unacceptable cold-start for interactive applications. This is partly why DynamoDB use is so popular with Lambda, it does not have to be in VPC.