4 ms·

I really don't know why Google doesn't just let users pick the how many copies to keep, and how many writes must be ACK'ed before the VM sees a write as complet

I really don't know why Google doesn't just let users pick the how many copies to keep, and how many writes must be ACK'ed before the VM sees a write as complete. Then users can decide the cost vs reliability vs durability of data written milliseconds before an outage. Perhaps give users a web-based calculator where you can put the numbers, and see how much it would cost in $ per gb per day, the mean time to committed data loss based on historic data in years/centuries, and the typical increase in write latency (loss of performance) compared to a single replica. Then the user can decide.

kccqzy 2y ago
If you were a team working at Google internally, then yes you can pick these parameters. You can even pick between straight up replication versus Reed Solomon codes. I just don't understand why this is not exposed in their cloud offering.
- foota 2y ago
  This is replication between zones, not within a single zone.
  - londons_explore 2y ago
    The form would look like this: CREATE PERSISTENT DISK ---------------------- Replicas: W Zones to split replication across: X How many replicas must complete a write to allow the VM to continue: Y How many zones must complete a write to allow the VM to continue: Z With the above settings, you can expect approximately: Write latency 5-95%: 0.5-2.5 milliseconds Mean time to committed data loss: 37 years Mean time to failure to write: 3 years Mean time to data loss of data over 1 minute old: 18327 years. Cost: $0.18/GB/month The user would set those 4 parameters, with help text for guidance and a big red warning if you set the parameters to something insanely slow/unreliable.
    - gtirloni 2y ago
      And the customer support teams would waste countless hours debugging scenarios that the customer had no idea what they were doing. All of that with a SLO breathing down their neck.
      - londons_explore 2y ago
        Easy enough to say "You had the replication parameters set to 1 replica, so data loss was expected every 1 month on average, and now some of your data has been lost. You can try to recover data with these opensource tools, or you can delete the disk and start again. We recommend replication of at least 1.15 to get a mean time between data loss of 150 years."
- Balinares 2y ago
  > I just don't understand why this is not exposed in their cloud offering. Probably because the mental model for the underlying implementation that you are basing this idea on does not accurately reflect the actual reality of the implementation. As a very broad rule of thumb, whenever you find yourself saying, "Given <thing A>, I don't understand <thing B>," you probably want to re-examine Thing A.

kccqzy 2y ago

If you were a team working at Google internally, then yes you can pick these parameters. You can even pick between straight up replication versus Reed Solomon codes. I just don't understand why this is not exposed in their cloud offering.

foota 2y ago
This is replication between zones, not within a single zone.
- londons_explore 2y ago
  The form would look like this: CREATE PERSISTENT DISK ---------------------- Replicas: W Zones to split replication across: X How many replicas must complete a write to allow the VM to continue: Y How many zones must complete a write to allow the VM to continue: Z With the above settings, you can expect approximately: Write latency 5-95%: 0.5-2.5 milliseconds Mean time to committed data loss: 37 years Mean time to failure to write: 3 years Mean time to data loss of data over 1 minute old: 18327 years. Cost: $0.18/GB/month The user would set those 4 parameters, with help text for guidance and a big red warning if you set the parameters to something insanely slow/unreliable.
  - gtirloni 2y ago
    And the customer support teams would waste countless hours debugging scenarios that the customer had no idea what they were doing. All of that with a SLO breathing down their neck.
    - londons_explore 2y ago
      Easy enough to say "You had the replication parameters set to 1 replica, so data loss was expected every 1 month on average, and now some of your data has been lost. You can try to recover data with these opensource tools, or you can delete the disk and start again. We recommend replication of at least 1.15 to get a mean time between data loss of 150 years."
Balinares 2y ago
> I just don't understand why this is not exposed in their cloud offering. Probably because the mental model for the underlying implementation that you are basing this idea on does not accurately reflect the actual reality of the implementation. As a very broad rule of thumb, whenever you find yourself saying, "Given <thing A>, I don't understand <thing B>," you probably want to re-examine Thing A.