5 ms·

Does RDS not do this to their customers? This alone would make me seriously consider taking my business away from GCP.

ciguy 6y ago
Absolutely not. I'm sure it's happened but I have never personally seen an RDS instance go down, in over 10 years consulting and building in AWS. Google Cloud SQL was going down for multiple minutes every month for one of my clients, and their support just said there was nothing they could do about it. That cost Google cloud at least a million dollars in revenue over the next few years as this was a fast growing startup.
- plasma 6y ago
  I’ve been a long time Azure user, their SQL Azure product was very glitchy and had constant outages (“just add retries!”), it’s gotten better in the last year or so, but it was totally unacceptable.
  - jiggawatts 6y ago
    AFAIK that's a "feature", not a bug. They throttle concurrent requests and return a different error code compared to the on-premises SQL Server. I just watched a load test hit that issue, the developers cranked up the traffic until the response times were measured in minutes. At that point Azure SQL started throwing errors related to too many concurrent queries. Compared to just locking up or timing out, it's probably better. See: https://docs.microsoft.com/en-us/azure/azure-sql/database/resource-limits-logical-server#sessions-and-workers-requests https://docs.microsoft.com/en-us/azure/azure-sql/database/re... And: https://docs.microsoft.com/en-us/azure/azure-sql/database/resource-limits-vcore-elastic-pools#general-purpose-service-tier-generation-4-compute-platform-part-1 https://docs.microsoft.com/en-us/azure/azure-sql/database/re... E.g.: the 4-CPU pool has only 210 Max concurrent workers per pool, but if you think about it, that's over 50 queries running at once per CPU core, which is nuts!
    - plasma 6y ago
      I’ve experienced that too, but these errors were literally “server is currently unavailable” and downtime. Ps: if you’re working on a production load system, and has decent traffic, lean towards premium/business critical tiers for true HA
- paulyy_y 6y ago
  Do y'all just like, ignore SLO's? Cloud SQL has a 99.95% SLO, going down for multiple minutes every month is within that. No smoke and mirrors here, there are ways to mitigate it but it's not Google messing around with expectations. HA doesn't mean 100% uptime.
  - rswail 6y ago
    Does that include or exclude "maintenance windows". AWS tells you when your maintenance window for an RDS instance will be and you can delay/reschedule as needed. I wouldn't consider a maintenance window as "downtime". That 99.95 applies to all the time outside the window.
  - byteofbits 6y ago
    It’s worth noting that the dreaded maintenance everyone is talking about here is not part of the SLA: “Downtime as part of Scheduled Maintenance will not be counted towards any Downtime Period.” - https://cloud.google.com/sql/sla https://cloud.google.com/sql/sla I’ve never seen an SLO document for CloudSQL though so the SLOs may be slightly higher internally?
  - ciguy 6y ago
    Yet AWS manages to give near 100% uptime with RDS at a similar price point. CloudSQL downtime in the cases I've seen was caused by them rebooting both master instances at the same time. This is amateurish and totally unnecessary, the whole point of having multiple masters is to the ability to do maintenance reboots in a staggered schedule. This should be a trivial problem for Google to solve and would result in much better reliability for their users, yet it's been years with no change.
ericbarrett 6y ago
I had some instances that needed to be rebooted because they were running a vulnerable MySQL version. AWS gave six months warning (before forced reboot) and ended up extending it to 12.

Absolutely not. I'm sure it's happened but I have never personally seen an RDS instance go down, in over 10 years consulting and building in AWS. Google Cloud SQL was going down for multiple minutes every month for one of my clients, and their support just said there was nothing they could do about it. That cost Google cloud at least a million dollars in revenue over the next few years as this was a fast growing startup.

plasma 6y ago
I’ve been a long time Azure user, their SQL Azure product was very glitchy and had constant outages (“just add retries!”), it’s gotten better in the last year or so, but it was totally unacceptable.
- jiggawatts 6y ago
  AFAIK that's a "feature", not a bug. They throttle concurrent requests and return a different error code compared to the on-premises SQL Server. I just watched a load test hit that issue, the developers cranked up the traffic until the response times were measured in minutes. At that point Azure SQL started throwing errors related to too many concurrent queries. Compared to just locking up or timing out, it's probably better. See: https://docs.microsoft.com/en-us/azure/azure-sql/database/resource-limits-logical-server#sessions-and-workers-requests https://docs.microsoft.com/en-us/azure/azure-sql/database/re... And: https://docs.microsoft.com/en-us/azure/azure-sql/database/resource-limits-vcore-elastic-pools#general-purpose-service-tier-generation-4-compute-platform-part-1 https://docs.microsoft.com/en-us/azure/azure-sql/database/re... E.g.: the 4-CPU pool has only 210 Max concurrent workers per pool, but if you think about it, that's over 50 queries running at once per CPU core, which is nuts!
  - plasma 6y ago
    I’ve experienced that too, but these errors were literally “server is currently unavailable” and downtime. Ps: if you’re working on a production load system, and has decent traffic, lean towards premium/business critical tiers for true HA
paulyy_y 6y ago
Do y'all just like, ignore SLO's? Cloud SQL has a 99.95% SLO, going down for multiple minutes every month is within that. No smoke and mirrors here, there are ways to mitigate it but it's not Google messing around with expectations. HA doesn't mean 100% uptime.
- rswail 6y ago
  Does that include or exclude "maintenance windows". AWS tells you when your maintenance window for an RDS instance will be and you can delay/reschedule as needed. I wouldn't consider a maintenance window as "downtime". That 99.95 applies to all the time outside the window.
- byteofbits 6y ago
  It’s worth noting that the dreaded maintenance everyone is talking about here is not part of the SLA: “Downtime as part of Scheduled Maintenance will not be counted towards any Downtime Period.” - https://cloud.google.com/sql/sla https://cloud.google.com/sql/sla I’ve never seen an SLO document for CloudSQL though so the SLOs may be slightly higher internally?
- ciguy 6y ago
  Yet AWS manages to give near 100% uptime with RDS at a similar price point. CloudSQL downtime in the cases I've seen was caused by them rebooting both master instances at the same time. This is amateurish and totally unnecessary, the whole point of having multiple masters is to the ability to do maintenance reboots in a staggered schedule. This should be a trivial problem for Google to solve and would result in much better reliability for their users, yet it's been years with no change.

ericbarrett 6y ago

I had some instances that needed to be rebooted because they were running a vulnerable MySQL version. AWS gave six months warning (before forced reboot) and ended up extending it to 12.