5 ms·
The root of this is definitely GCS. We noticed some keys in a GCS bucket go inaccessible since like 19:15-19:20ish PDT, and then we noticed increased timeouts/5
by jhgg 8y ago
The root of this is definitely GCS. We noticed some keys in a GCS bucket go inaccessible since like 19:15-19:20ish PDT, and then we noticed increased timeouts/503's ramping up at 19:30 and plateauing at 19:45. But error rate for us seems to have gone back to nominal levels since 20:10... but the keys that originally went unavailable are still returning 503's.
Fun times.
- jedmeyers 8y agoGCS might be relying on the same system as Gmail/YouTube does, so it’s not necessarily GCS itself. Spanner?
- parent5446 8y agoBlobstore
- mehrdadn 8y agoIt will be very interesting to see what timing issues might be involved if this is indeed due to Spanner.
- ohaideredevs 8y agoIf Google's GCP marketing material is to be believed, you are truly on their infrastructure on GCP, so if GCP is having issues so are Google's main services.