4 ms·

I can just imagine some piece of automation de-allocating spare capacity or something similarly obtuse. The anticipation of traffic coming back would have me o

I can just imagine some piece of automation de-allocating spare capacity or something similarly obtuse. The anticipation of traffic coming back would have me on edge. Maybe everything expires from caches and new traffic comes in on cold caches and slams databases with requests causing the databases to slow down making them unable to handle any traffic...

websap 3y ago
Yikes! If they end up deprovisioning capacity and then some of those subreddits come back after 48h, the engineering teams are gonna be in for a rough couple of hours. I'm expecting that they have frozen autoscaling during this time.
- hayst4ck 3y ago
  I don't doubt Reddit's ops competency, I doubt we will see any problems. I am sure they have people going through the mental exercise of "what is this 'strike' going to do to our infra?" As an ops person the only time I have ever thought "It's going very well" is when something I expected to cause an outage didn't or a new automation seems to be working properly. Everything else is change and failure is proportional to change. 5000 subreddits going dark is definitely change, and I would definitely be anxious about the uncharted territory, not happy to see low load on the servers. I think I would be worried about these people who are downloading their post histories, deleting their post histories, or potentially mass deletion of accounts. Requests against old posts and uncached content could potentially be expensive. Of course, without a deep understanding of Reddit's infra/architecture, it's hard to really estimate what kind of problems might arise.
  - websap 3y ago
    Well this didn't age well rather quickly. https://www.theverge.com/2023/6/12/23758002/reddit-crashing-api-protest-subreddit-private-going-dark https://www.theverge.com/2023/6/12/23758002/reddit-crashing-...
atraac 3y ago
> I can just imagine some piece of automation de-allocating spare capacity or something similarly obtuse. Remember we're talking about a company that can't even make a functioning text input in their new UI.
- znpy 3y ago
  Eh, front end development and site reliability engineering are two different teams for sure. If the ui code is bugged, the sre team will still get that delivered it to your web browser.
  - atraac 3y ago
    Of course they are two different teams. They are still one company though, one that displays absolute disdain for their users, why would they suddenly care about properly up and downscaling when they can't even get absolute basic UX/UI right?

websap 3y ago

Yikes! If they end up deprovisioning capacity and then some of those subreddits come back after 48h, the engineering teams are gonna be in for a rough couple of hours. I'm expecting that they have frozen autoscaling during this time.

hayst4ck 3y ago
I don't doubt Reddit's ops competency, I doubt we will see any problems. I am sure they have people going through the mental exercise of "what is this 'strike' going to do to our infra?" As an ops person the only time I have ever thought "It's going very well" is when something I expected to cause an outage didn't or a new automation seems to be working properly. Everything else is change and failure is proportional to change. 5000 subreddits going dark is definitely change, and I would definitely be anxious about the uncharted territory, not happy to see low load on the servers. I think I would be worried about these people who are downloading their post histories, deleting their post histories, or potentially mass deletion of accounts. Requests against old posts and uncached content could potentially be expensive. Of course, without a deep understanding of Reddit's infra/architecture, it's hard to really estimate what kind of problems might arise.
- websap 3y ago
  Well this didn't age well rather quickly. https://www.theverge.com/2023/6/12/23758002/reddit-crashing-api-protest-subreddit-private-going-dark https://www.theverge.com/2023/6/12/23758002/reddit-crashing-...

atraac 3y ago

> I can just imagine some piece of automation de-allocating spare capacity or something similarly obtuse. Remember we're talking about a company that can't even make a functioning text input in their new UI.

znpy 3y ago
Eh, front end development and site reliability engineering are two different teams for sure. If the ui code is bugged, the sre team will still get that delivered it to your web browser.
- atraac 3y ago
  Of course they are two different teams. They are still one company though, one that displays absolute disdain for their users, why would they suddenly care about properly up and downscaling when they can't even get absolute basic UX/UI right?