4 ms·

Instagram is on GCloud? Facebook has a heavy investment in their own DCs, I thought they would have transitioned to them by now.

sneak 7y ago
Just because you run your own ram and compute doesn’t mean you run your object store, too. Running drives is a commodity now, and crypto is cheap since aes-ni. No idea if this is what instagram does or not, just in general. Drives are hot and need lots of power and it’s expensive to out-S3 S3.
- actuator 7y ago
  Object store won't cause Instagram to go down though, no? Assets might fail to load.
  - jjeaff 7y ago
    What's left of instagram without the assets? And some of the assets required to fully load are probably in object store as well.
luu 7y ago
No, the graphs are highly misleading because they autoscale the y axis to the highest point and the they do very loose string matching to detect errors. If you click through to something not on Google cloud, you see moderately elevated error rates (e.g., Instagram is up by 4x) but if you click through to something actually on Google, you see very highly elevated error rates (e.g., roughly 50000x for Snapchat). If you read the "error reports", they actually report that Instagram isn't down (same for Twitter if you check Twitter). The error report detection seems to be just string matching. Here's an actual "error report" from downdetector that's the caused of allegedly elevated error rates: > my twitter timeline: why isn’t snapchat working? anyone’s snapchat not working? snapchat’s being dumb. rip snapchat. The Twitter "error report" is literally a report that Twitter isn't down.
- actuator 7y ago
  Ah. I was looking at it on phone and it wasn't clear. Thanks for clarifying.
  - notatoad 7y ago
    it's no clearer on desktop. that's actually pretty funny that the instagram graph spikes just because people are complaining about snapchat being down. i always thought downdetector was doing something a bit more clever than just reporting the rate of tweets containing the word "instagram" or "facebook" or something. but apparently not.

sneak 7y ago

Just because you run your own ram and compute doesn’t mean you run your object store, too. Running drives is a commodity now, and crypto is cheap since aes-ni. No idea if this is what instagram does or not, just in general. Drives are hot and need lots of power and it’s expensive to out-S3 S3.

actuator 7y ago
Object store won't cause Instagram to go down though, no? Assets might fail to load.
- jjeaff 7y ago
  What's left of instagram without the assets? And some of the assets required to fully load are probably in object store as well.

luu 7y ago

No, the graphs are highly misleading because they autoscale the y axis to the highest point and the they do very loose string matching to detect errors. If you click through to something not on Google cloud, you see moderately elevated error rates (e.g., Instagram is up by 4x) but if you click through to something actually on Google, you see very highly elevated error rates (e.g., roughly 50000x for Snapchat). If you read the "error reports", they actually report that Instagram isn't down (same for Twitter if you check Twitter). The error report detection seems to be just string matching. Here's an actual "error report" from downdetector that's the caused of allegedly elevated error rates: > my twitter timeline: why isn’t snapchat working? anyone’s snapchat not working? snapchat’s being dumb. rip snapchat. The Twitter "error report" is literally a report that Twitter isn't down.

actuator 7y ago
Ah. I was looking at it on phone and it wasn't clear. Thanks for clarifying.
- notatoad 7y ago
  it's no clearer on desktop. that's actually pretty funny that the instagram graph spikes just because people are complaining about snapchat being down. i always thought downdetector was doing something a bit more clever than just reporting the rate of tweets containing the word "instagram" or "facebook" or something. but apparently not.