4 ms·

edit: added details edit: redacted my phone number edit: big mistake to add phone number edit: I think illic is right, probably not me edit: removed details

dannyw 6y ago
It’s probably not you.
- contravariant 6y ago
  Although since it seems to be back up now I suppose there is one way we could find out for sure.
rubatuga 6y ago
What are IAM permissions?
- kristopolous 6y ago
  Just permissions. The "IAM" can be safely dropped. It's exactly what you think it'd be: restrictions and privileges. "IAM" is basically the name for a specific model of doing it. Unless something really crazy happened, this user is unlikely to be correct. Accounts are supposed to be firewalled/sandboxed in a way that you can't contagion across to someone else's let alone systemwide. It's possible (some sweeping script on a powerful connection that smashes just the right things or some exploit to break the sandboxing), just probably not likely - especially unintentionally. But crazier things have happened https://books.google.com/books?id=rRp7DkTegMEC&newbks=0&printsec=frontcover&pg=PA127 https://books.google.com/books?id=rRp7DkTegMEC&newbks=0&prin...
- bgilroy26 6y ago
  It's what Amazon calls your cloud login account Identity and Access Management (IAM)
lalwanivikas 6y ago
What exactly did you do that makes you so confident?
- mozinator 6y ago
  Exactly after my setIamPolicy API request to Google Cloud was the exact moment everything went down.
  - scep12 6y ago
    Do you have an exact time?
  - lxgr 6y ago
    Probably dozens of other people executed comparable requests in the instant you did.
    - duncanawoods 6y ago
      Indeed. I deleted a gcp project at the same time. However, it would be fun if it had a UUID clash with a google service :)
  - adriancr 6y ago
    I'd wager a guess that you set up some weird 'expression'? coupled with some bug in the IAM service, maybe some stale resources that you were deleting at the same time? I'd then assume once expression is evaluated the services end up busy looping / proxies throwing out internal errors and taking out capacity. Still, you shouldn't be able to cause downtime to more then a few servers in the extremely unlikely case I am anywhere close. PS: - I haven't used googles IAM so guessing after a few min of reading docs. - you are incredibly unlikely to have triggered this at google's scale.
  - YinglingLight 6y ago
    Big Tech is practicing their shutdown maneuvers as we get closer to Inauguration Day in the US.
ttn 6y ago
If you are right; congrats! you just got few googlers fired!
- gberger 6y ago
  Google doesn't fire people who cause outages.
reutsharabani 6y ago
I highly doubt it.
rk06 6y ago
I am not google, but if i were to start looking for idiots to throw under a bus for this outage, you will on top of the list. besides google won't be exactly calling you because you made that comment here. other commenters are right that you should not expose your personal number on internet regardless of it was you or not
- Ueland 6y ago
  > Last thing you want is a conversation with google's lawyer Google more or less want people find their weaknesses so they can patch and secure them. A person accidentally triggering a global outage is not something that would cause that person to get lawyers on them. Especially not something that only affects his or hers GCP-project.
- mozinator 6y ago
  obviously it was an accident
spbaar 6y ago
This isn't true, but considering every Google outage is a one in a billion rube goldberg domino machine, it could be true. Put this comment in the post mortem!
illic 6y ago
If it was you, why the hell on earth would you bother exposing your personal phone number on the internet and asking google to call you on this post? Like, seriously... Wouldn't you rather call them directly on the hotline...? That's a phone number from Bulgaria, and it looks like it should be part of the Vivacom GSM network, so I guess it's his personal mobile phone number or a scam.
- mozinator 6y ago
  yes personal phone number. Had to do what it takes... To save the company I work for and all of our customers data. Which all is in Google Cloud!
- capableweb 6y ago
  > why the hell on earth would you bother exposing your personal phone number on the internet Why not? It's not like you're gonna receive death threats exactly. I've had my personal phone number on my public website and in the footer of every outgoing email for 15+ years, never had any problems, spam or otherwise.
Heliosmaster 6y ago
Babashka is dangerous...
ecf 6y ago
Or the more realistic scenario is that the outage happened before you did getIamPolicy which is what caused the garbage data.
- mozinator 6y ago
  That makes sense
ccmcarey 6y ago
It was obviously not you.
akmarinov 6y ago
Коле, ти ли си?
- thegeomaster 6y ago
  Hahahahaha this is gold
wewake 6y ago
Ummm.. no.
jiggawatts 6y ago
That reminds me of the time when I plugged a network cable back into an Active Directory domain controller. At the exact same time as the RJ-45 plug would have made the little "click", a door slammed shut and a polished steel tanker truck drove by the window, shining a bright light into my eyes. I had just plugged into a cable into the most important server in the organisation and I saw a bright flash and hear a bang. All was well, it was just a coincidence, and a good reminder that sometimes shit happens and it's not us, it's just timing. Relax.

dannyw 6y ago

It’s probably not you.

contravariant 6y ago
Although since it seems to be back up now I suppose there is one way we could find out for sure.

rubatuga 6y ago

What are IAM permissions?

kristopolous 6y ago
Just permissions. The "IAM" can be safely dropped. It's exactly what you think it'd be: restrictions and privileges. "IAM" is basically the name for a specific model of doing it. Unless something really crazy happened, this user is unlikely to be correct. Accounts are supposed to be firewalled/sandboxed in a way that you can't contagion across to someone else's let alone systemwide. It's possible (some sweeping script on a powerful connection that smashes just the right things or some exploit to break the sandboxing), just probably not likely - especially unintentionally. But crazier things have happened https://books.google.com/books?id=rRp7DkTegMEC&newbks=0&printsec=frontcover&pg=PA127 https://books.google.com/books?id=rRp7DkTegMEC&newbks=0&prin...
bgilroy26 6y ago
It's what Amazon calls your cloud login account Identity and Access Management (IAM)

lalwanivikas 6y ago

What exactly did you do that makes you so confident?

mozinator 6y ago
Exactly after my setIamPolicy API request to Google Cloud was the exact moment everything went down.
- scep12 6y ago
  Do you have an exact time?
- lxgr 6y ago
  Probably dozens of other people executed comparable requests in the instant you did.
  - duncanawoods 6y ago
    Indeed. I deleted a gcp project at the same time. However, it would be fun if it had a UUID clash with a google service :)
- adriancr 6y ago
  I'd wager a guess that you set up some weird 'expression'? coupled with some bug in the IAM service, maybe some stale resources that you were deleting at the same time? I'd then assume once expression is evaluated the services end up busy looping / proxies throwing out internal errors and taking out capacity. Still, you shouldn't be able to cause downtime to more then a few servers in the extremely unlikely case I am anywhere close. PS: - I haven't used googles IAM so guessing after a few min of reading docs. - you are incredibly unlikely to have triggered this at google's scale.
- YinglingLight 6y ago
  Big Tech is practicing their shutdown maneuvers as we get closer to Inauguration Day in the US.

ttn 6y ago

If you are right; congrats! you just got few googlers fired!

gberger 6y ago
Google doesn't fire people who cause outages.

reutsharabani 6y ago

I highly doubt it.

rk06 6y ago

I am not google, but if i were to start looking for idiots to throw under a bus for this outage, you will on top of the list. besides google won't be exactly calling you because you made that comment here. other commenters are right that you should not expose your personal number on internet regardless of it was you or not

Ueland 6y ago
> Last thing you want is a conversation with google's lawyer Google more or less want people find their weaknesses so they can patch and secure them. A person accidentally triggering a global outage is not something that would cause that person to get lawyers on them. Especially not something that only affects his or hers GCP-project.
mozinator 6y ago
obviously it was an accident

spbaar 6y ago

This isn't true, but considering every Google outage is a one in a billion rube goldberg domino machine, it could be true. Put this comment in the post mortem!

illic 6y ago

If it was you, why the hell on earth would you bother exposing your personal phone number on the internet and asking google to call you on this post? Like, seriously... Wouldn't you rather call them directly on the hotline...? That's a phone number from Bulgaria, and it looks like it should be part of the Vivacom GSM network, so I guess it's his personal mobile phone number or a scam.

mozinator 6y ago
yes personal phone number. Had to do what it takes... To save the company I work for and all of our customers data. Which all is in Google Cloud!
capableweb 6y ago
> why the hell on earth would you bother exposing your personal phone number on the internet Why not? It's not like you're gonna receive death threats exactly. I've had my personal phone number on my public website and in the footer of every outgoing email for 15+ years, never had any problems, spam or otherwise.

Heliosmaster 6y ago

Babashka is dangerous...

ecf 6y ago

Or the more realistic scenario is that the outage happened before you did getIamPolicy which is what caused the garbage data.

mozinator 6y ago
That makes sense

ccmcarey 6y ago

It was obviously not you.

akmarinov 6y ago

Коле, ти ли си?

thegeomaster 6y ago
Hahahahaha this is gold

wewake 6y ago

Ummm.. no.

jiggawatts 6y ago

That reminds me of the time when I plugged a network cable back into an Active Directory domain controller. At the exact same time as the RJ-45 plug would have made the little "click", a door slammed shut and a polished steel tanker truck drove by the window, shining a bright light into my eyes. I had just plugged into a cable into the most important server in the organisation and I saw a bright flash and hear a bang. All was well, it was just a coincidence, and a good reminder that sometimes shit happens and it's not us, it's just timing. Relax.