9 ms·
I have gained admin access to numerous GCloud Organizations by accident
in Google Cloud, you can assign admin, billing, etc to a google group.
Years ago I made a google group for google cloud administration
A company in Spain, a bunch of startups, etc have added that google group (by accident) as an IAM user with varying level of roles attached
I now have billing access to one account, admin access to another, can just hop into the database of at least two of the accounts
I try to reach out to google support but because I don’t have “business” or “enterprise” level support I can’t even submit a ticket
I’m trying to let them know but can’t, they do t do chat, no phone number, even billing contact is an automated chatbot only
GCloud should have like “emergency reach out to a person” link or something
- TurkishPoptart 3y agoI'm surprised anyone would ever use Google Cloud Platform after reading this. No support at all? AWS blows them out of the water because they value customer support, I guess.
- anon223345 3y agoUpdate #3 Got an honorable mention from Google Bug Hunters They said they’re gonna see if it’s worth fixing and will get back to me. They didn’t award a bug bounty, but I’ll take the kudos.
- gooob 3y agothat is hilarious
- LinuxBender 3y agoAs an admin of their group can you see their group admin contact email address? If so maybe they have an enterprise account and can reach a human in Google. I am not a lawyer but there is probably risk in accessing any of their data, audit trails and all.
- anon223345 3y agoYa I’m not doing anything, I saw these random orgs on my console and just clicked and it took me to those pages Didn’t poke around, was more like what the heck is this? I only noticed because I logged in and the page defaulted to Spanish (it picked the first org, which happens to be a Spanish car company) Then I noticed in the drop downs. I actually thought I was hacked, then realized what was going on. Still trying to find a way to get ahold of Google lol
- LinuxBender 3y agoI have no idea if this [1] is still valid but it has a number that might work. Perhaps tell them you may have experienced a security incident related to a Google UI/UX bug. Another approach could be reaching out to their Project Zero team. [2] or try some of the contacts here [3] and tell them you may have a security incident that has enabled access to other organizations. [1] - https://www.businessinsider.com/guides/tech/how-to-contact-google-support https://www.businessinsider.com/guides/tech/how-to-contact-g... [2] - https://googleprojectzero.blogspot.com/ https://googleprojectzero.blogspot.com/ [3] - https://about.google/contact-google/ https://about.google/contact-google/
- invalidname 3y agoAs a person who paid for Google's "Gold" support. They are less than useless. Don't go into these accounts at all. Not even to try and help/contact them. Laws about this are very vague and no one within the ORG would want to admit that they made a mistake by adding you.
- leesalminen 3y agoI’ve had access to both Google’s and Amazon’s paid support options (up to and including enterprise support). Amazon’s support has gone above and beyond for me over the years in ways I didn’t even expect or ask them to. In comparison, I agree with you that Google’s support is useless. My experiences with AWS support have actually left me with a positive impression of the platform, while my experiences with Google reinforced that they don’t know how to do support. At all. Want to guess where our 8-digit cloud spend goes?
- elzbardico 3y agoOut of curiosity, I tried to search for the specific function of Support Engineer both for AWS and Google on several job boards. It is a mostly un-scientific approach of course, but the results are as expected. It doesn't even look like google hires Support Engineers. LOL
- tapoxi 3y agoThey outsource it. I'm on the U.S. East Coast. My last GCP support ticket ended up in Romania.
- ethbr0 3y agoAnd nothing against Romanians! I work with a ton of brilliant folks there. Outsourcing inevitably creates a firewall between engineering and support that shouldn't exist though. In a properly functioning org, support has a way to escalate quickly to engineering if it's confirmed "This is broken." Engineering in turn uses those incoming requests to recognize flaws in their own products. Outsourcing creates "Hide behind the SLAs and remain ignorant of any issues you've created" barriers that will ultimately sink a company.
- wheaties 3y agoYou are prompted to confirm an external group being added as admin. Someone purposefully ignored it. Good luck. You're trying to do the right thing but if they lawyer you, remind them they added you not you added them.
- leesalminen 3y agoA few months ago I stumbled upon a bug in a state machine that allowed me to obtain stuff without having to pay for it. It was a weird combination of steps and was kind of hard to explain. I submitted a ticket to the support team advising them in painstaking detail the steps needed to reproduce this vulnerability. They could also look at my account and see that I got stuff without paying. A couple days later I got a reply from a support manager that my concern wasn’t valid and there was no bug. The next week I happened to be at a conference where the company in question was a sponsor. So, I visited their booth and spoke with the VP of Eng. He asked me to forward the ticket to security@. Within 8 hours I got a reply from them saying that they had fixed the bug. I guess I’m saying that even if Google let you submit a support ticket it might get ignored because they aren’t trained to deal with security reports.
- toomuchtodo 3y agoTo your point, there should be some easy way to get a security incident report to the security team through an easily discoverable form or similar. This is as easy as "security incident" option in a support ticket drop down, and triage is required whether this is an ingest point or security@ email.
- killjoywashere 3y agoIsn't that what the bug bounty program is? https://bughunters.google.com/ https://bughunters.google.com/ Also, it doesn't shock me that somebody got a common group name early on in an internet-scale service's lifecycle. I've had a couple such experiences. Simple example: in the early days of Google Hangouts, you could choose your own meeting name in the URL. I chose "compass" for a meeting and accidentally landed in a meeting of Google engineers who were very surprised by my appearance. Fortunately my meeting was a meeting I had arranged so I beat feet and changed my URL to the default auto-generated URL before the rest of my participants arrived.
- lazide 3y agoFun times when it becomes common knowledge that to get attention if support isn’t working is to claim a security incident - and everyone starts doing it, hah.
- renewiltord 3y agoPerhaps if you have IAM you can see their users and then email them.
- we_never_see_it 3y agoI have hard times feeling any sympathy for these companies. When you trust an ad company like Google what did you expect? Maybe Google will shutdown this product and fix the secuity hole in the process.
- elzbardico 3y agoKnowing the byzantine ways of Google support, I wouldn't be surprised if Google's reaction to this would be to ban the account of everyone involved in this episode.
- peddling-brink 3y agoGoogle should shut down Cloud because they allow groups to be added to IAM? That’s an interesting take.
- elzbardico 3y agoI think you misinterpreted OP. He is making a pun with the widespread beliefs that a) Google doesn't care about giving user support for their products even if you pay b) Over a not-so-long time the survival rate of every Google product seems to drop to zero unless it is related to search and ads. So, the joke is that the problem would solve itself when google predictably kills this product.
- peddling-brink 3y agoI’m not sure that’s a pun. But on re-read I am seeing the humor. On my first read I just saw the tired vitriol, “google == bad”.
- ghusto 3y agoFair enough, but it's also worth noting that this mistake is difficult to make in AWS. You can do it, but you have to be so explicit about what you're doing that I can't imagine anyone managing it accidentally. The system is broken if this has happened _multiple_ times to this guy.
- 3y ago
- elzbardico 3y agoEven if you manage to reach out to Google, I doubt they will do anything like remove your group from those roles. From their POV you could be just trying to social engineer them into removing someone who has legitimate access. I think you have better chances contacting people in the org who added your group to those roles.
- nocommandline 3y agoGoogle shouldn't automatically remove you but 1. They should contact the firms involved, make them aware of the situation and then the firms will take a decision on whether to remove or not. 2. They should then look over GCP design and see if there's something that they can do to prevent a reoccurrence of this type of error/mistake
- ethbr0 3y agoI have a similar-but-different problem: a commonly used Gmail address that apparently someone(s) not me was using out in the wild for serious business. Among other things, I received: Interview requests for jobs to which I never applied A background screening for a FL sheriff's job Legal communications for buying a home Business relationship emails Account and subscriptions for a variety of services Relevant point being -- every single one of these counterparties had no idea what to do with me responding "I am not the person who you've been talking with about this. They appear to be using my email. Please ask them to update their email." It made me realize how shitty most people are at dealing with anything other than business-as-usual.
- nightpool 3y agoAgreed. I know Google is famously hard to get in touch with, but I don't understand how this fall on Google's plate or is really Google's fault at all. Maybe if they shared some more info about what IAM group they created that managed to trick people into adding it Google could create rules to ban group names like that from being created?
- kadoban 3y ago
- Hamuko 3y agoAsk them to increase "your" GPU quota by 100x. That should surely be enough of a red flag.
- deleted 3y ago[deleted]
- secondcoming 3y agoBe very careful. Even though you're trying to do The Right Thing don't alert these companies to the fact you've been accessing their accounts without permission.
- ghusto 3y agoI'm seeing a lot of these kinds of comments, and think this might just be an American worry (because it's such a litigious place)? People in the Netherlands and England where I also lived for a short while, are pretty chill with these kinds of things. I can't imagine them doing anything other than thanking you profusely. I mention this because I'd rather this kind of attitude wasn't imported to Europe.
- yesco 3y agoI was under the impression that the key concern here is being criminally prosecuted, not sued. Even if they obviously didn't do this on purpose, depending on how they communicate it to the companies involved, the worse case scenario is that it could be perceived as some kind of phishing attack / fakeout done with malicious intent. Even if they could prove their innocence, no one wants to deal with something scary like that in court. While I'm not familiar with the nuances of each European nation's computer fraud laws regarding this, I can't imagine this would be any different there. Especially as Cybersecurity becomes an increasingly international concern.
- laweijfmvo 3y agoFor me it's a Google worry. I'd be terrified they'd delete my Google (Gmail) account with no way to recover.
- michaelt 3y ago> People in the Netherlands and England where I also lived for a short while, are pretty chill with these kinds of things. Imagine that, at 5pm on Friday, you discover your IT system has been the target of a huge hack, possibly by russians or north koreans, that they got access to everything, it's been going on for months, and it's certainly a notifiable breach under GDPR. Would you be chill? I can say from experience, many people call the cops and lawyers first, and only find the support ticket that first-level support fobbed off with a canned response much later.
- cookieperson 3y agoJust be careful y'all. Even though something is a bug or a mistake you could get in bigtime shit over it, or a bill.
- slowmotiony 3y agoIf there's no way to contact them then I'd probably just delete their stuff altogether. What are they gonna do, contact google support? :-)
- boilerupnc 3y agoI would have thought that being "added" to anything is a two-way confirmation: 1. One from the party wanting to add the group to their account. Based on a prior comment, sounds like you are prompted to confirm an external group being added as admin. 2. One from the party administering/owning an external google group being requested to be added. Is there any confirmation here? Without the 2nd confirm, I start imagining security exposures in the family of Ransomware - let's call it "RansomAdd". You randomly add external google groups until you get someone to poke around "too much" and then threaten them with legal action unless they pay up. Ugh.
- deleted 3y ago[deleted]
- lazide 3y agoHah, probably wouldn’t work well though. The types of folks who have money AND would be fooled by something like that would almost never have the time or curiosity to go poking around.
- jacobsenscott 3y agoIsn't this a little like reaching out to Linus because someone changed their home directory permission to rwxrwxrwx? It sucks for them, but what could google do?
- NegativeK 3y agoMaybe if Linus maintained a paid product that included that home directory.
- mulmen 3y agoContact the customer.
- scarmig 3y agoThey could make the default such that you couldn't grant anyone outside your organization any particular role, unless principals associated with that domain are explicitly whitelisted (by domain). (And, in the other direction, there should be a request/response flow when you're added to some random project/org you have no interest in, which can make you vulnerable both to legal attacks by the org mistakenly adding you and to phishing.)
- lazide 3y agoMany folks have contract admins, it would add a lot of friction for the normal case just to try to prevent something that should be transparently dumb anyway.
- yamtaddle 3y agoIf it were happening a bunch, there might be a good case to be made for changing permission-granting UI. Maybe not kernel-level, but OS-level, at least. In fact, lots of distros now warn when a user attempts certain sudo actions, for similar reasons—mistakes were being made, and adding a little or the right kind of friction could prevent them.
- IYasha 3y agofor g in * do wget g; done;
- IYasha 3y agoOk, ok, I see people downvoting this, so I'm correcting myself: for g in * do wget --recursive "g"; done;
- nopoint 3y agoAt my previous job i brought credentials leakage to higher ups attention but it went unfixed for a year. Nothing to gain Other than wasting our time.
- tazjin 3y agoEx-Googler here. Try reporting it through the security disclosure program: https://www.google.com/appserve/security-bugs/m2/new https://www.google.com/appserve/security-bugs/m2/new You can also assume that by virtue of you having posted this here and being on the frontpage, it's probably made it to the internal Google SRE IRC chat by now and someone is trying to find a contact. This almost always works :) Maybe edit your OP with a way to contact you, so that someone can reach out.
- koolba 3y ago> You can also assume that by virtue of you having posted this here and being on the frontpage, it's probably made it to the internal Google SRE IRC chat by now and someone is trying to find a contact. In that case no point in following up at all right? Just post on HN and hope someone in the right spot sees it? > This almost always works :) That’s the type of SLA one can rely on! > Maybe edit your OP with a way to contact you, so that someone can reach out. Having to break online anonymity so that a company can impose the Hollywood rule, “don’t call us, we’ll call you!”, is a truly lousy support structure.
- fnimick 3y ago> Just post on HN and hope someone in the right spot sees it? That's how a lot of Google tech support happens. If you get banned by mistake, you have far better luck making noise here or on Twitter vs actually going through support. We had an app mistakenly banned that we only got human eyes on by calling in favors from old friends who work at Google. It's asinine.
- geraldwhen 3y agoApples anonymous emails may work for this purpose.
- kwhitefoot 3y agoOr Firefox Relay.
- anon223345 3y ago
- mikyd1954 3y agoNot sure what support could do for you that you could not do yourself, ie: undertake to degrade the 'years ago group' and alert responders as needed.
- anon223345 3y agoGood call, the problem is I use the group for my own projects… I will painstakingly change that to not use groups and then delete the group if it lets me It’s just kinda stupid people are allowed to just add my group with my group not even confirming
- darkwater 3y agoJust to better understand, was it a "generic enough" Google Group name that people used its name in the policy thinking they were granting access to their own "google cloud administrators"? Or were people/companies actively part of that Google Group you created?
- anon223345 3y agoYes exactly, it’s a group with just a generic name I made many years ago…
- deleted 3y ago[deleted]
- thecarokann 3y agoThis is a known 'issue': https://news.ycombinator.com/item?id=34193047 https://news.ycombinator.com/item?id=34193047
- anon223345 3y agoHa! This is exactly what I’m seeing… I’m looking through these comments to see how I can reach out to google cloud from these links I do not even want this access…!!!
- deleted 3y ago[deleted]
- mikyd1954 3y agoSeems like nothing support could do that you could not do yourself, ie: degrade the 'years ago group' and assist responders as needed.
- Trencin 3y agoAt least twice I have left a review about a Business on Google Maps and ended up as an admin of their business profile. I don't know what's going on with Google.
- anon223345 3y agoThat’s awesome lol
- tpoacher 3y agoDid you reply to your review at least, to complete the circle? /s
- lopkeny12ko 3y agoJust ignore it move on. There's no winning there. In the worst case, the company may try to file charges against you for computer abuse/fraud. In the best case, an otherwise harmless association is removed from your account. It is impossible to get ahold of anyone at Google if you are not an enterprise customer. Just forget about it and do nothing.
- deleted 3y ago[deleted]
- yolo3000 3y agoI think you can reach them on their forum as well, for example https://www.googlecloudcommunity.com/gc/Security/Welcome-to-the-Security-Space-of-the-Google-Cloud-Community/m-p/175460#M89 https://www.googlecloudcommunity.com/gc/Security/Welcome-to-....
- tedivm 3y agoMost of those "community" forums are not monitored by actual staff. Google uses "communities" as a way to get free tier one support out of the community, and to save even more money they just never bothered with any tiers beyond that.
- breakingrules 3y ago[dead]
- deleted 3y ago[deleted]
- throwawayadvsec 3y agoif you have access to the DB, create a collection/table with a warning/contact info
- crazygringo 3y agoAs other commenters here have noted, a company can't just do this accidentally. If they add an external group, there's a warning message. Because many times it may be a mistake, but there are also many times a company will have a legitimate reason to do so. This isn't a bug, it's a feature. If you want to do the right thing, the correct course of action isn't to notify Google, it's to send an e-mail to the companies so they can revoke access to the group. It's not Google's problem. Or if you don't want to deal with that and the group isn't used for anything anymore and you still want to be a good citizen, just delete everybody else from the group.
- anon223345 3y agoI’m just going to delete the group
- anonymouskimmer 3y agoBad idea. You're a good person, the next person to create the group name (which you've helpfully published here) may not be.
- anon223345 3y agoUPDATE: I have just submitted a bug bounty request That would really help my career and life if I get that! I won’t do anything with the accounts I accidentally have access to
- dboreham 3y agoAlmost as funny as naming your kid "delete from users".
- pirsquare 3y agoAs usual, the expected Google Clown Platform support. FWIW, 3 months ago they shutdown my servers for some minor issue and I'm only able to get them to reactivate after a week. Source: https://news.ycombinator.com/item?id=35133917 https://news.ycombinator.com/item?id=35133917
- oaksoul 3y agoI was successful in the past reaching out through this: https://issuetracker.google.com/issues/new?component=187161&template=1162660 https://issuetracker.google.com/issues/new?component=187161&... I was told "issuetracker" generates messages directly to support/engineering teams and they do look into it. Submit a "defect" and they will answer.
- andrewstuart 3y agoMaybe you could get thousands of dollars big bounty!
- AtNightWeCode 3y agoSurely there should be a way for an owner of a group to revoke these permissions. I am not familiar with the tech though. If it is not too much hassle I would create a new group, switch to it and delete the old one. This is just one of many reasons corps add prefixes to their naming conventions in the cloud. I would not go down the path of contacting the companies. You have to see it from their point of view when it comes to security and legal processes. Just because you know that you have not done anything wrong does not mean anything for how they will proceed. They will start from the objectives. Somebody has access to our stuff.
- 0xbadcafebee 3y agoGCloud security is horrible. It's like they designed the whole thing to be insecure by default. Coming from AWS, the amount of permissions they give by default in the most commonly-used roles is insane. They also seem to lack some functionality necessary to make fine-grained access permissions to access some of their advertised features. It's really crazy.
- kevingmccall 3y agoI'm the SRE oncall for Cloud IAM. Can you send me a message on linkedin (link in my profile)? I'll give you my Google corp account email address.
- anon223345 3y agoHey! I sent you a friend request in LinkedIn, it didn’t let me message you directly without having LinkedIn premium Can send you the whole 9 yards over there
- crazygringo 3y agoFYI there's no such thing as direct messages on HN. You need to put a means of contact in your profile, or edit your comment to add it. It can be a disposable e-mail (like https://temp-mail.org/en/ https://temp-mail.org/en/) if you want to enable a short-term communication like in this case. (Side note, I've seen this crop up so much that it kind of seems like it would be good to have a DM functionality in HN, even if messages were auto-deleted after 7 days or something, or if it just forwarded to a non-public e-mail address.)
- kevingmccall 3y agoThanks, I added my linkedin to my profile because I already treat that as spam :)
- SillyUsername 3y agoPerhaps related to this bug which Google has known about for 12 years, and is potentially in breach of GDPR? https://issuetracker.google.com/issues/35889152 https://issuetracker.google.com/issues/35889152
- anon223345 3y agoUpdate #2 - they actually responded to my bug bounty request. Seems they think it may be worth fixing but not a big enough deal to pay out a bounty to me. Obviously I’d like the bounty but if I got any recognition that would be awesome —- Hi, Thanks again for your report. I've filed a bug with the responsible product team based on your report. The product team will evaluate your report and decide if a fix is required. We'll let you know if the issue was fixed. Regarding our Vulnerability Reward Program: At first glance, it seems this issue is not severe enough to qualify for a reward. However, the VRP panel will take a closer look at the issue at their next meeting. We'll update you once we've come to a decision. If you don't hear back from us in 2-3 weeks or have additional information, let us know! Regards, Google Security Team
- kevingmccall 3y agoYou have my respect!
- sir_rob 3y agoShouldn't you contact the org that added your account? This is much more a config issue / user error than a Google bug.