8 ms·
Reading Uber’s Internal Emails: Bug Bounty report worth $10K
- tyingq 10y agoOuch. No domain verification required by Sendgrid before allowing you to inject a hook that dumps email contents. That's much broader than just Uber. Edit: Yes, it's been fixed, but the fact that it existed for quite some time is still troubling. I'm also curious if the fix retroactively disabled any existing unverified hooks.
- wahnfrieden 10y agoFrom the article: >Also at the moment of writing this bug, it has come to my attention that SendGrid has added extra verification which forces you to have a verified domain before adding a inbound parse webhook.
- innoying 10y agoI believe they did retroactively search for accounts. Source: I had a number of accounts banned when testing different iterations of this bug.
- gizmo 10y agoThis looks like a massive security flaw on sendgrid's side. They should use DNS validation like everybody else to prove ownership of the subdomain.
- wahnfrieden 10y agoFrom the article: >Also at the moment of writing this bug, it has come to my attention that SendGrid has added extra verification which forces you to have a verified domain before adding a inbound parse webhook.
- innoying 10y agoTechnically they did employ some DNS validation. You had to setup a MX record to point to SendGrid before you could add the domain to your account. The problem was in order to send emails from a domain you had to add the same MX record. If you never setup the receiving end as well (on SendGrid) you were vulnerable to a takeover from another SendGrid account.
- ransom1538 10y agoTotally just curious. The law for "exceeds authorized access" is 20 years in prison. I think uber has a bug fixing program. Maybe sendgrid does? Do the DNS carriers? Does his ISP? I read the law. If one of these companies refutes access - this guy is facing 20 years [1]? W(why)TF do people do this? The US government is notoriously creative in these prosecutions [3]. Companies refute access all the time to not look like idiots EVEN WITH a bounty program. [2]. If I came across this I wouldn't be blogging about it for 10k? [1] "(2) intentionally accesses a computer without authorization or exceeds authorized access, and thereby obtains—(C) information from any protected computer;" (C) except as provided in subparagraphs (E) and (F), a fine under this title, imprisonment for not more than 20 years, or both, in the case of—" https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act [2] http://motherboard.vice.com/blog/facebook-is-refusing-to-pay-a-bug-bounty-to-the-researcher-who-hacked-zuck http://motherboard.vice.com/blog/facebook-is-refusing-to-pay... [3] https://en.wikipedia.org/wiki/Aaron_Swartz https://en.wikipedia.org/wiki/Aaron_Swartz
- tyingq 10y agoUber's bug bounty program seems like explicit permission to me. The twist here, as you mention, is that it is sendgrid's infrastructure. Chasing bug bounties does seem a bit risky in this "cloud" era where it's not just one entity running the targeted service.
- beejiu 10y agoIt is not listed as an in-scope domain on Uber's bug bounty policy. You would have to argue that the access was unintentional - that you believed you had authorization to access the server from Uber and that Uber had the ability to assign that authorization on behalf of SendGrid.
- tyingq 10y agoThat is a bit tricky. You could argue that the MX records for a domain that is "in scope" (www.uber.com) are in play. They don't specifically say you can only follow CNAME and A records...and you do have to follow CNAME for some of the listed domains.
- hartator 10y agoIt's just me, or $10k is far from being generous?
- tyingq 10y agoThe bit that's not generous to me is that Sendgrid, who made the mistake, is paying $0.
- Hydraulix989 10y agoI thought so, too. Uber pays its full time employees some of the highest salaries in the Valley. This exploit could have easily fetched 10x the money on the black market.
- innoying 10y agoWhy does this comment appear on every bug bounty HN thread? Straight from the horse's mouth [0]: The black market is very unlikely to be a place you could sell a bug in a specific website or service. It is not “worth millions”. Please stop repeating this. [0] - https://medium.com/@collingreene/to-the-bounty-hunters-9259b1544325 https://medium.com/@collingreene/to-the-bounty-hunters-9259b...
- Hydraulix989 10y agoWhat a cute strawman (and completely incorrect strawman from the wrong horse at that, this is Trumpist drivel from the guy who runs the actual bug bounty who of course has some acute rationalizations for underpaying for some of the most intricate technical work in the industry). Nobody said it was "worth millions" but I have second-degree connections in Scandinavia that would pay 10x, like I said ($100,000). @collingreene doesn't sound too familiar with these rather-illicit organizations, he strikes me as a product manager type person with a loud voice, not someone who actually has found and sold zero-days before. Maybe he doesn't have the technical acumen to do so, but hey, I'm not one to judge. It's hard to establish proof that the market value on the black market is, in fact, much higher given that it is the black market (you're not going to find these people on Medium); However, one public example of this is the leaked Stuxnet details showing similarly high 5-digit prices for zero-days. This isn't a specific bug either (it wasn't "oh, the log files for that one UberEATS micro-service were visible"), this flaw allows you to intercept the emails of pretty much any single one of SendGrid's clients. Imagine the damage someone could do with that, had it gotten into the wrong hands. Only $10k, what an insult. EDIT: Upon further examination, it turns out that said author also contradicts himself and corroborates my own argument: https://medium.com/@collingreene/why-product-security-is-hard-52e3f73178 https://medium.com/@collingreene/why-product-security-is-har... Primary source: https://www.wired.com/2016/09/top-shelf-iphone-hack-now-goes-1-5-million/ https://www.wired.com/2016/09/top-shelf-iphone-hack-now-goes...
- ComputerGuru 10y agoThe last time there was a SendGrid article on here, the feedback from the community was far from kind [0]. I again re-iterate that SendGrid has no business sending emails [1]. [0]: https://news.ycombinator.com/item?id=12142728 https://news.ycombinator.com/item?id=12142728 [1]: https://news.ycombinator.com/item?id=12145019 https://news.ycombinator.com/item?id=12145019
- dinersanddrives 10y agoThis! I actually started using them when they helped me out a hackathon but switched to a more dev friendly service.
- hogrammer 10y agoYou "again reiterate?" Do you know what those two words mean?
- mikesea 10y agoSomeone reported this same vulnerability to us via HackerOne months ago. We worked with Sendgrid support to re-claim the domain and they said they were urgently working to fix the issue, or not. Edit: just saw this post was from September. Author probably made thousands in rewards circulating this vulnerability.
- innoying 10y agoI do not believe the author circulated this report to multiple companies, however once it was made public a number of other reporters in the community did and continued to iterate on it until SendGrid fixed the issues. Source: I am a member of said community: https://bugcrowd.com/bored-engineer https://bugcrowd.com/bored-engineer, https://hackerone.com/bored-engineer https://hackerone.com/bored-engineer, etc
- ndaiger 10y agoSendgrid allowed attackers to social engineer control of my company's account and intercept password resets, despite an explicit warning from us a week prior (we received a chat transcript of the failed attempt and let them know that it was not us and someone was actively trying to social engineer access to our account). Then they had the gall to try to convince me on the phone that it must have been my fault (after our blog post about it blew up). Needless to say, I think they are terrible. Previous HN discussion: https://news.ycombinator.com/item?id=7476836 https://news.ycombinator.com/item?id=7476836
- andrewvc 10y agoAs a former customer (it has admittedly been a few years) I'm not surprised. Sendgrid's entire business is based on price. Emails are one of the few costs that don't really scale that well. They cost a lot more than people think they should. When I list did a cost analysis they beat out a lot of other providers. I haven't used them in years, so I'm not sure if they are still the low cost leader, but they are definitely the kind of thing where you get what you pay for. Their lack of investment in product and infrastructure showed for the years I used them, and their slowness in delivering updates was incredibly frustrating.
- philliphaydon 10y agoWho would you recommend then?
- jc4p 10y agoI'm not the person you asked but I too have had bad experiences with SendGrid. It really depends on which aspect of SendGrid we're talking about. Transactional e-mail? Mailgun is a easy to use API on top of Amazon's SES. You can even set up incoming e-mail hooks e.g. "When a new e-mail arrives, POST the contents to this address and attach any attachments on the e-mail as file uploads." Newsletters? Drip campaigns? I have less experience on this side of the realm, but HubSpot has been the best experience I've seen. Their web management UI is also powered by their own open API, if I remember correctly.
- zaroth 10y agoI found this write-up a bit confusing and hard to follow. The vulnerability is that any SendGrid user could configure a webhook callback which would POST back all received emails for any domain which had its MX set to 'mx.sendgrid.net'. OP exploited this against Uber to receive copies of their emails. Presumably there was no way to tell from one account that another account is web-hooking your email out from under you. So you have to wonder, if it's as easy as just typing the domain you want to listen in on.... who else was getting all their emails tapped this way? From Sendgrid's documentation; Setup The following steps are required to begin parsing email: Point the MX Record of a Domain/Hostname or Subdomain to mx.sendgrid.net Associate the Domain/Hostname and the URL in the Parse API settings page. Shocking omission by Sendgrid, where's their write-up and apology?
- deleted 10y ago[deleted]
- downandout 10y agoThe problem is on SendGrid's side, however I doubt they would have paid $10K for it. He was smart to take it to Uber, who is likely (by far) SendGrid's largest customer, and who certainly has much deeper pockets than SendGrid.
- uraniumhacker 10y agoAt the time of the exploit, there was only one check, the MX record. Also once I claimed the domain, another user cannot claim it from their account. So this could be exploited silently as well because the victimized company won't know about it for a while.
- wongarsu 10y agoHonest question: Do bug bounties with such low bounties really do more good than harm? A bug bounty incentivizes people to look for bugs. But when you find an interesting bug like this you have the option of either having the possibility of making millions from the social engineering possibilities alone, or claiming the bug bounty and get $10k. Of course claiming the bounty is the moral thing to do, but some people with both the skill and motivation to do bug bounties are bound to have lower moral standards.
- arkem 10y agoI think it's hard to monetize most bug bounty bugs. There isn't really a market for most XSS, CSRF or even RCEs bugs for web properties. Getting the bounty payout for a bug from the owner is often the best deal available. I think the only exception to the no-market situation is browser RCEs and smart phone OS jailbreaks.
- Hydraulix989 10y agoRight, and this one could have easily been monetized because it allows attackers to intercept and read any one of SendGrid's customer's private internal emails!
- arkem 10y agoI don't agree that this could have been easily monetized. This vulnerability only allows you to intercept future incoming emails that are delivered to sendgrid domains. Even if it did allow access to all of their customers internal emails there aren't many buyers (and no obvious marketplaces) out there for that kind of access. The risk is simply too high, if you're Lyft (for example) being caught with that access (even if it was never used) would be a possible terminal event for the company. Purchasers would be buying a giant liability and a small competitive advantage. Maybe there are shady companies or criminal groups who would be interested in the access but even then I feel $10k would be roughly how much money they'd be willing to spend (considering the same amount of money could also buy thousands of verified credit cards or a million email addresses at legitimate providers).
- bdittmer 10y agoThis is a lot like a bug I found in Heroku's system a few years ago. Basically, if someone doesn't claim the wildcard subdomain for their primary domain and has a wildcard SSL cert anyone could (can?) claim subdomains. A quick google search yielded hundreds of exploitable domains. At the time it seemed like a pretty big vector for phishing. I have no idea if they fixed this and they gave me a t-shirt.
- dzhiurgis 10y ago> they gave me a t-shirt. I had same experience with Salesforce
- brandur 10y agoI can't recall all the exact details, but there is some validation logic in place along the lines of "if there's a wildcard domain installed then newly added subdomains must be on the same account as the wildcard's owner". You could give it a shot, but I don't think this attack would work. (I used to help maintain the system responsible for this, but don't work there anymore.)
- bdittmer 10y agoThis was probably 2012 or 2013. I'll try to find the write up I sent them, but this check definitely did not exist then edit: here's what we sent to heroku https://gist.github.com/bdittmer/6461b7a5093acd7d6263 https://gist.github.com/bdittmer/6461b7a5093acd7d6263
- joshmn 10y agoWould have been worth more to SendGrid, don't you think?
- abalone 10y agoLooks like they found a similar "3rd party email service / subdomain MX record" exploit with Slack.[1] Although not so severe in Slack's case because it's a lesser used feature. Looks like Slack is using Mailgun.[2] [1] http://blog.pentestnepal.tech/post/150381068912/how-i-snooped-into-your-private-slack-messages http://blog.pentestnepal.tech/post/150381068912/how-i-snoope... [2] http://mxtoolbox.com/SuperTool.aspx?action=mx%3aslack.com&run=toolpage http://mxtoolbox.com/SuperTool.aspx?action=mx%3aslack.com&ru...
- foota 10y agoSeveral months ago I reported a bug to a popular service which was allowing password reset & other emails to be retrieved with just a serial id from a service like sendgrid.
- uraniumhacker 10y agoHi thank you for the post. If anyone has question, feel free to reach out to me here or at uranium@uraniumhacker.tech. As a writer of this blog, I will be able to provide the feedback necessary and clear any misunderstanding/false positives regarding this.