13 ms·

Multiple Vulnerabilities in Pocket

533 points by ers35 11y ago

falcolas 11y ago
One more vulnerability which is possible when you can request an instance's metadata: Any AMI roles which have been given to that instance (for example to enable S3 access or decrypt data using AWS' key service) would be visible. These keys are rotated relatively frequently, but it opens up a whole new level of exploits against the company which runs those AWS servers.
deleted 11y ago
[deleted]
mrbig4545 11y ago
running as root??? seriously?????
- falcolas 11y ago
  Most developers don't think (or know) about security issues like this, and running something as root avoids a whole class of deploy problems, so I understand why this can happen. It's certainly not right, and indicative of the need for either a system administrator with an eye for deployment best practices, or an explicit security position who sets up audits to look for these kinds of flaws.
  - mahouse 11y ago
    To be honest I think you're trying to come up with an explanation for the unexplainable. Running absolutely nothing as root is Security 101
    - zobzu 11y ago
      they generally don't give a shit its about deploying as fast as possible and dealing with problems is something for sre's/people later not a very nice dev models for customers, but works well for corporations since they need to fail as fast as possible
    - acdha 11y ago
      This was a stronger argument in the past when multi-user/app systems were the rule. If you have only one app running on a box and it can access all of the data which matters, what precisely does getting root give you which getting that app user does not? Yes, someone could install a rootkit but these days the way to deal with a compromise is to replace the entire system and in either case it's most likely that that process would be initiated by some external clue. (edit: to be clear, I still don't deploy as root but that's more for other reasons like isolation and I'd be surprised if that was the most pressing security concern on many sites as opposed to things like insecure local services, overly-broad, chainable credentials, etc.)
      - detaro 11y ago
        The ability to go around the host firewall. Accessing data from sub-services that otherwise might run isolated under their own users. The ability to change application source code. Not applicable in all cases, but probably often enough.
        acdha 11y ago
        My point wasn't that those aren't good but that they're hard enough to do effectively that most places won't see much benefit until they've done a bunch of other things first. e.g. how many places use least-privilege auth credentials vs. having something like AWS keys or shared database credentials which have access to a ton of shared resources? I'd want to compartmentalize something like that well before changing the UID which code runs under since it's available without any further exploits.
      - marcosdumay 11y ago
        It'll probably get you ability to rewrite the logs, interfere with monitor, and all other capacities needed to hide the breach. (Yes, the logs should be remote and write-only. Now, what are the actual odds of that?)
        acdha 11y ago
        It's certainly possible that something will be logged and it's even possible that someone is actually watching the logs but it's still a gamble that the attacker does something which draws attention: if they have a solid exploit and don't do something which disables legitimate service it's unlikely to be noticed at most organizations.
  - stephengillie 11y ago
    > a system administrator with an eye for deployment best practices Many companies call this position "DevOps Manager" or something similar. They usually own their servers, manage builds and deploys, maintain the repos, and communicate to management and developers.
  - stefantalpalaru 11y ago
    > running something as root avoids a whole class of deploy problems Only one class of deploy problems: you're not at all familiar with Linux or other Unix clones.
- nadams 11y ago
  After seeing a popular indie game developer connect directly to MySQL from his game [1] - nothing surprises me anymore. [1] http://forums.somethingawful.com/showthread.php?noseen=0&threadid=2803713&pagenumber=258#post398884189 http://forums.somethingawful.com/showthread.php?noseen=0&thr...
  - sp332 11y ago
    Sorry, you must be a registered forums member to view this page.
    - _yy 11y ago
      https://webcache.googleusercontent.com/search?sourceid=chrome-psyapi2&ion=1&espv=2&es_th=1&ie=UTF-8&q=cache%3Ahttp%3A%2F%2Fforums.somethingawful.com%2Fshowthread.php%3Fnoseen%3D0%26threadid%3D2803713%26pagenumber%3D258%23post398884189&oq=cache%3Ahttp%3A%2F%2Fforums.somethingawful.com%2Fshowthread.php%3Fnoseen%3D0%26threadid%3D2803713%26pagenumber%3D258%23post398884189 https://webcache.googleusercontent.com/search?sourceid=chrom...
    - coldpie 11y ago
      The short version: Super Meat Boy for PC connected directly to a MySQL database to upload levels created in the level editor. The DB address, username, and password were all stored in plaintext in the binary. The DB user had UPDATE and INSERT permissions, but not DELETE, so the game author figured there was no harm to make that user public. Contemporary HN thread about it: https://news.ycombinator.com/item?id=3387628 https://news.ycombinator.com/item?id=3387628
      - MichaelGG 11y ago
        I've seen a serious multitenant business app that manages financial info do this. Except the credentials were root. And also valid for SSH. Users would run a Java applet, which would SELECT from users (after connecting as root) to determine if their login was successful. 7-8 figures per install and this was a "known issue".
    - 11y ago
cddotdotslash 11y ago
Really interesting write up! I'm surprised they are still running in EC2-Classic. However, even if they are, security groups should still be restrictive enough to prevent some of the things discussed. For example, bypassing the load balancer shouldn't be possible. A security group applied to the back end instances should only allow HTTP/S traffic from the load balancer group. SSH security groups should only allow inbound traffic from known IPs (like the office network), etc. Unfortunately, not enough people do this, and once you can query instance meta data or obtain an SSH key, it's game over.
- skarap 11y ago
  > once you can query instance meta data or obtain an SSH key, it's game over It's usually the public SSH key which is stored in instance metadata. IAM roles/STS is much more scarier in this case.
mike-cardwell 11y ago
http://help.getpocket.com/customer/portal/articles/1225832-pocket-security-overview http://help.getpocket.com/customer/portal/articles/1225832-p... "Pocket does not provide monetary compensation for any identified or possible vulnerability." Cheapskates. This could have cost them money if somebody abusive had discovered it first. He deserves a monetary award. [edit] Should we be concerned about the massive number of people listed on that page who have found security problems with Pocket? I counted 153 separate people...
- misterjinx 11y ago
  was going to ask about the bounty and then saw this. bummer.
- drzaiusapelord 11y ago
  >He deserves a monetary award. Heaven forbid, people submit security issues because they want to be helpful or care about the project.
  - Mikushi 11y ago
    Heaven forbid we ask of a corporation that they compensate us for a job they should have done and preventing huge losses and a PR nightmare. Had I been OP I would have sold the exploits with no remorse, we don't need companies like that on the market, rewarding corporations for bad behaviour with free labour is fucked up on every level.
    - mrestko 11y ago
      You would have committed a felony?
      - SCHiM 11y ago
        Depending on where OP lives, that might not have been a felony. I'm pretty sure that in my country I can sell whatever software I want, as long as I brand it as 'automatic patch management testing' or something...
billyhoffman 11y ago
I'm really not a fan of EC2 exposing instance meta data as a RESTful HTTP API running on Local-Link IP addresses. If its only supposed to be queried locally, why aren't these just environment variables? Perhaps they are dynamic and that won't work but come on! At the very least, run it on localhost:10101 or something. Don't give us another range to have to filter!
- JamesLeonis 11y ago
  That IP for the metadata is internal to Amazons AWS. Under normal circumstances it isn't exposed to the outside network. Because Pocket is remembering URLs and fetching them for us, it leaks it's own private network in the responses.
- skarap 11y ago
  One of the biggest selling points of EC2 (at least for me) is that you get a real VM with a real kernel and userspace, and not some "user-friendly" thing the provider made with loads of alien services running as root. So EC2 can't define vars inside your instances + as you said, they can be dynamic. localhost:SOMETHING will also not work for same reason - they would need to run a service inside your instance. There is one more popular solution to the metadata problem - providing it as an emulated cdrom, which would be also vulnerable in this case. And it can't be dynamic.
- zeendo 11y ago
  Both of those solutions require Amazon to do "something" to the instance itself and thus limit the kinds of machines you can run on EC2. Custom kernels, FreeBSD, etc.. Their current metadata approach works across OSes. And, yes, the data are dynamic. Things like AWS access keys change over time and can be accessible via the metadata if you've given the instances IAM profiles. I'm surprised the author didn't mention this. I agree that the approach feels uncomfortable in general but it seems like the best approach for the functionality they wanted.
jerf 11y ago
A little tip for people trying to armor themselves against this problem: If your app reaches out to do network transactions, it really ought to block localhost. However, bear in mind that localhost isn't "127.0.0.1"... it's "127.0.0.0/8" (or "127.x.x.x" if you don't casually speak CIDR). Ping 127.2.88.33 on your console now... you'll see replies. On the flip side, if you're doing a security test like this, I've gotten mileage out of convincing apps to access local resources with things like 127.88.23.245, precisely because the developer blocked 127.0.0.1 specifically and thought they were done. You should also usually block all internal and external IPs for your entire network, but especially in the cloud this can begin to get tricky. Still, you should. And don't forget IPv6.
- pfg 11y ago
  Also worth noting: don't just regex-check the URL with (localhost|127.*) or something similar. Any hostname could point to 127.0.0.1. iptables with --uid-owner denying traffic to local/private IP space (plus infrastructure-specific stuff like EC2's instance metadata service) would probably be the best option.
  - mike-cardwell 11y ago
    For those who don't know about uid-owner. Here's how you would block any process running as the "mike" user from accessing anything on localhost: iptables -A OUTPUT -m owner --uid-owner mike -d 127.0.0.0/8 -j REJECT ip6tables -A OUTPUT -m owner --uid-owner mike -d ::1 -j REJECT
  - fidget 11y ago
    -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT # Create a chain for outgoing proxy traffic -N PROXY_OUT -A OUTPUT -m owner --uid-owner proxy -j PROXY_OUT # Allow replies to the requestor -A PROXY_OUT -p tcp --sport 8080 -j ACCEPT # Prevent proxy from talking to anything non HTTP{,s} (and DNS) -A PROXY_OUT -p tcp -m multiport ! --dports 80,443,53 -j DROP # Allow DNS udp -A PROXY_OUT -p udp --dport 53 -j ACCEPT # Allow the proxy to specific private ips (demos servers) -A PROXY_OUT -d xxx.xxx.xxx.xxx/32 -j ACCEPT # Prevent proxy from talking to anything private -A PROXY_OUT ! -o <%= @public_iface %> -j DROP -A PROXY_OUT -d 10.0.0.0/8 -j DROP -A PROXY_OUT -d 172.16.0.0/12 -j DROP -A PROXY_OUT -d 192.168.0.0/16 -j DROP # Prevent proxy from talking to services via public ips <% @aws_public_ips.each do |name, facts| %> # <%= name %> -A PROXY_OUT -d <%= facts['ec2_public_ipv4'] %>/32 -j DROP <% end %> Anything I missed? Blocking outgoing ports is to taste.
ddlatham 11y ago
If you were Pocket how would you handle the vulnerability created by having internal services hit user-supplied URLs? Some ideas: - Move the service doing the fetching to an untrusted network. At least it would be unable to access any internal services and any compromises there would be hopefully limited. You still have the problem that the local machines there could potentially be compromised. - Validate / verify the URL to ensure it's not hitting anything internal. This sounds hard. Pre-resolve the name and check to see if the IP is in an internal range? Seems easy to get our of date as your network changes. Make sure to repeat for any redirects? Is there a better way to validate? - Ensure that all internal services require authentication. This also sounds hard and easy to miss something.
- option_greek 11y ago
  Regarding point 2 (Validate), won't blocking the entire private address IPV4 range (10.xx, 172.xx, 169.xx, 127.xx and what not) suffice ?
  - jffry 11y ago
    There was also the address of the other internal services. I think the best way here is to put the "fetch random URLs" service out in its own subnet, where it cannot access any other internal services like the EC2 status service. You'll also have to validate the URLs (no non-HTTP-or-HTTPS URLs) and prevent things like the redirect attack from working.
  - pfg 11y ago
    It would, but it's quite easy to miss something in the actual implementation: - How do you extract the hostname from the URL? If the algorithm isn't the same as the one used by your network lib, it might be possible to trick your check into checking the wrong hostname. - You'd have to check for redirects. - If you pre-resolve DNS hostnames for your check, and then let your network lib open another socket to the host, it might resolve to another (internal) IP, because the attacker might control the DNS zone of that host, returning 127.0.0.1 on every other request. You'd have to make sure to open a socket to the IP returned during the check. The safer option would be to work with iptables: https://news.ycombinator.com/item?id=10079554 https://news.ycombinator.com/item?id=10079554
halosghost 11y ago
The one thing I don't like about this article (and indeed, much of the discourse around the Pocket integration) is its characterization of the Pocket integration itself. It calls it an “opt-out non-removable [extension]”. The truth is that you can easily disable it just as you can easily disable many other things that Firefox includes by-default. In fact, if you use Classic Theme Restorer (I use it not because I dislike australis, but because I really do not want a navigation toolbar), it has an option baked in to disable Pocket along with webrtc, et al. Admittedly, I suppose it would be nice if Firefox actually packaged Pocket as a real extension that could be removed from the Extensions menu, but they have already integrated several things without using that schema. I still use firefox, just with more and more things disabled, because none of the other browsers out there even come close to having what I need in a GUI browser (though, I would note that I'm evermore tempted to abandon GUI browsing altogether). Either way, the write-up is great, and everything in the article other than that one characterization (which rubbed me a bit the wrong way in the wake of all the fevered discussions around the Pocket Integration) was a truly enjoyable read. Not to mention, it's great that the Pocket devs fixed things quickly; that's always a plus!
- sigmar 11y ago
  >you can easily disable it just as you can easily disable many other things that Firefox includes by-default >it would be nice if Firefox actually packaged Pocket as a real extension that could be removed from the Extensions menu These two statements you made seem to corroborate his characterization of it being "opt-out" (meaning on by default, but capable of being disabled) and "non-removable" (baked into firefox as opposed to an extension). Not sure what you find wrong with his characterization.
  - halosghost 11y ago
    “opt-out” is certainly correct (though, as I understand it, Pocket, as with all parts of Firefox, is only loaded when it is actually used, so “opt-out” does not seem to tell the whole story to me). I do disagree with the “non-removable” bit. Fully disabling it, to me, counts as removable (though I can understand why someone would disagree). Perhaps I just read something into the Author's tone (probably due to all the vitriole from the typical discourse around the integration) that wasn't really there. If that's the case, and all the author meant by that statement was that the code itself could not be completely erased from Firefox as-packaged, then that seems to be factually true, and I simply read it incorrectly.
pdkl95 11y ago
This is the problem with services that store user information: it is highly probably that vulnerabilities like these exist. Security is rarely given the time and attention it requires. I'm not trying to single out Pocket; they are just the latest evidence that even in the few cases where "you can trust us with your data" is said honestly, it isn't a promise that can be kept in practice.
luxoria 11y ago
>Grab ssh private keys from autoprovisioned EC2 user’s home directory using 301 redirect to file URI (after all, we’re running as root, we can read them). This is not a fair assumption to make. Maybe they are running a LSM like AppArmor.
- skarap 11y ago
  What is more important - the existence of SSH private keys on the EC2 instances us unlikely... There is chance there are SSH private keys there, but they would most probably be SSH deploy keys for some private repo (configuration management, software).
BenjaminWill 11y ago
Oh Mozilla, why couldn't you resist the money. Your recent so called "services" are not welcome. You know it. But well, money makes the world go around. How much did Telefonica pay you for the Hello integration? But sure, our surfing history will be secure ... https://blog.mozilla.org/advancingcontent/2015/05/21/providing-a-valuable-platform-for-advertisers-content-publishers-and-users/ https://blog.mozilla.org/advancingcontent/2015/05/21/providi... Did you guys acutally read your PR-bullshit here? But soon a new small, fast, free, secure open-source browser will arrive and Mozilla will be history. But your pocket full. Well done.
- lucian1900 11y ago
  > a new small, fast, free, secure open-source browser will arrive Really? How exactly? Who will fund its development?
  - nivla 11y ago
    One way would be to keep forking Firefox's codebase for bug fixes and new features while customizing it to fit the needs of the privacy conscious users. It may not be "ethical" but it will serve the main premises of open source and is also doable as a community. A new name would also be required as Firefox is trademarked by Mozilla (See Iceweasel Browser [1]). [1] https://wiki.debian.org/Iceweasel https://wiki.debian.org/Iceweasel
  - brongondwana 11y ago
    Jon von Tetzchner. He's got a team of experienced browser devs and fuck-you money from selling his last browser company.
- rntz 11y ago
  Never attribute to malice what can be adequately explained by incompetence. Mozilla almost certainly isn't getting any money out of the Pocket integration deal. Mozilla has far more money than Pocket does.
- reitanqild 11y ago
  FWIW: Reading this on Pale Moon which is basically a rebranded fork of Firefox before Aurora. Can't say anything about PaleMoons security but I like the old design better, the team seems more responsive and they haven't yet (had the chance to) give in to hypocrites and fire "Brendan Eich".
hundt 11y ago
Security researchers out there: on what side of "the line" do you view this kind of exploitation to be? It was not done for nefarious purposes, but it did involve intentionally accessing resources that were clearly not intended to be accessed, like /etc/passwd. Would you worry if you did this that the company might call the police instead of thanking you?
schmichael 11y ago
Is there a name for this sort of attack? We were just protecting against some similar attacks earlier this week, and it would have been nice to have a short name to refer to them as instead of "that attack vector where we make unrestricted HTTP requests based on user input".
- Manishearth 11y ago
  "Server Side Request Forgery" (SSRF)
skarap 11y ago
Yet another service discovered which was built/deployed with no regard for security whatsoever. I'm beginning to realize - this is the norm. Security is the least important thing for most of the IT companies. I guess the DevOps trend (i.e. not hiring sysadmins) should take it's share of blame. Or maybe it's the other way around - you don't care for security, so there is no point in hiring security experts?
- mmatants 11y ago
  It's hard to get an intuitive feel for security. If team principals are mostly young and don't have exposure to very specific kind of experiences dealing with threat modelling, then the security work will always take a backseat to feature-building. It's how humans work - it's hard to cut time from nice visible things and dedicate it to a hypothetical (at that point) abstract goal like security. Everyone agrees that the latter is important, but it just never ends up getting any oxygen. What's worse, if the team does not know what tightening systems feels like, they don't even know where to start. No "muscle memory" for it. Maybe at some point we'll get more baseline collective wisdom about it throughout the industry, but it will also take the people signing the cheques (CEO, investors) having a bit more (justified) paranoia and respect for these priorities, and consequently requiring them from the outset. And DevOps has little to do with it - security has to be established as a priority from the leadership levels on down anyway because it necessarily means reducing time spent on more immediately-visible things.
  - skarap 11y ago
    I agree but partially. We are not talking about some complex interactions between multiple components which lead to a security vulnerability. This is some trivial stuff like "don't give your passwords to anybody" or "don't run everything as root". The most complex vulnerability mentioned in the article is with proxying. If you have opened /etc/squid/squid.conf at least once you should have noticed the to_localhost ACL and the comments which explain why it is important. So is the Pocket team building a multi-million user service which has a proxying component without trying to configure squid once? Absolutely! Also I consider too optimistic hoping for the situation to improve - it's moving in the opposite direction for now with steady speed.
Iuz 11y ago
Is it ironic that I saved the article on pocket?
dafrankenstein2 11y ago
i prefer using offline bookmark..specially the bookmark manager of Opera browser is impressive. ============================================= and for online bookmarking there is 'raindrop.io'
dafrankenstein2 11y ago
i prefer using offline bookmark..specially the bookmark manager of Opera browser is impressive. ============================================= and for online bookmarking there is 'raindrop.io'
_navaneethan 11y ago
You should receive some gift from pocket team :)
robn_fastmail 11y ago
If you want a laundry list of SSRF methods you should protect against, a great place to start is this slide deck from a talk at ONsec a couple of years ago: http://www.slideshare.net/d0znpp/ssrf-attacks-and-sockets-smorgasbord-of-vulnerabilities http://www.slideshare.net/d0znpp/ssrf-attacks-and-sockets-sm... Its thrilling and terrifying :)

533 points by ers35 11y ago

One more vulnerability which is possible when you can request an instance's metadata: Any AMI roles which have been given to that instance (for example to enable S3 access or decrypt data using AWS' key service) would be visible. These keys are rotated relatively frequently, but it opens up a whole new level of exploits against the company which runs those AWS servers.

deleted 11y ago

[deleted]

mrbig4545 11y ago

running as root??? seriously?????

falcolas 11y ago
Most developers don't think (or know) about security issues like this, and running something as root avoids a whole class of deploy problems, so I understand why this can happen. It's certainly not right, and indicative of the need for either a system administrator with an eye for deployment best practices, or an explicit security position who sets up audits to look for these kinds of flaws.
- mahouse 11y ago
  To be honest I think you're trying to come up with an explanation for the unexplainable. Running absolutely nothing as root is Security 101
  - zobzu 11y ago
    they generally don't give a shit its about deploying as fast as possible and dealing with problems is something for sre's/people later not a very nice dev models for customers, but works well for corporations since they need to fail as fast as possible
  - acdha 11y ago
    This was a stronger argument in the past when multi-user/app systems were the rule. If you have only one app running on a box and it can access all of the data which matters, what precisely does getting root give you which getting that app user does not? Yes, someone could install a rootkit but these days the way to deal with a compromise is to replace the entire system and in either case it's most likely that that process would be initiated by some external clue. (edit: to be clear, I still don't deploy as root but that's more for other reasons like isolation and I'd be surprised if that was the most pressing security concern on many sites as opposed to things like insecure local services, overly-broad, chainable credentials, etc.)
    - detaro 11y ago
      The ability to go around the host firewall. Accessing data from sub-services that otherwise might run isolated under their own users. The ability to change application source code. Not applicable in all cases, but probably often enough.
      - acdha 11y ago
        My point wasn't that those aren't good but that they're hard enough to do effectively that most places won't see much benefit until they've done a bunch of other things first. e.g. how many places use least-privilege auth credentials vs. having something like AWS keys or shared database credentials which have access to a ton of shared resources? I'd want to compartmentalize something like that well before changing the UID which code runs under since it's available without any further exploits.
    - marcosdumay 11y ago
      It'll probably get you ability to rewrite the logs, interfere with monitor, and all other capacities needed to hide the breach. (Yes, the logs should be remote and write-only. Now, what are the actual odds of that?)
      - acdha 11y ago
        It's certainly possible that something will be logged and it's even possible that someone is actually watching the logs but it's still a gamble that the attacker does something which draws attention: if they have a solid exploit and don't do something which disables legitimate service it's unlikely to be noticed at most organizations.
- stephengillie 11y ago
  > a system administrator with an eye for deployment best practices Many companies call this position "DevOps Manager" or something similar. They usually own their servers, manage builds and deploys, maintain the repos, and communicate to management and developers.
- stefantalpalaru 11y ago
  > running something as root avoids a whole class of deploy problems Only one class of deploy problems: you're not at all familiar with Linux or other Unix clones.
nadams 11y ago
After seeing a popular indie game developer connect directly to MySQL from his game [1] - nothing surprises me anymore. [1] http://forums.somethingawful.com/showthread.php?noseen=0&threadid=2803713&pagenumber=258#post398884189 http://forums.somethingawful.com/showthread.php?noseen=0&thr...
- sp332 11y ago
  Sorry, you must be a registered forums member to view this page.
  - _yy 11y ago
    https://webcache.googleusercontent.com/search?sourceid=chrome-psyapi2&ion=1&espv=2&es_th=1&ie=UTF-8&q=cache%3Ahttp%3A%2F%2Fforums.somethingawful.com%2Fshowthread.php%3Fnoseen%3D0%26threadid%3D2803713%26pagenumber%3D258%23post398884189&oq=cache%3Ahttp%3A%2F%2Fforums.somethingawful.com%2Fshowthread.php%3Fnoseen%3D0%26threadid%3D2803713%26pagenumber%3D258%23post398884189 https://webcache.googleusercontent.com/search?sourceid=chrom...
  - coldpie 11y ago
    The short version: Super Meat Boy for PC connected directly to a MySQL database to upload levels created in the level editor. The DB address, username, and password were all stored in plaintext in the binary. The DB user had UPDATE and INSERT permissions, but not DELETE, so the game author figured there was no harm to make that user public. Contemporary HN thread about it: https://news.ycombinator.com/item?id=3387628 https://news.ycombinator.com/item?id=3387628
    - MichaelGG 11y ago
      I've seen a serious multitenant business app that manages financial info do this. Except the credentials were root. And also valid for SSH. Users would run a Java applet, which would SELECT from users (after connecting as root) to determine if their login was successful. 7-8 figures per install and this was a "known issue".
  - 11y ago

cddotdotslash 11y ago

Really interesting write up! I'm surprised they are still running in EC2-Classic. However, even if they are, security groups should still be restrictive enough to prevent some of the things discussed. For example, bypassing the load balancer shouldn't be possible. A security group applied to the back end instances should only allow HTTP/S traffic from the load balancer group. SSH security groups should only allow inbound traffic from known IPs (like the office network), etc. Unfortunately, not enough people do this, and once you can query instance meta data or obtain an SSH key, it's game over.

skarap 11y ago
> once you can query instance meta data or obtain an SSH key, it's game over It's usually the public SSH key which is stored in instance metadata. IAM roles/STS is much more scarier in this case.

mike-cardwell 11y ago

http://help.getpocket.com/customer/portal/articles/1225832-pocket-security-overview http://help.getpocket.com/customer/portal/articles/1225832-p... "Pocket does not provide monetary compensation for any identified or possible vulnerability." Cheapskates. This could have cost them money if somebody abusive had discovered it first. He deserves a monetary award. [edit] Should we be concerned about the massive number of people listed on that page who have found security problems with Pocket? I counted 153 separate people...

misterjinx 11y ago
was going to ask about the bounty and then saw this. bummer.
drzaiusapelord 11y ago
>He deserves a monetary award. Heaven forbid, people submit security issues because they want to be helpful or care about the project.
- Mikushi 11y ago
  Heaven forbid we ask of a corporation that they compensate us for a job they should have done and preventing huge losses and a PR nightmare. Had I been OP I would have sold the exploits with no remorse, we don't need companies like that on the market, rewarding corporations for bad behaviour with free labour is fucked up on every level.
  - mrestko 11y ago
    You would have committed a felony?
    - SCHiM 11y ago
      Depending on where OP lives, that might not have been a felony. I'm pretty sure that in my country I can sell whatever software I want, as long as I brand it as 'automatic patch management testing' or something...

billyhoffman 11y ago

I'm really not a fan of EC2 exposing instance meta data as a RESTful HTTP API running on Local-Link IP addresses. If its only supposed to be queried locally, why aren't these just environment variables? Perhaps they are dynamic and that won't work but come on! At the very least, run it on localhost:10101 or something. Don't give us another range to have to filter!

JamesLeonis 11y ago
That IP for the metadata is internal to Amazons AWS. Under normal circumstances it isn't exposed to the outside network. Because Pocket is remembering URLs and fetching them for us, it leaks it's own private network in the responses.
skarap 11y ago
One of the biggest selling points of EC2 (at least for me) is that you get a real VM with a real kernel and userspace, and not some "user-friendly" thing the provider made with loads of alien services running as root. So EC2 can't define vars inside your instances + as you said, they can be dynamic. localhost:SOMETHING will also not work for same reason - they would need to run a service inside your instance. There is one more popular solution to the metadata problem - providing it as an emulated cdrom, which would be also vulnerable in this case. And it can't be dynamic.
zeendo 11y ago
Both of those solutions require Amazon to do "something" to the instance itself and thus limit the kinds of machines you can run on EC2. Custom kernels, FreeBSD, etc.. Their current metadata approach works across OSes. And, yes, the data are dynamic. Things like AWS access keys change over time and can be accessible via the metadata if you've given the instances IAM profiles. I'm surprised the author didn't mention this. I agree that the approach feels uncomfortable in general but it seems like the best approach for the functionality they wanted.

jerf 11y ago

A little tip for people trying to armor themselves against this problem: If your app reaches out to do network transactions, it really ought to block localhost. However, bear in mind that localhost isn't "127.0.0.1"... it's "127.0.0.0/8" (or "127.x.x.x" if you don't casually speak CIDR). Ping 127.2.88.33 on your console now... you'll see replies. On the flip side, if you're doing a security test like this, I've gotten mileage out of convincing apps to access local resources with things like 127.88.23.245, precisely because the developer blocked 127.0.0.1 specifically and thought they were done. You should also usually block all internal and external IPs for your entire network, but especially in the cloud this can begin to get tricky. Still, you should. And don't forget IPv6.

pfg 11y ago
Also worth noting: don't just regex-check the URL with (localhost|127.*) or something similar. Any hostname could point to 127.0.0.1. iptables with --uid-owner denying traffic to local/private IP space (plus infrastructure-specific stuff like EC2's instance metadata service) would probably be the best option.
- mike-cardwell 11y ago
  For those who don't know about uid-owner. Here's how you would block any process running as the "mike" user from accessing anything on localhost: iptables -A OUTPUT -m owner --uid-owner mike -d 127.0.0.0/8 -j REJECT ip6tables -A OUTPUT -m owner --uid-owner mike -d ::1 -j REJECT
- fidget 11y ago
  -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT # Create a chain for outgoing proxy traffic -N PROXY_OUT -A OUTPUT -m owner --uid-owner proxy -j PROXY_OUT # Allow replies to the requestor -A PROXY_OUT -p tcp --sport 8080 -j ACCEPT # Prevent proxy from talking to anything non HTTP{,s} (and DNS) -A PROXY_OUT -p tcp -m multiport ! --dports 80,443,53 -j DROP # Allow DNS udp -A PROXY_OUT -p udp --dport 53 -j ACCEPT # Allow the proxy to specific private ips (demos servers) -A PROXY_OUT -d xxx.xxx.xxx.xxx/32 -j ACCEPT # Prevent proxy from talking to anything private -A PROXY_OUT ! -o <%= @public_iface %> -j DROP -A PROXY_OUT -d 10.0.0.0/8 -j DROP -A PROXY_OUT -d 172.16.0.0/12 -j DROP -A PROXY_OUT -d 192.168.0.0/16 -j DROP # Prevent proxy from talking to services via public ips <% @aws_public_ips.each do |name, facts| %> # <%= name %> -A PROXY_OUT -d <%= facts['ec2_public_ipv4'] %>/32 -j DROP <% end %> Anything I missed? Blocking outgoing ports is to taste.

ddlatham 11y ago

If you were Pocket how would you handle the vulnerability created by having internal services hit user-supplied URLs? Some ideas: - Move the service doing the fetching to an untrusted network. At least it would be unable to access any internal services and any compromises there would be hopefully limited. You still have the problem that the local machines there could potentially be compromised. - Validate / verify the URL to ensure it's not hitting anything internal. This sounds hard. Pre-resolve the name and check to see if the IP is in an internal range? Seems easy to get our of date as your network changes. Make sure to repeat for any redirects? Is there a better way to validate? - Ensure that all internal services require authentication. This also sounds hard and easy to miss something.

option_greek 11y ago
Regarding point 2 (Validate), won't blocking the entire private address IPV4 range (10.xx, 172.xx, 169.xx, 127.xx and what not) suffice ?
- jffry 11y ago
  There was also the address of the other internal services. I think the best way here is to put the "fetch random URLs" service out in its own subnet, where it cannot access any other internal services like the EC2 status service. You'll also have to validate the URLs (no non-HTTP-or-HTTPS URLs) and prevent things like the redirect attack from working.
- pfg 11y ago
  It would, but it's quite easy to miss something in the actual implementation: - How do you extract the hostname from the URL? If the algorithm isn't the same as the one used by your network lib, it might be possible to trick your check into checking the wrong hostname. - You'd have to check for redirects. - If you pre-resolve DNS hostnames for your check, and then let your network lib open another socket to the host, it might resolve to another (internal) IP, because the attacker might control the DNS zone of that host, returning 127.0.0.1 on every other request. You'd have to make sure to open a socket to the IP returned during the check. The safer option would be to work with iptables: https://news.ycombinator.com/item?id=10079554 https://news.ycombinator.com/item?id=10079554

halosghost 11y ago

The one thing I don't like about this article (and indeed, much of the discourse around the Pocket integration) is its characterization of the Pocket integration itself. It calls it an “opt-out non-removable [extension]”. The truth is that you can easily disable it just as you can easily disable many other things that Firefox includes by-default. In fact, if you use Classic Theme Restorer (I use it not because I dislike australis, but because I really do not want a navigation toolbar), it has an option baked in to disable Pocket along with webrtc, et al. Admittedly, I suppose it would be nice if Firefox actually packaged Pocket as a real extension that could be removed from the Extensions menu, but they have already integrated several things without using that schema. I still use firefox, just with more and more things disabled, because none of the other browsers out there even come close to having what I need in a GUI browser (though, I would note that I'm evermore tempted to abandon GUI browsing altogether). Either way, the write-up is great, and everything in the article other than that one characterization (which rubbed me a bit the wrong way in the wake of all the fevered discussions around the Pocket Integration) was a truly enjoyable read. Not to mention, it's great that the Pocket devs fixed things quickly; that's always a plus!

sigmar 11y ago
>you can easily disable it just as you can easily disable many other things that Firefox includes by-default >it would be nice if Firefox actually packaged Pocket as a real extension that could be removed from the Extensions menu These two statements you made seem to corroborate his characterization of it being "opt-out" (meaning on by default, but capable of being disabled) and "non-removable" (baked into firefox as opposed to an extension). Not sure what you find wrong with his characterization.
- halosghost 11y ago
  “opt-out” is certainly correct (though, as I understand it, Pocket, as with all parts of Firefox, is only loaded when it is actually used, so “opt-out” does not seem to tell the whole story to me). I do disagree with the “non-removable” bit. Fully disabling it, to me, counts as removable (though I can understand why someone would disagree). Perhaps I just read something into the Author's tone (probably due to all the vitriole from the typical discourse around the integration) that wasn't really there. If that's the case, and all the author meant by that statement was that the code itself could not be completely erased from Firefox as-packaged, then that seems to be factually true, and I simply read it incorrectly.

pdkl95 11y ago

This is the problem with services that store user information: it is highly probably that vulnerabilities like these exist. Security is rarely given the time and attention it requires. I'm not trying to single out Pocket; they are just the latest evidence that even in the few cases where "you can trust us with your data" is said honestly, it isn't a promise that can be kept in practice.

luxoria 11y ago

>Grab ssh private keys from autoprovisioned EC2 user’s home directory using 301 redirect to file URI (after all, we’re running as root, we can read them). This is not a fair assumption to make. Maybe they are running a LSM like AppArmor.

skarap 11y ago
What is more important - the existence of SSH private keys on the EC2 instances us unlikely... There is chance there are SSH private keys there, but they would most probably be SSH deploy keys for some private repo (configuration management, software).

BenjaminWill 11y ago

Oh Mozilla, why couldn't you resist the money. Your recent so called "services" are not welcome. You know it. But well, money makes the world go around. How much did Telefonica pay you for the Hello integration? But sure, our surfing history will be secure ... https://blog.mozilla.org/advancingcontent/2015/05/21/providing-a-valuable-platform-for-advertisers-content-publishers-and-users/ https://blog.mozilla.org/advancingcontent/2015/05/21/providi... Did you guys acutally read your PR-bullshit here? But soon a new small, fast, free, secure open-source browser will arrive and Mozilla will be history. But your pocket full. Well done.

lucian1900 11y ago
> a new small, fast, free, secure open-source browser will arrive Really? How exactly? Who will fund its development?
- nivla 11y ago
  One way would be to keep forking Firefox's codebase for bug fixes and new features while customizing it to fit the needs of the privacy conscious users. It may not be "ethical" but it will serve the main premises of open source and is also doable as a community. A new name would also be required as Firefox is trademarked by Mozilla (See Iceweasel Browser [1]). [1] https://wiki.debian.org/Iceweasel https://wiki.debian.org/Iceweasel
- brongondwana 11y ago
  Jon von Tetzchner. He's got a team of experienced browser devs and fuck-you money from selling his last browser company.
rntz 11y ago
Never attribute to malice what can be adequately explained by incompetence. Mozilla almost certainly isn't getting any money out of the Pocket integration deal. Mozilla has far more money than Pocket does.
reitanqild 11y ago
FWIW: Reading this on Pale Moon which is basically a rebranded fork of Firefox before Aurora. Can't say anything about PaleMoons security but I like the old design better, the team seems more responsive and they haven't yet (had the chance to) give in to hypocrites and fire "Brendan Eich".

hundt 11y ago

Security researchers out there: on what side of "the line" do you view this kind of exploitation to be? It was not done for nefarious purposes, but it did involve intentionally accessing resources that were clearly not intended to be accessed, like /etc/passwd. Would you worry if you did this that the company might call the police instead of thanking you?

schmichael 11y ago

Is there a name for this sort of attack? We were just protecting against some similar attacks earlier this week, and it would have been nice to have a short name to refer to them as instead of "that attack vector where we make unrestricted HTTP requests based on user input".

Manishearth 11y ago
"Server Side Request Forgery" (SSRF)

skarap 11y ago

Yet another service discovered which was built/deployed with no regard for security whatsoever. I'm beginning to realize - this is the norm. Security is the least important thing for most of the IT companies. I guess the DevOps trend (i.e. not hiring sysadmins) should take it's share of blame. Or maybe it's the other way around - you don't care for security, so there is no point in hiring security experts?

mmatants 11y ago
It's hard to get an intuitive feel for security. If team principals are mostly young and don't have exposure to very specific kind of experiences dealing with threat modelling, then the security work will always take a backseat to feature-building. It's how humans work - it's hard to cut time from nice visible things and dedicate it to a hypothetical (at that point) abstract goal like security. Everyone agrees that the latter is important, but it just never ends up getting any oxygen. What's worse, if the team does not know what tightening systems feels like, they don't even know where to start. No "muscle memory" for it. Maybe at some point we'll get more baseline collective wisdom about it throughout the industry, but it will also take the people signing the cheques (CEO, investors) having a bit more (justified) paranoia and respect for these priorities, and consequently requiring them from the outset. And DevOps has little to do with it - security has to be established as a priority from the leadership levels on down anyway because it necessarily means reducing time spent on more immediately-visible things.
- skarap 11y ago
  I agree but partially. We are not talking about some complex interactions between multiple components which lead to a security vulnerability. This is some trivial stuff like "don't give your passwords to anybody" or "don't run everything as root". The most complex vulnerability mentioned in the article is with proxying. If you have opened /etc/squid/squid.conf at least once you should have noticed the to_localhost ACL and the comments which explain why it is important. So is the Pocket team building a multi-million user service which has a proxying component without trying to configure squid once? Absolutely! Also I consider too optimistic hoping for the situation to improve - it's moving in the opposite direction for now with steady speed.

Iuz 11y ago

Is it ironic that I saved the article on pocket?

dafrankenstein2 11y ago

i prefer using offline bookmark..specially the bookmark manager of Opera browser is impressive. ============================================= and for online bookmarking there is 'raindrop.io'

dafrankenstein2 11y ago

i prefer using offline bookmark..specially the bookmark manager of Opera browser is impressive. ============================================= and for online bookmarking there is 'raindrop.io'

_navaneethan 11y ago

You should receive some gift from pocket team :)

robn_fastmail 11y ago

If you want a laundry list of SSRF methods you should protect against, a great place to start is this slide deck from a talk at ONsec a couple of years ago: http://www.slideshare.net/d0znpp/ssrf-attacks-and-sockets-smorgasbord-of-vulnerabilities http://www.slideshare.net/d0znpp/ssrf-attacks-and-sockets-sm... Its thrilling and terrifying :)