14 ms·
Remote code execution vulnerability in apt/apt-get
- deleted 8y ago[deleted]
- kuhhk 8y ago> Thank you to the apt maintainers for patching this vulnerability quickly, and to the Debian security team for coordinating the disclosure. This bug has been assigned CVE-2019-3462. From the article. Good stuff
- aboutruby 8y ago> a malicious mirror could still exploit a bug like this, even with https. But I suspect that a network adversary serving an exploit is far more likely than deb.debian.org serving one or their TLS certificate getting compromised Exactly, this is far more easily exploitable because apt is using HTTP by default instead of HTTPS
- rabi_penguin 8y agoHmm, it's almost as if the author of https://whydoesaptnotusehttps.com/ https://whydoesaptnotusehttps.com/ may have overlooked a few things.
- loeg 8y agoYou know that HTTPS has the Location header too, right? This attack works just as well against HTTPS clients.
- jamp897 8y agoHe never explains why he wants to use HTTP, it’s only about why he thinks HTTPS isn’t nesscary.
- AndyKelley 8y agoHTTP is the null hypothesis, since it's simpler. Usually there is a great reason to reject this null hypothesis - it prevents security vulnerabilities. But if there is no added value, then there is no reason to do it. Consider, why not double-wrap your stream? Put TLS on top of TLS on top of HTTP?
- jamp897 8y agoYet it’s actually not simpler for the user, since their transfer can then be tampered with either by accident or intentionally leaving the user with a broken download and then what do they do? A redownlaod from a different mirror makes no difference.
- AndyKelley 8y agoThe situation you described is the same thing that happens with a MITM attack with HTTPS. You would get a failed download from any mirror. Do you have a response to my question? "Consider, why not double-wrap your stream? Put TLS on top of TLS on top of HTTP?"
- nneonneo 8y agoBecause that just makes things slower for no good reason?
- loeg 8y agoSounds like an argument for rejecting HTTP+TLS single-wrap too. (For apt — not in general.)
- nneonneo 8y agoI was being glib because I didn’t think I needed to explain fully, but here we go. Double-encrypting something with the same technique is pretty much always a sign of cargo cult crypto. Modern ciphers, like those used by TLS, are strong enough that there’s no reasonable way to break them applied once, and the downside is that applying them twice is making things slower than they need to be for zero added benefit. On the other hand, TLS and PGP are very different things serving very different purposes, so nesting those makes sense. There is an added benefit from TLS, namely that you ensure that everything is protected in transit - including the HTTP protocol itself (which is currently not protected and which might be subject to manipulation as shown in this post). Plus, it provides some resistance to eavesdropping (and with eSNI + mirrors hosted on shared hosts, that resistance should improve further).
- LoSboccacc 8y agoalso, the apt way to fix this would be to a) move release.gpg out of the package path and b) require the release.gpg to be wrapped and signed with the previous valid key instead of being accepted blindly
- guest2143 8y agoSome country's firewalls distrupt https, which makes downloading things via https difficult.
- jamp897 8y agoWhich countries? I’ve only seen HTTP connections tampered with in practice, and China’s GFW blocks HTTP no different than HTTPS from what I’ve seen.
- DoctorOetker 8y agoso if north korea is subjugating some poor souls over there, the whole world must suffer along? there could be a setting with a big warning to disable the default HTTPS behaviour...
- tinus_hn 8y agoSo you create an non default http mirror for that minority, instead of making the majority insecure.
- eeZah7Ux 8y agoAlso some companies, to allow IDS to inspect traffic without having to extract keys from clients.
- moviuro 8y agoOTOH, they would have been right if there had been (yet another) bug in openssl/whatever lib would be used for https. FWIW: 16 vulns in apt in NVD [0]; but 202 for openssl [1] [0] https://nvd.nist.gov/vuln/search/results?form_type=Advanced&results_type=overview&search_type=all&cpe_vendor=cpe%3A%2F%3Adebian&cpe_product=cpe%3A%2F%3A%3Aapt https://nvd.nist.gov/vuln/search/results?form_type=Advanced&... [1] https://nvd.nist.gov/vuln/search/results?form_type=Advanced&results_type=overview&search_type=all&cpe_vendor=cpe%3A%2F%3Aopenssl&cpe_product=cpe%3A%2F%3A%3Aopenssl https://nvd.nist.gov/vuln/search/results?form_type=Advanced&...
- jwilk 8y agoHow many of these would result in RCE?
- dijit 8y ago64? https://nvd.nist.gov/vuln/search/results?form_type=Advanced&results_type=overview&query=RCE&search_type=all&cpe_vendor=cpe%3A%2F%3Aopenssl&cpe_product=cpe%3A%2F%3A%3Aopenssl https://nvd.nist.gov/vuln/search/results?form_type=Advanced&...
- detaro 8y agoFulltext search for "rce", which finds "resou_rce_", "sou_rce_", does not give a number of RCE vulnerabilities.
- dijit 8y agoExcept that "EXACT MATCH" is enabled, try yourself. (It should be noted that it /does/ match on "possible RCE", which buffer overflows are often tagged with.)
- detaro 8y agoI have. Search for "ontgome" and it finds the bugs containing "Montgomery" (I have taken your url and just replaced the search word): https://nvd.nist.gov/vuln/search/results?form_type=Advanced&results_type=overview&query=ontgome&search_type=all&cpe_vendor=cpe%3A%2F%3Aopenssl&cpe_product=cpe%3A%2F%3A%3Aopenssl https://nvd.nist.gov/vuln/search/results?form_type=Advanced&... I'm not saying none of the results from your search are RCEs, but not all are, and many are fairly speculative.
- jwilk 8y agoDiscussed yesterday: https://news.ycombinator.com/item?id=18958679 https://news.ycombinator.com/item?id=18958679
- kerng 8y agoMakes me chuckle, since my comment on the HN post yesterday that highlighted that someone will be bitten if security principles are ignored got downvoted.
- black_holed 8y agoWell, they didn’t fall into the black hole of “moderation” at least...
- ru999gol 8y agoI was laughing as well, software people (the HN and open source community in particular) are so stupidly opinionated sometimes, it makes me just cringe whenever its impossible to ignore their dribble. There really is no excuse not to use HTTPS in 2019, period.
- spuz 8y agoI think I understand the exploit but I don't understand whether apt using https would prevent it or not. The author says: > Yes, a malicious mirror could still exploit a bug like this, even with https. and: > I wouldn’t have been able to exploit the Dockerfile at the top of this post if the default package servers had been using https. So which is it?
- hannob 8y agoHTTPS: A malicious mirror operator can pwn you. HTTP: Everyone can pwn you. Not saying the first one is ideal, but the second one is definitely worse.
- trulyrandom 8y agoWith HTTP an attacker still has to MITM the connection between you and the mirror operator. So, definitely not "everyone".
- DoctorOetker 8y agothe moment we are talking about monitoring user's software base, we are practically already talking attackers at the skill level of nation states, so yeah "everyone" in the subset of plausible attackers.
- cryptonector 8y agoThat includes: coffee shops, ISPs, employers, everyone who can hack their routers, anyone who can spoof DNS, etc. That might as well be "everyone". STOP IT. Though shall use HTTPS.
- trulyrandom 8y agoFair enough. I agree that HTTPS is valuable here. I was just being overly pedantic, my bad.
- detaro 8y agoBoth. Without HTTPS, you can execute the attack if you can MITM the connection to the package repository. If HTTPS is used, you need to be the package repository to do the attack, or need a certificate to MITM the connection so you can pretend to be it.
- _wmd 8y agoEvery time this site comes up people entirely miss the point in this regard -- Debian operates a large voluntary network of mirrors. You are not trusting content coming from Debian, you're trusting it coming from the mirror. SSL only secures the link between the client and the potentially compromised mirror, it does not solve problems like the one from the article. Meanwhile it's worth pointing out that OpenSSL has historically been one of the buggiest pieces of code in existence. Despite this being a game over RCE, it's the first of its kind in many years. If OpenSSL had been in the mix, Apt would have required forced upgrades /far/ more often. https://www.openssl.org/news/vulnerabilities.html https://www.openssl.org/news/vulnerabilities.html
- raesene9 8y agoI don't think that an argument that using HTTPS actively decreases the security of a connection really holds all that well. If you don't think OpenSSL is a high enough quality implementation, there are many others to choose from. Even with a range of mirrors, it would still raise the bar for attackers, to require HTTPS.
- all_blue_chucks 8y agoThat's because security requires defense in depth. If the failure of a single security control can invalidate your security model, your security model is inadequate. It should require multiple things to go wrong for catastrophic failure. This is a lesson from engineering that hasn't made its way to software development yet (outside of security engineering, anyway).
- whydoyoucare 8y agoI believe several terms here need a context, without which they are meaningless. For example, what is an acceptable security model to download and install an OS, how do you exactly define "defense in depth" for the act of downloading and installing an OS, etc.. That will help us define what controls should be in place for the overall OS-download-and-install experience to be secure.
- all_blue_chucks 8y agoDefense in depth is just an industry term for redundant security. For example, you can mitigate tampering with data transfers by signing the data itself AND ALSO by signing the channel it's transferred over with TLS. If a flaw is found in one of those methods, the other will still protect you. The process of listing all the security failure points and documenting the redundant mechanisms to protect them is called threat modeling. For a system that installs OS-level binaries as root, it would absolutely be appropriate to threat model it and hold it to a defense in depth standard. In defense systems, they often require three levels of defense in depth, the last being an air gap network.
- whydoyoucare 8y agoYes, this works. In theory. In practice, a simple model is needed that everyone can follow and implement consistently. That, does not exist. Lookup "threat modeling" and you will see how abstract a notion it is (even your comment calls for a "redundant mechanism" that may not be exactly what you are looking for), and how little information is available. End result? Most do it for the "checkbox effect". Don't get me wrong, I am not trying to obliterate what you said, just putting some factual data around it.
- squirrelicus 8y agoI was just coming here to post that
- snek 8y agoI will never understand why people are so proud to have gotten rid of "needless" security. It always ends the same way...
- lvs 8y agoThey (ubuntu, at least) have always been overtly hostile about this issue, which seems strange. I recall seeing open issues that just devolved into argument for many years on this. I don't get it.
- feikname 8y agoSeems like the discovery of this vuln was a direct result of yesterday's discussion about HTTPS on apt here on HN (https://news.ycombinator.com/item?id=18958679 https://news.ycombinator.com/item?id=18958679).
- kpcyrd 8y agoIt seems debian testing and unstable are still vulnerable: https://security-tracker.debian.org/tracker/CVE-2019-3462 https://security-tracker.debian.org/tracker/CVE-2019-3462
- ChrisSD 8y agoYep, this is our daily reminder that they're called "testing" and "unstable" for a reason. They're not meant for production.
- gtirloni 8y agoNew fixes often land in unstable/testing before going to stable branches.
- jwilk 8y agoFixed apt has been just uploaded to unstable: https://packages.qa.debian.org/a/apt/news/20190122T190408Z.html https://packages.qa.debian.org/a/apt/news/20190122T190408Z.h...
- jwilk 8y agoIt's now in testing, too.
- kanox 8y agoI was expect a buffer overflow but this is a quoting issue which is equally applicable to all languages. Are the any language-level methods which make such bugs impossible (or much harder)? Heavy-weight strongly-typed HTTP libraries can force you to always construct headers in a way that handles quoting for you but people seem to love "light" solutions.
- devit 8y agoUsing a binary protocol library for RPC (e.g. gRPC, raw protobufs, etc.) rather than writing your own text-based protocol incorrectly.
- aasasd 8y agoThe method is to treat the protocol as structured data instead of a bunch of concatenated text. I.e. use a library that processes each piece of data, escaping any invalid characters, before putting the pieces together. Strong typing isn't relevant here, this is applicable in any language. But the lib needs to know when you're putting text in a header name and when in the value.
- Sephr 8y agoUsing HTTP for apt may seem bad, but you should really pay attention to Ubuntu itself: https://github.com/canonical-websites/www.ubuntu.com/issues/4394 https://github.com/canonical-websites/www.ubuntu.com/issues/... Ubuntu ISOs aren't served securely and are trivially easy to MitM attack. This vulnerability is still being exploited: https://www.bleepingcomputer.com/news/security/turkish-isp-swapped-downloads-of-popular-software-with-spyware-infected-apps/ https://www.bleepingcomputer.com/news/security/turkish-isp-s...
- nyolfen 8y agouse torrents
- Sephr 8y agoThe official torrent links are also distributed over insecure HTTP and are also trivially easy to MitM attack.
- btown 8y agoThat's ridiculous. https://www.ubuntu.com/download/alternative-downloads https://www.ubuntu.com/download/alternative-downloads There is no technical excuse for a torrent file to be distributed over HTTP.
- nneonneo 8y agoUgh couldn’t they just put magnet links on their nice HTTPS site? It sounds like they just really don’t want to have to update their site when new releases come out.
- peterwwillis 8y agoAm I crazy, or is the bigger problem here not the fact that Apt will just install whatever random package the server provides, whether your system trusts its GPG key or not? What the hell is the point of the keys if the packages are installed anyway??
- thriqon 8y agoUsually, the packages themselves are not signed with GPG, only the Release file is (containing the hashes of all .deb files). This is actually the default of both Debian and Ubuntu. I never quite understood the reasons behind it... I'd not expect this vuln to happen, though. More info: https://blog.packagecloud.io/eng/2014/10/28/howto-gpg-sign-verify-deb-packages-apt-repositories/ https://blog.packagecloud.io/eng/2014/10/28/howto-gpg-sign-v...
- peterwwillis 8y agoI kind of get why they did it that way (my guess: managing lots of dev keys was problematic, so they used one key to sign a list of "official-seeming" files). But then why wasn't the Release file's contents verified (assuming this doesn't involve generating collisions for those packages)? "The parent process will trust the hashes returned in the injected 201 URI Done response, and compare them with the values from the signed package manifest. Since the attacker controls the reported hashes, they can use this vulnerability to convincingly forge any package." Wtf? This sounds like Apt is just downloading a gpg file and checking if it matches a hash in an HTTP header, and if it does, it just uses whatever is specified, regardless of whether your system already had the right key imported? This makes no sense. Any mirror could return whatever headers it wanted. This is the real vuln, not header injection. If Apt isn't verifying packages against the keys I had before I started running Apt, there was never any security to begin with. An attacker on a mirror could just provide their own gpg key and Releases file and install arbitrary evil packages. Can somebody who knows C++ please verify that their fixes actually stop installing packages if the GPG key wasn't already imported into the system? https://github.com/Debian/apt/commit/690bc2923814b3620ace1ffcb710603f81fa217f https://github.com/Debian/apt/commit/690bc2923814b3620ace1ff...
- jamieweb 8y agoAll my servers do an update and dist-upgrade every 24 hours, and it emails me the log. I saw this post just a few minutes after checking the log for today. I imagine that this is a higher risk for virtualized servers in a public cloud. I use Linode, so somebody else could have set up a Linode to MITM everybody and serve the exploit. If it were a private home or corporate network, somebody would either have to be on your network, or on a piece of major infrastructure between you and the mirrors. Is there a way to tell from the apt log whether I am affected? It looks like you can see it trying to install an extra dependency package. Anyway, the logs are not immutable or verifiable, so if somebody got root they could theoretically kill apt, write a fake log in its place and then email that to me... I took full images of all my servers a few days ago, so at least I have those should I need them.
- perennate 8y ago> I imagine that this is a higher risk for virtualized servers in a public cloud. I think it might be the other way around (at least in terms of virtualized servers versus physical servers, both on public cloud) -- it is easier to implement IP address and other filtering measures with virtualized servers than inside physical network switches. Linode and other virtual machine providers almost universally implement this filtering, but many dedicated server providers are not as robust.
- jamieweb 8y agoThat's a good point actually - although when using dedicated hardware I usually have in my mind that everything is raw rather than abstracted by a hypervisor, so this sort of thing should be more expected. With a public cloud you don't really know how it's set up on their end, as there are countless different ways to do it.
- mehrdadn 8y agohttps://news.ycombinator.com/item?id=16224684 https://news.ycombinator.com/item?id=16224684
- ChrisCinelli 8y agoIncidentally binaries from support.apple.com are also served on http.
- aboutruby 8y agoThe downloads are in an HTTPS page leading to HTTPS download links, and HTTP redirects to HTTPS: https://support.apple.com/downloads/quicktime https://support.apple.com/downloads/quicktime for instance.
- ptx 8y ago> and HTTP redirects to HTTPS ...or to anywhere a MITM attacker wants to redirect you.
- ptx 8y agoI guess some people disagree? But if any response in the redirect chain is served over HTTP, it can be replaced with a different response containing any "Location" header of the attacker's choosing instead of the original one. So it doesn't matter if the eventual intended URL is an HTTPS URL, because it will never be reached. The redirect will go to the attacker's site instead. (And in the case of a download, the user will never notice because the file URL is usually not prominently displayed.) So a HTTP response anywhere in a redirect chain is equivalent to serving it over HTTP. Perhaps this was exactly the point of the parent post, but I thought it would be useful to make it explicit.
- aasasd 8y agoI'm just wondering if the author should've given people more time to pull the updated apt, before publishing the issue. This is only a few days old, right? I'm not familiar with established procedures in such cases, and am curious about the rationale for omitting a time window for the update.
- pfg 8y agoPublic disclosure once patches are available is a fairly common policy. Google's Project Zero operates like that as well.
- nisa 8y agoFWIW you can have https today just install apt-transport-https and use this sources.list: deb https://deb.debian.org/debian stable main contrib non-free deb https://deb.debian.org/debian-security stable/updates main contrib non-free deb https://deb.debian.org/debian stretch-backports main contrib non-free
- wtetzner 8y agoDo you have to install apt-transport-https over http?
- richardwhiuk 8y agoNo, but it might be a pain not to: wget https://cdn-aws.deb.debian.org/debian-security/pool/updates/main/a/apt/apt-transport-https_1.4.9_amd64.deb https://cdn-aws.deb.debian.org/debian-security/pool/updates/... dpkg -i apt-transport-https_1.4.9_amd64.deb apt-transport-https is going away in apt 1.5 anyway
- explainplease 8y ago$ sudo sed -i 's/http:/https:/g' /etc/apt/sources.list $ sudo apt-get update ... Err https://us.archive.ubuntu.com trusty/main Sources Failed to connect to us.archive.ubuntu.com port 443: Connection refused ... Err https://security.ubuntu.com trusty-security/main amd64 Packages Failed to connect to security.ubuntu.com port 443: Connection refused ??? $ curl -v https://security.ubuntu.com/ubuntu/ * Hostname was NOT found in DNS cache * Trying 91.189.88.149... * connect to 91.189.88.149 port 443 failed: Connection refused * Trying 91.189.88.152... * connect to 91.189.88.152 port 443 failed: Connection refused * Trying 91.189.88.161... * connect to 91.189.88.161 port 443 failed: Connection refused * Trying 91.189.88.162... * connect to 91.189.88.162 port 443 failed: Connection refused * Trying 91.189.91.23... * connect to 91.189.91.23 port 443 failed: Connection refused * Trying 91.189.91.26... * connect to 91.189.91.26 port 443 failed: Connection refused * Failed to connect to security.ubuntu.com port 443: Connection refused * Closing connection 0 curl: (7) Failed to connect to security.ubuntu.com port 443: Connection refused So even security.ubuntu.com is unavailable over HTTPS? Am I missing something?
- deathanatos 8y agoAs was discussed recently on HN (and linked to elsewhere in the comments for this article), packages are signed, and APT checks those signatures; however, APT does download both the packages and the signatures in the clear. So, normally, the signatures get checked, which ensures that you get the package you intended. This is fine, mostly. (If you don't care about privacy, but it does prevent tampering, normally.) APT does not, however, give privacy, which HTTPS/TLS would. (Those in favor argue that TLS doesn't help here, as you can still see that you're connecting to Ubuntu, so it's still obvious that you're downloading updates. I personally disagree w/ this stance: I think there is value in protecting which packages you're pulling updates for, as what packages you have installed can inform someone about what you're doing. I think there's further argument that the sizes of the responses give away which updates you're pulling, but IDK, that seems harder to piece together, and TLS at least raises the bar for that sort of thing.) The bug discussed in the article circumvents the signature checking, by lying to the parent process about the validity of the signature by being able to essentially execute a sort of XSS/injection attack.
- systematical 8y agoSo will someone more in the know than myself tell me if using apt-transport-https is a reasonable solution to this problem, or, at least mitigates the problem? As someone who uses Linux as their personal O.S. and administers some at work, but doesn't think in bash, I'd really like an answer :-)
- kelnos 8y agoIt will mitigate the possibility of someone MitM'ing your connection to the package mirror. It won't protect you against a malicious package mirror.
- moosingin3space 8y agoIt makes it so that your mirror would have to be exploiting apt, instead of effectively anyone. As a result, using TLS for downloads would mitigate this (but not fix it). I've read the arguments against HTTPS for apt many times. They're wrong.
- whydoyoucare 8y agohttps://www.openbsd.org/papers/bsdcan-signify.html https://www.openbsd.org/papers/bsdcan-signify.html is a good read on this topic.
- Kliment 8y agoIf this were one of those bugs that do the full PR bullshit run and use a catchy name and a landing page, I'd propose it be called "Inapt"
- ptx 8y agoSo how do I safely update apt to the patched version on Debian Jessie? It's apparently fixed in version 1.0.9.8.5: https://security-tracker.debian.org/tracker/CVE-2019-3462 https://security-tracker.debian.org/tracker/CVE-2019-3462 ...but the suggested apt -o Acquire::http::AllowRedirect=false update fails because security.debian.org wants to do a redirect. Manually downloading the packages listed in the announcement doesn't work either, since that's the Stretch version. I can get the source package here: https://packages.debian.org/jessie/apt https://packages.debian.org/jessie/apt ...but the key is not in /usr/share/keyrings/debian-archive-keyring.gpg. (And I'm not entirely sure how to build a source package.) I tried adding a different source, as suggested in the announcement: deb http://cdn-fastly.deb.debian.org/debian-security stable/updates main ...but it seems it can't find the right versions of all the dependencies: # apt -o Acquire::http::AllowRedirect=false install apt apt-utils libapt-pkg5.0 libapt-inst2.0 liblz4-1 ... The following packages have unmet dependencies: libapt-pkg5.0 : Depends: liblz4-1 (>= 0.0~r127) but 0.0~r122-2 is to be installed E: Unable to correct problems, you have held broken packages.
- jwilk 8y agoNo idea why would would security.debian.org make a redirect. Can you paste the whole error message? Regarding the other error, jessie ≠ stable. You want "jessie/updates", not "stable/updates".
- ptx 8y agoAh, silly blind copy-pasting on my part. With "jessie/updates" it works. Thanks! The message I was referring to, which looks like it's indicating that security.debian.org is trying to redirect: # apt -o Acquire::http::AllowRedirect=false update ... Err http://security.debian.org jessie/updates/main Sources 302 Found [IP: 217.196.149.233 80] Err http://security.debian.org jessie/updates/main amd64 Packages 302 Found [IP: 217.196.149.233 80] Err http://security.debian.org jessie/updates/non-free amd64 Packages 302 Found [IP: 217.196.149.233 80] Err http://security.debian.org jessie/updates/main i386 Packages 302 Found [IP: 217.196.149.233 80] Err http://security.debian.org jessie/updates/non-free i386 Packages 302 Found [IP: 217.196.149.233 80] Fetched 422 kB in 2s (169 kB/s) W: Failed to fetch http://security.debian.org/dists/jessie/updates/main/source/Sources 302 Found [IP: 217.196.149.233 80] W: Failed to fetch http://security.debian.org/dists/jessie/updates/main/binary-amd64/Packages 302 Found [IP: 217.196.149.233 80] W: Failed to fetch http://security.debian.org/dists/jessie/updates/non-free/binary-amd64/Packages 302 Found [IP: 217.196.149.233 80] W: Failed to fetch http://security.debian.org/dists/jessie/updates/main/binary-i386/Packages 302 Found [IP: 217.196.149.233 80] W: Failed to fetch http://security.debian.org/dists/jessie/updates/non-free/binary-i386/Packages 302 Found [IP: 217.196.149.233 80] E: Some index files failed to download. They have been ignored, or old ones used instead.