12 ms·
Detecting the use of "curl | bash" server-side
- jamescostian 8y agoThis is immune to the attack: bash -c "$(curl -sSLf $URL)" The key is to download first and then run
- Sir_Cmpwn 8y agoWhy not start with checking the user agent?
- Waterluvian 8y agoSo yes, curl bash can be dangerous. But it's just so darn convenient. And when it's coming from a very prominent trusted source like for Get Pip or Amazon AWS it's hard not to just go with. Surely there's some compromise middle ground? Let me download "safe-curl-bash" (scb) that only runs a script if it's trusted in some manner? Maybe the checksum matches from a crowdsourced database. "Sorry only 9 people have declared this script valid and your threshold is 100. Here's a cat of the script and we will ask you if it looks valid or not or don't know." I also think it's a bit more realistic than the, "anyone who does this should be reading the script first to check that it's safe." Yes, and I check the passenger jet for flaws before I board, too! Just spitballing.
- arminiusreturns 8y agoI think the arch user repo does this correctly by asking, "do you want to edit the packagebuild?" You can just press n to accept defaults, or y to take a look under the hood and make edits before continuing.
- smacktoward 8y ago> Yes, and I check the passenger jet for flaws before I board, too! There is an entire infrastructure of people and processes in place to make sure that you don't have to check your passenger jet for flaws to be reasonably sure it's safe. No such infrastructure exists to protect you from the consequences of curl-bashing software off some random Web site.
- Waterluvian 8y agoYes that's fair. My point is that most people using these scripts have nowhere near the expertise to be able to evaluate their safety.
- true_religion 8y agoWho are these "most people". I've never seen curl | bash outside of developer circles. "Most" people install things directly from the app store, or use web apps.
- ghthor 8y agoGood use case for IPFS. You could curl a hash and check that hash against signatures commited to a blockchain. If the set of signatures exist that you trust, then it's golden, if not, no exec.
- gant 8y agoIf by "signatures committed to blockchain" you mean git with a GPG signed tip/tag, then yes, sure, blockchain. If you mean that other thing, no.
- cjbprime 8y agoNeat! But it's not obviously a bad idea. You have a TLS connection with the site you're downloading from. `curl | bash` is no worse than downloading a .dmg or .deb from the same server would be.
- mikeash 8y agoThe difference is that you can inspect it before you run it if you download it. If you pipe it into bash you don’t know what you’re getting, even if you previously inspected the data provided by that URL.
- staticassertion 8y agoI don't feel the need to review the source code for every install script I run. I don't read the source code for almost any of the code on my machine today. In most cases where I see `curl | bash`, I'd probably already be screwed even if I review it. Most install scripts and up doing "hit website, install thing" anyways - am I reviewing the second stage install script also?
- mikeash 8y agoThat’s fair. It’s probably not an important difference.
- cjbprime 8y agoThat's a way in which "curl | bash" distributed software is better than .deb/.dmg distributed software, right? Because you have the potential to inspect the script first, if you have some kind of ridiculous confidence in your ability to perform security review of an entire software product in the moments before you decide to install it. But it's never presented in that way, as a feature. It's presented as a terrible way to distribute software.
- twr 8y agoIt doesn't take ridiculous confidence to analyze shell scripts. In the hundreds of scripts I have read, few were more than 100 lines long. It shouldn't take more than 60 seconds (probably 30 or less) to mentally build a list of all possible operations a short script can perform. Bourne shell scripts don't have much room to hide surprising behavior, and when they do, it immediately stands out. If they are permanently installed, and invoked later by other parts of the system, then they may need more probing, but we're talking about installation scripts. .deb and .dmg can be easily extracted. The former is just an `ar` archive containing tarballs, which you can (and should) extract to read the install scripts. (.dmg specifics escape me, since I only dealt with them one time, years ago.) Binary code isn't inscrutable. Some good tools for this are, among many, many more, IDA, Hopper, and radare2. How long this takes depends on what your goals are, how comprehensive you are, and the program complexity. I don't think I've yet spent years on one project, fortunately, but the months-long efforts, for undoing some once-prominent copyright protection systems, were pretty brutal. Smaller programs have taken me just several hours to appropriately examine.
- megaman22 8y agoIs there no way to sign scripts with a code signing certificate? I certainly have to do some hoop jumping to execute the equivalent of curl |bash on powershell.
- michaelmior 8y agoOn the evading detection side, one other simple way to avoid this is to add sponge[0] between curl and bash in the pipeline, i.e. curl ... | sponge | bash. sponge consumes all input until EOF before outputting anything, stopping bash from executing a partially downloaded script. [0] https://linux.die.net/man/1/sponge https://linux.die.net/man/1/sponge
- skunkworker 8y agoIf you're on a Mac or a system that doesn't have sponge installed by default, use moreutils to install. https://joeyh.name/code/moreutils/ https://joeyh.name/code/moreutils/ https://rentes.github.io/unix/utilities/2015/07/27/moreutils-package/ https://rentes.github.io/unix/utilities/2015/07/27/moreutils...
- mehrdadn 8y agoThanks, this is helpful!
- andrewstuart2 8y agoJust curl it to tee or redirect to a file and you know it won't change before you execute the script file. There's nothing stopping somebody from even more trivially just sending each IP a benign script once (per curl user agent) and a malicious script the second time. Putting it in a file and executing the file brings it entirely into your domain of control.
- deleted 8y ago[deleted]
- h000per 8y agoRegardless of the installation method it sounds like we need to be running all applications in their own individual virtual machines (e.g. Qubes OS) or within a restricted environment with limited permissions (iOS)
- geggam 8y agoHow do you install the virtual machine software ? Where do you put the trust ?
- taeric 8y agoWorse, what happens when I do want the applications to communicate? An amusing gotcha I found with docker was how do I convince the servers I communicate with from in the container that I am me? Best bet was to map my user into the user on the container, but that was actually ridiculously fraught with trouble. (There is a chance this has since been fixed...)
- chii 8y ago> I do want the applications to communicate? QubeOS adopted the "manual authentication" method (of having to confirm everything, such as clipboard copy/paste). This is probably not quite scalable (not to mention annoying). May be there's some way to have a short session token, so during a work session of a few hours, it works without any intervention.
- taeric 8y agoThe problem came when I wanted the app to communicate to another on behalf of me. Do I have to constantly reconfigure an openid connection for every app on my machine? (Not the worst of ideas, I suppose...)
- curlpypshellrip 8y agoHalf the problem with `curl | bash` installation is not related to whether or not you trust what your downloading... The more important reason why it is a _horrible_ _stupid_ mechanism for software installation is that it is not _repeatable_. It is well understood that casual .deb .rpm usage requires an equivalent level of trust as downloading anything else off the internet... but they have the added advantage of being _consistent_ _repeatable_ and _mirrorable_... I can copy the entire repository of any version of debian I want to my local file server, and use that to spin up however much infrastructure I want. And the only person I need to rely on after I have fetched the initial packages is myself.
- kccqzy 8y agoPlenty of those scripts simply detect your operating system and then interact with the system package manager (adding a new repository, updating the package index, then asking the package manager to install it).
- curlpypshellrip 8y agoI would far prefer that software projects assumed competence in the installers knowledge of their systems package manager and just listed the specifics... honestly how hard is it to just say "here is our $gpg key" "here is our $apt_repo_url" the package name is "$foobar". And let competent system admins take it from there. Most of the time you end up de-composing the scripts into ansible/$sys_automation_tool_of_the_week anyhow.
- enos 8y agoBecause your $apt_repo_url is useless to a yum user. The package management community needs to stop balkanizing. How else do you propose releasing software to all Unix-like OSes without learning a half dozen package manager formats and quirks? Only other choice is a container.
- geggam 8y agoLet's say for discussion that we allow this curl| bash process because it is from a "safe" source. How do we come back next week and ensure some other process hasn't changed the files ? Package your files with a signed system. Auditing the files is trivial after that.
- deleted 8y ago[deleted]
- kazinator 8y agoYou could just have the script detect that its stdin is a pipe. E.g., Linux specific: $ echo 'ls -l /proc/$$/fd/0' | bash lr-x------ 1 kaz kaz 64 Jul 28 21:03 /proc/23814/fd/0 -> pipe:[4307360] Here, our script consists of the ls command; it shows that when we pipe it to bash, it finds fd0 to be a pipe. We can make some code conditional on this to produce a "don't run this script from a pipe" diagnostic. This is superior to the dodgy, delay-based server side detection because it is reliable. Also, it still works when someone does this: $ curl <url> > file $ cat file | bash Of course, no protection for $ bash file
- ericpauley 8y agoThis logic would be detectable to a user who reads the script. The goal here is to trick users who first inspect the script and then `curl | bash`
- lxe 8y agoRecent Chrome Canaries show ERR_CERT_SYMANTEC_LEGACY on this host. Gotta refresh those TSL certificates before September!
- dev_dull 8y agoThe obsession against shell pipes is so absolutely absurd. You’d download a dmg and drag it to your apps but not shell pipe? You’ll sudo dpkg -i but not a shell pipe? Can anyone point to a single case of a shell pipe ever being abused ever?
- throwbacktictac 8y agoIf a tree falls in a forest and no one is around to hear it, does it make a sound? I certain that someone has been exploited using shell pipes.
- eboyjr 8y agoI'd like to point out that the author is not directly discrediting shell pipes. > a knowledgable user will most likely check the content first The obvious workaround would be to download with curl, inspect, then run the virtually same inspected file through bash. This workflow is easier without necessarily using pipes. Package files can also be inspected before running and are not directly inspected in the browser. Trust on the other hand is more complicated. Without doing tedious manual inspecting, you have to rely on the distributor. In this case, public keys aid in this regard, but also does not work with the `curl | bash` workflow.
- blub 8y agoBash: execute an unsigned script to install an unsigned payload. Probably requires admin rights. dmg: download an archive file which contains a signed payload which is copied to Apps. Admin rights are used for copying only. The difference is blindingly obvious.
- homakov 8y agoI have proposed a solution to this: have the snippet self-hashed and verify with multiple sources. eg: A=$(curl -L https://get.rvm.io);echo https://get.rvm.io);echo "$A" | shasum -a 256 | grep -q 05b6b5f164d4df5aa593f9f925806fc1f48c4517daeeb5da66ee49e8562069e1 && (echo "$A")
- charleslmunger 8y agoThe sleep cleverness is excessive though - what you really want to know is if the script you're returning is being executed as it's sent. If it is, then you can be pretty confident that a human isn't reading it line by line. 1. Send your response as transfer-encoding: chunked and tcp_nodelay 2. Send the first command as curl www.example.com/$unique_id Then the server waits before sending the next command - if it gets the ping from the script, we know that whatever is executing the script is running the commands as they're sent, and is therefore unlikely to be read by a human before the next command runs. If it doesn't ping within a second or so, proceed with the innocent payload. For extra evil deniability, structure your malicious payload as a substring of a plausibly valid sequence of commands - then simply hang the socket partway through. Future investigation will make it look like a network issue.
- Evan-Purkhiser 8y agoYou could even get more clever with this you could drop the unique_id and just match up the remote host IP. You could probably even disguise the command as something like a "network connectivity test" in the script. # Check network connectivity so we can continue the install if ! curl --fail www.example.com; then exit; fi Of course, what actually is happening is that we've just informed the server to now serve our malicious code.
- charleslmunger 8y agoRemote host IP isn't ideal because of NAT (request from another host on the network exposes your malfeasance), or if your target may be using something like TOR (two requests might have differing remote IPs). But there's a bunch of tricks to get unique info out of a network request that you control the parameters to. Presumably there aren't that many concurrent invocations of your script, so only a few bits of entropy are actually required. Best way is probably to have a bunch of domains and make it look like they're various mirrors you're downloading binaries from - then it's not suspicious that it changes for different machines or requests.
- true_religion 8y ago
- growthexecutive 8y ago> a knowledgable user will most likely check the content first Really? Are they going to read every line of code and every line of code in every dependency that the install script installs? The bash detection is clever but I think its a solution to a problem that doesn't exist.. Its already very easy to install hide malicious code in plain sight, why go to all this trouble to detect if the user is piping to bash? For example, see how easy it is to publish a fake npm package or a .deb package: https://hackernoon.com/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5 https://hackernoon.com/im-harvesting-credit-card-numbers-and... https://github.com/ChaitanyaHaritash/kimi https://github.com/ChaitanyaHaritash/kimi
- rollthehard6 8y agoErm, why not just configure auditd to detect it instead? Though not sure how widely available that facility is outside of Red Hat.
- jwilk 8y ago(2016) Previous discussion: https://news.ycombinator.com/item?id=11532599 https://news.ycombinator.com/item?id=11532599
- Proven 8y agoThis is silly. You have no idea whether someone is simply using a honeypot VM is o test what gets sent or actually using this approach to install.
- ithkuil 8y agoI wish there was a standard way to check a checksum, so that download instructions could just include that in the snippet to copy paste. I wrote a tool that could be used like that but it's useless if its not ubiquitous (https://github.com/mmikulicic/runck https://github.com/mmikulicic/runck)
- pcl 8y agoI've got a similar python script in my collection of scripts: $ curl <url> | pipe-checksum <expected> | bash https://gist.github.com/pcl/64bd2f56695fcf8e1fad51443aab1f1e https://gist.github.com/pcl/64bd2f56695fcf8e1fad51443aab1f1e
- e12e 8y agoSince copy-pasting to the terminal is also unsafe[1], it's not really a solution... At any rate - code-signing doesn't really help if the author is the attacker. [1] http://thejh.net/misc/website-terminal-copy-paste http://thejh.net/misc/website-terminal-copy-paste
- ithkuil 8y agoSure, but that's harder to hide. Any user could paste somewhere where nothing gets executed and the expose the hack attempt. Pipe to bash has the interesting aspect of letting the author inject hacks only to people who are not looking. Anyway, the use case for my runck utility is scripts such as dockefiles or CI automation where I want to download and run installers and I don't want to reduce the bash boilerplate.
- lamby 8y agoShall we call this "Curl rebinding" after "DNS rebinding"? :)
- fibo 8y agoI use curl and bash to launch a script I wrote to install my versioned home files, and it is worth to use this method. It depends on what you need, in my use case is really comfortable. It is a matter of trust.
- z3t4 8y agoI always read the code first, but many times there's additional curl > bash and if I check that url there are yet more curl > bash several layers deep with branches. And they want me to run this as root ...
- zimbatm 8y agoThe main reason to use `curl | bash` is because it's installed almost everywhere. Everything else can be bootstrapped from that. If there was a standard `curlbash <URL> <SHA256>` program that was installed everywhere it would allow to work around all of these issues.
- kazinator 8y ago"curl > file ; [...] ; bash file" is also installed in all the same places.
- xg15 8y agoWhat I leaned in addition to the sleep trick: > Execution in bash is performed line by line and so the speed that bash can ingest data is limited by the speed of execution of the script. So even without any clever detection logic, thinking of curl | bash as "downloading a script, then immediately executing it" is already wrong. It's more "give this remote host a poor man's shell to my machine and hope that they will always execute the same sequence of commands on it".
- Pawamoy 8y agoYou could prevent the detection by wrapping contents in a block, so Bash reads it entirely before evaluating it: `safe_curl() { printf "{\n"; curl "$@"; printf " \n}"; }`
- dTal 8y agoQuite a sensible precaution anyway, as network interruption may otherwise leave you with a partially executed script.