10 ms·
I agree that the "just curl this into bash" instructions are nightmare - on any platform. I think a lot of this is a result of what I like to call the "Kumbaya
by stephenr 12y ago
I agree that the "just curl this into bash" instructions are nightmare - on any platform.
I think a lot of this is a result of what I like to call the "Kumbaya approach to project/team management":
This is where you have a team (either for a single project or a team at a consulting agency, etc) that is effectively all development-focused staff, possibly with some who dabble in Infrastructure/Ops. In this environment, when a decision about something like "how do we get a reliable build of X for our production server deployment system" needs to be made or a system needs to be supported, no idea is "bad", because no one has the experience or confidence to be able to say "that's a stupid idea, we are not making `curl http://bit.ly/foo http://bit.ly/foo | sudo bash` the first line of a deployment script"[1]
[1] yes this is an exaggeration, but there are some simply shocking things happening in real environments, that are not far off that mark.
Edit: to make it absolutely clear about what I was referring to with [1]:
The specific point I was making was running something they don't even see (how many people would actually look at the script before piping it to bash/sh ?) from a non-encrypted source, and relying on a redirection service that could remove/change your short url at any time.
Unfortunately I was stupid enough to ddg it (duckduckgo it, as opposed to google it) and apparently this exact use-case was previously the recommended way of installing RVM[2]
[2] http://stackoverflow.com/questions/5421800/rvm-system-wide-install-script-url-broken-what-is-replacement http://stackoverflow.com/questions/5421800/rvm-system-wide-i...
-
- tribaal 12y ago> I think a lot of this is a result of what I like to call the "Kumbaya approach to project/team management" I'm totally stealing this :)
- yAnonymous 12y ago>[1] yes this is an exaggeration No, it's not :(
- tormeh 12y agoIt's just automated copy-pasting of commands you don't understand from the internet, which is something everyone who runs Linux (and is not a wizard) does all the time. It's really really bad, but people will continue doing it until commands/things become so easy we can actually understand what we're doing. Unfortunately, this has never been a priority in Unix-land as far as I've gathered.
- TeMPOraL 12y agohttps://xkcd.com/1168/ https://xkcd.com/1168/ comes to mind. And yes, I Google half of the command invocations too (but usually type them in by hand so that I can remember them faster instead of copy-pasting).
- shanemhansen 12y agoI usually get tar right on the first try. I only have to remember 2 variants (extract file and create file): tar xf ./foo #automagically works with bz2 and gz files tar cf /tmp/out.tar . #add z for compression
- TeMPOraL 12y agoOh cool. So that works? I've already memorized: tar -xvvzf foo.tar.gz tar -xvvjf bar.tar.bz2 tar -xvvf baz.tar Thanks!
- Gracana 12y agoThat second one will create a tarbomb[1], which isn't necessarily wrong and maybe it's what's right for your application, but for more general usage this is friendlier: tar cf <mydir.tar> <mydir> [1] http://www.linfo.org/tarbomb.html http://www.linfo.org/tarbomb.html
- markrages 12y agoAnd some of those switches are just for convenience, e.g.: tar c . | gzip > /tmp/out.tar.gz
- falcolas 12y agoI don't get this. Tar isn't that hard. x = eXtract files from an archive f = File path to the archive c = Create a new archive from files v = print Verbose output z = apply gZip the input or output That's 99% of common tar right there. The remaining one percent is: j = apply bzip2 to the input or output (I admit, j is a weird one here, though that has made it stick in my memory) --list = does what's on the tin --exclude = does what's on the tin --strip-components = shortcut for dropping a leading directory from the extracted I haven't used a flag outside of these in recent memory.
- jkyle 12y ago> Unfortunately I was stupid enough to ddg it (duckduckgo it, as opposed to google it) and apparently this exact use-case was previously the recommended way of installing RVM[2] Not only "previously", it's the current recommended way to install rvm. From their front page: >> curl -sSL https://get.rvm.io https://get.rvm.io | bash -s stable [1] http://rvm.io/ http://rvm.io/
- stephenr 12y agoOh I agree there are problems still, but its an improvement over the previous - it's using HTTPS and it's calling the RVM domain - before it was plain HTTP to bit.ly
- sciurus 12y agoAlso the installer is now signed via GPG. https://rvm.io/rvm/security https://rvm.io/rvm/security
- acveilleux 12y agoBut then there's a circular dependency because the GPG key is retrieved by the bash script that is wget'd.
- lmm 12y agoIt doesn't have to be circular. The script is secured by HTTPS (and hopefully has the key embedded in the script itself?) which can then retrieve the installer and verify it using the key.
- acveilleux 12y agoThe problem is that in this scenario, the GPG key and signature serves no practical purpose. The whole security, whether GPG is invoked or not, relies on the security of the HTTPS connection alone. If the HTTPS cannot be trusted alone, then everything is lost as a compromised HTTPS connection can be used supply both a comprimised GPG key and a compromised package, or, indeed, anything at all that is legal to `| sudo bash`... And HTTPS security boils down to: 1. The difficulty of altering (or exploiting privileged position wrt) the global routing table to setup MitM or MitS scenarios. 2. The difficulty of obtaining a valid looking certificate for an arbitrary domain. Any situation where a government actor is the adversary poses intractable challenges to both 1 and 2 above. (And before you say NSA/GCHQ would never care about XYZ, consider China...
- 23david 12y agoIn my experience, this is mainly describing the sad state of sysadmin work at tech startups. Larger and profitable tech companies tend to take sysadmin work a bit more seriously and give more resources and authority (and pay...) to their TechOps/Devops/Security teams.
- stephenr 12y agoIt seems reasonably common in agency type companies - at the start often their "infra" is an account with a managed web hosting company, ahd when their needs grow it doesn't always become a core part of the business
- tammer 12y agoAm dealing with this situation right now. Apparently wget -qO- https://get.docker.com/ https://get.docker.com/ | sh as root[1] is the "supported" way of installing Discourse [1]: https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md https://github.com/discourse/discourse/blob/master/docs/INST...
- yebyen 12y agoNo, that's Discourse install instructions quick, hand-wavy way of telling you to install Docker if you don't already have it. If your cloud environment already has Docker installed, you can skip that step. Are you really trying to say that the instructions for installing Docker should be considered in-scope for a guide to install Discourse on a cloud server? They've included a short snippet that will get you a Docker, in the way recommended by Docker, for whatever your base system is. Many production systems do not move at the pace of Docker development, so it's not practical to run Docker from your distribution's package archive. Some distros will not have distributed packaged Docker releases at all. What's wrong with these instructions? If you are really "dealing with this" right now, it is worth noting that something like 20 or more supported platforms have specific Docker installation instructions from the Docker website. https://docs.docker.com/installation/#installation https://docs.docker.com/installation/#installation From a quick sample of those instructions, only the Ubuntu instruction page uses the wget|sh method, and it's using an SSL connection to Docker's own website to add an apt source with signatures in the supported way. This way should work on any Debian-based or Yum-based distro, and writing the instructions like this must likely save Discourse from getting a lot of "How do I docker" issues and e-mails from their clueless users. So, would you prefer that part just says "installing Docker is out of scope" or should the Discourse developers go through every distro and cloud system and document the specific instructions for that? To do that would completely defeat the purpose of even using Docker at all.
- tammer 12y agoI concur - to elaborate, I'm not actually the one installing Discourse. I support a 'Kumbaya' group of data scientists who have never heard of docker. Quick directions like these aren't questioned by users who just want to get things done, and they invite security risks just as the parent & article suggest.
- matt_kantor 12y agoThere are many more depressing examples of this at http://curlpipesh.tumblr.com http://curlpipesh.tumblr.com
- squar1sm 12y agoFunny tumblr but makes me care-confused. I understand that curl pipe sh could have security problems but I also don't see it as that much different than the "normal" and "ok" way of doing things. I would consider something like the below pretty normal. wget https://whatever.io/latest.tgz tar xzf latest.tgz cd whatever-stable ./configure && make sudo make install Because of familiarity, we aren't going to be too worried about what we are doing. If we are on a secure system (like a bank or something) then we've probably already gone through a bunch of hoops (source check, research) and we mitigate it like anything else. What is so different about curl https://whatever.io/installer.sh | sudo bash We didn't check the md5s in the first example, so yolo, we don't care about the content of the tarball we just `make install`-ed. We're assuming the webserver isn't compromised and that https is protecting the transfer. Is it because the tarball hit the disk first? Does that give us a warm fuzzy? Is it because "anything could be in installer.sh!!!?! aaaaah!". Well, anything could be in Makefile too right? Anything could be in main.c or whatever. I agree that curl sh | sudo bash makes my spidey sense tingle. But if I really cared, I would read the source and do all the normal stuff anyway. So I think it's some kind of weird familiarity phase we're all in.
- stephenr 12y agoI wouldn't consider what you presented as the "normal" or "ok" way of doing things either, especially not on anything resembling a live (i.e. not development/sandbox) environment. A distro (or official vendor, or possibly a trusted third-party) repo of pre-built, signed packages would always be my first choice. If one of those isn't available, my next step would be to create a package for the tool in question, part of which is setting up a file for `uscan` to download new source archives, and compare against the signatures. In this scenario we (as in the organisation) are now responsible for actually building and maintaining the package, but we can still be assured that it's built from the original sources, we can still install it on production (and even dev, staging, whatever) servers with a simple call to apt/aptitude, and dependencies, removal, upgrades, etc are still handled cleanly.
- DannoHung 12y agoI think part of this is because there aren't any trusted, fully open source, artifact repositories that work with the various package indices out there. Like, most of the way deployment should work is that you come up with some collection of packages that need to be installed and you iterate through and install them. Bob's your uncle. Thing is, all the packages you need live out in the wild internet. Ideally, you'd just be able to take a package, vet it, and put it in your local artifact store and then when your production deployment system (using apt or yum or pip or gems or maven or whatever) needs a package, it looks at your local artifact store and grabs it and goes about its business. Never knowing or touching the outside world. And your developers would all write their apps to deploy through the normal packaging methods that everyone and their mother is already familiar with and they could just put them into the existing package index as well. But you've gotta lay out pretty serious moola (from when I last looked into available solutions to this) or set up a half dozen different artifact stores if you want to do things that way. And good luck managing your cached and private artifacts if you do. And on top of that developers don't necessarily know how to set up a PyPi or a RPM index or whatever so that the storage is reliable and you've got the right security settings or whatever else. (I know I sure don't and I'm not really interested in reading all of the ones I'd end up needing).
- StillBored 12y ago"And on top of that developers don't necessarily know how to set up a PyPi or a RPM index or whatever so that the storage is reliable and you've got the right security settings or whatever else. (I know I sure don't and I'm not really " Setting up RPM is shockingly easy. It can get more complex, but the basic system is: REPOBASE=/srv/www/htdocs/ createrepo -v $REPOBASE gpg -a --detach-sign --default-key "Sign Repo" $REPOBASE/repodata/repomd.xml gpg -a --export "Sign Repo" > $REPOBASE/repodata/repomd.xml.key That will create a repo from all the .rpm files in the REPOBASE. Also you will of course need a GPG key pair, but that can be generated with `gpg --gen-key` where you give it a description of "Sign Repo" (or change the above commands to the key description you used). Then you get to decide on the deployment machine if you want to trust the repo (or you don't trust any and import the key via some other process, aka direct gpg import). Of course you can find a bunch of more detailed explanations with $SEARCHENGINE, but if it takes more than a day to figure it out, your doing something wrong. Building a set of RPM's isn't that much harder if you have a proper build system. But these are the kinds of things you give up when you decide to grab the latest immature hotness created by someone on their day off.
- jsprogrammer 12y agoIt's also the recommended way to install Kubernetes.
- tracker1 12y agoIt's easy enough to download a given/checked version of the script at http://foo.com/ubuntu/install http://foo.com/ubuntu/install and have that copied and run inside your docker image... for that matter, it's usually adding a given repository to your repo manager, then installing a given package from that software's corporate sponsors. I don't think the problem is as rampant as it's made out to be in TFA... that said, most people don't look at said script(s), so it's entirely possible something could have been slipped in. For that matter, I think the issues outlined in the article relate more to overly complicated Java solutions (the same happens in the .Net space) that are the result of throwing dozens of developers some with more or less experience than others at a project, and letting a lot of code that isn't very well integrated slide through whatever review process does or doesn't exist.
- mreiland 12y agoI honestly don't see the issue. I see the issue with doing it for the general public ala RVM, but internally where you control everything I don't see the issue with curl into sh.