40 ms·

# The URL is downloaded once, cached forever, and then sourced import "https://git.io/fAWiz" Call me old and cranky, but I thought this naive fad of piping t

by ary 6y ago

# The URL is downloaded once, cached forever, and then sourced import "https://git.io/fAWiz" Call me old and cranky, but I thought this naive fad of piping the unchecked contents of URLs to the shell would die off eventually once people got clued in to how dangerous it is. At a bare minimum the contents of any shell script coming from a remote source should be manually reviewed the first time, and then verified to have not changed by comparing against a locally stored cryptographic hash. This is no better than curl with some automatic caching. At the end of the day why not store your dependencies along side your scripts and then not worry about it changing unless you change it yourself?

endgame 6y ago
The dhall language does https imports, but with two important differences: 1. The language is _pure_ and _total_, so it will _eventually_ terminate and cannot perform side-effects 2. You can specify the SHA256 of the referenced resources inline, next to the URL. This gets you caching and breakage warnings.
treve 6y ago
Piping a url into bash is not more or less risky than downloading a windows binary, installing a snap, installing a npm/composer/whatever package with post-install hooks, downloading a binary for your Mac. It's impossible to read every line of code we execute (many of which are closed source anyway). Downloading a large script and reading it before running it in bash is also hardly good security measure. It's much more practical to teach people how to recognize trustworthy sources, and how HTTPS works. Or you can switch to a platform like iOS where there's complete control, sandboxes and reviews.
- ary 6y ago
  The risks involved with executing code are legion, and wanting to promote cultural change in the direction of better safety isn't invalidated by pointing out failings in other areas. > Downloading a large script and reading it before running it in bash is also hardly good security measure. > It's much more practical to teach people how to recognize trustworthy sources, and how HTTPS works. Or you can switch to a platform like iOS where there's complete control, sandboxes and reviews. Downloading bash scripts and inspecting them before running is definitely a good security measure, as is running untrusted software in a VM or container where possible, and generally promoting caution when running anything acquired from the Internet. Even if the source is considered trustworthy there is still the possibility that it has been compromised. npm, pypi, and even the Linux kernel source tree are all examples of this. In this particular situation the use case is extremely weak. The problem is solved by incorporating dependencies locally, as I pointed out above.
  - treve 6y ago
    This is all fair, it just sometimes feel people are a little too focused on running bash scripts, when something like `npm i create-react-app` downloads 200K lines of code, all of which can autorun. 'Read everything you download' is just not incredibly helpful. From all the code we trust and not read is just a drop in the ocean.
    - _frkl 6y ago
      I totally agree, and never understood the focus on curly bash script. People seem to be ok to 'git clone; make; sudo make install' something, but hesitate to pipe a remote bash script that comes from the same author, and often is hosted in the exact same repo... At least, nobody ever specifically pointed out how dangerous the make scenario is. There are a few minor things one should do when writing such a curly bash script, but overall I, personally, don't worry too much about it when I encounter them, I just do my usual security assessment and risk mitigation (which usually does not include reading the code -- that's just not practical).
      - craftinator 6y ago
        I think it's the programmer's equivalent of "get off my lawn you kids" or "if you make that face for too long, it'll get stuck like that forever".
      - deleted 6y ago
        [deleted]
  - smichel17 6y ago
    > Piping a url into bash is not more or less risky than downloading a windows binary, installing a snap, installing a npm/composer/whatever package with post-install hooks, downloading a binary for your Mac. I agree. All of these are risky operations where I vet the source before running the code. (and for that reason I prefer to stick to software in my distro's repos, when possible). Sometimes I am able to vet the source directly, like with bash scripts. I prefer this. Other times, I have to make do with verifying the reputation of the distributor.
- radarsat1 6y ago
  Still, downloading and executing code can be guarded by trusted signatures, as used by deb etc. You can protect such an "import" statement by restricting it to signed files, at the very least. That allows you to "trust" a source without reading every line of it. Not infallible, but much better than "download and run this thing from some obfuscated URL and assume it hasn't been hijacked"
- choeger 6y ago
  > Piping a url into bash is not more or less risky than downloading a windows binary, installing a snap, installing a npm/composer/whatever package with post-install hooks, downloading a binary for your Mac. Wrong. Even windows binaries are signed nowadays. When you use a package manager, you cannot be targeted (assuming the package manager uses https) so you get at least some safety from sheer numbers. A URL, otoh, can easily be personalized. I could easily serve you a specific malware whereas everyone else and GitHub just sees the benign code.
  - dividedbyzero 6y ago
    > I could easily serve you a specific malware whereas everyone else and GitHub just sees the benign code. Could you, though? For a GitHub URL?
    - SifJar 6y ago
      As far as I can tell, the URLs don't have to be GitHub URLs, you can provide any URL for people to "import"
      - dividedbyzero 6y ago
        That's true, but I read parent as specifically talking about Github. In general, if it's on a server I control, it's more doable. Still not easy, I guess – how do you tell me apart from everyone else who fetches that same dependency? But it may be doable.
    - choeger 6y ago
      I could obviously not change the content of a GitHub url. Well, at least not for a reasonable cost and effort. But I can easily create a "curl this and execute it" line in a README.md and have it serve malicious code just to you.
- easterncalculus 6y ago
  > Downloading a large script and reading it before running it in bash is also hardly good security measure. I don't know how large you mean, but things written in the shell are usually very conducive to a quick scan. You're operating at a high level of abstraction so underhanded code is forced to be more obvious. It's actually extremely easy to find code that is at least suspicious enough not to run (but not necessarily malicious) so there's plenty of reason to still do it as opposed to trusting leaps of faith. I guess if it's several thousand lines then it's not worth reading, but what script that's worth running is? The other thing this misses is that you're running unsigned software, which despite some protesting I've seen here is certainly better than blind trust. That's what you're getting out of most binaries on your system, and other packages being pulled down. Piping a url into bash is more risky than running something unsigned, always. HTTPS is also a pretty poor integrity enforcement tool. Sites are breached all the time making it trivial to change the code being served. You might not have anyone changing it on the way, but if you're drinking bad water straight out of the tap it doesn't matter.

by ary 6y ago

endgame 6y ago

The dhall language does https imports, but with two important differences: 1. The language is _pure_ and _total_, so it will _eventually_ terminate and cannot perform side-effects 2. You can specify the SHA256 of the referenced resources inline, next to the URL. This gets you caching and breakage warnings.

treve 6y ago

Piping a url into bash is not more or less risky than downloading a windows binary, installing a snap, installing a npm/composer/whatever package with post-install hooks, downloading a binary for your Mac. It's impossible to read every line of code we execute (many of which are closed source anyway). Downloading a large script and reading it before running it in bash is also hardly good security measure. It's much more practical to teach people how to recognize trustworthy sources, and how HTTPS works. Or you can switch to a platform like iOS where there's complete control, sandboxes and reviews.

ary 6y ago
The risks involved with executing code are legion, and wanting to promote cultural change in the direction of better safety isn't invalidated by pointing out failings in other areas. > Downloading a large script and reading it before running it in bash is also hardly good security measure. > It's much more practical to teach people how to recognize trustworthy sources, and how HTTPS works. Or you can switch to a platform like iOS where there's complete control, sandboxes and reviews. Downloading bash scripts and inspecting them before running is definitely a good security measure, as is running untrusted software in a VM or container where possible, and generally promoting caution when running anything acquired from the Internet. Even if the source is considered trustworthy there is still the possibility that it has been compromised. npm, pypi, and even the Linux kernel source tree are all examples of this. In this particular situation the use case is extremely weak. The problem is solved by incorporating dependencies locally, as I pointed out above.
- treve 6y ago
  This is all fair, it just sometimes feel people are a little too focused on running bash scripts, when something like `npm i create-react-app` downloads 200K lines of code, all of which can autorun. 'Read everything you download' is just not incredibly helpful. From all the code we trust and not read is just a drop in the ocean.
  - _frkl 6y ago
    I totally agree, and never understood the focus on curly bash script. People seem to be ok to 'git clone; make; sudo make install' something, but hesitate to pipe a remote bash script that comes from the same author, and often is hosted in the exact same repo... At least, nobody ever specifically pointed out how dangerous the make scenario is. There are a few minor things one should do when writing such a curly bash script, but overall I, personally, don't worry too much about it when I encounter them, I just do my usual security assessment and risk mitigation (which usually does not include reading the code -- that's just not practical).
    - craftinator 6y ago
      I think it's the programmer's equivalent of "get off my lawn you kids" or "if you make that face for too long, it'll get stuck like that forever".
    - deleted 6y ago
      [deleted]
- smichel17 6y ago
  > Piping a url into bash is not more or less risky than downloading a windows binary, installing a snap, installing a npm/composer/whatever package with post-install hooks, downloading a binary for your Mac. I agree. All of these are risky operations where I vet the source before running the code. (and for that reason I prefer to stick to software in my distro's repos, when possible). Sometimes I am able to vet the source directly, like with bash scripts. I prefer this. Other times, I have to make do with verifying the reputation of the distributor.
radarsat1 6y ago
Still, downloading and executing code can be guarded by trusted signatures, as used by deb etc. You can protect such an "import" statement by restricting it to signed files, at the very least. That allows you to "trust" a source without reading every line of it. Not infallible, but much better than "download and run this thing from some obfuscated URL and assume it hasn't been hijacked"
choeger 6y ago
> Piping a url into bash is not more or less risky than downloading a windows binary, installing a snap, installing a npm/composer/whatever package with post-install hooks, downloading a binary for your Mac. Wrong. Even windows binaries are signed nowadays. When you use a package manager, you cannot be targeted (assuming the package manager uses https) so you get at least some safety from sheer numbers. A URL, otoh, can easily be personalized. I could easily serve you a specific malware whereas everyone else and GitHub just sees the benign code.
- dividedbyzero 6y ago
  > I could easily serve you a specific malware whereas everyone else and GitHub just sees the benign code. Could you, though? For a GitHub URL?
  - SifJar 6y ago
    As far as I can tell, the URLs don't have to be GitHub URLs, you can provide any URL for people to "import"
    - dividedbyzero 6y ago
      That's true, but I read parent as specifically talking about Github. In general, if it's on a server I control, it's more doable. Still not easy, I guess – how do you tell me apart from everyone else who fetches that same dependency? But it may be doable.
  - choeger 6y ago
    I could obviously not change the content of a GitHub url. Well, at least not for a reasonable cost and effort. But I can easily create a "curl this and execute it" line in a README.md and have it serve malicious code just to you.
easterncalculus 6y ago
> Downloading a large script and reading it before running it in bash is also hardly good security measure. I don't know how large you mean, but things written in the shell are usually very conducive to a quick scan. You're operating at a high level of abstraction so underhanded code is forced to be more obvious. It's actually extremely easy to find code that is at least suspicious enough not to run (but not necessarily malicious) so there's plenty of reason to still do it as opposed to trusting leaps of faith. I guess if it's several thousand lines then it's not worth reading, but what script that's worth running is? The other thing this misses is that you're running unsigned software, which despite some protesting I've seen here is certainly better than blind trust. That's what you're getting out of most binaries on your system, and other packages being pulled down. Piping a url into bash is more risky than running something unsigned, always. HTTPS is also a pretty poor integrity enforcement tool. Sites are breached all the time making it trivial to change the code being served. You might not have anyone changing it on the way, but if you're drinking bad water straight out of the tap it doesn't matter.