5 ms·

It's not enough to check the hash of a downloaded executable if the hash you're checking against came from the same source as the suspect file. I find it so in

by billpg 12y ago

It's not enough to check the hash of a downloaded executable if the hash you're checking against came from the same source as the suspect file. I find it so infuriating when I see a download page with hashes and the download links next to each other, as if that's any help at all.

icelancer 12y ago
No, but it's another step in the process for sure. It's not useless.
- kaoD 12y ago
  > It's not useless. How not?
  - justcommenting 12y ago
    two examples: if the hashes are served over an HTTPS connection that you believe trustworthy, and/or signed by a key you trust and can obtain securely
    - colanderman 12y ago
      And the server's not compromised.
      - justcommenting 12y ago
        this a valid concern but if that happens, significantly more people are screwed than those accessing the server via a bad tor exit
    - marcosdumay 12y ago
      The hash is comming from the same channel so, no it can not improve anything. If it is an https connection where you trust the CA, you don't need the hash. The binary coming from that connection is just as safe. If it is signed by a key you can obtain securely, well, sign the executable for once, forget about the hash.
      - justcommenting 12y ago
        not necessarily--in terms of hashes, i was referring to the practice of serving hashes via TLS and larger binaries via http. i believe virtualbox does this, for example. in such cases as long as you trust the TLS connection used to obtain the hashes, you can reasonably trust the binary obtained insecurely as much as you trust your TLS connection and your hashing algorithm. and indeed, hashes could/should be replaced with signatures in those circumstances, and sometimes are (e.g. tor project, but they also serve tor browser binaries via tls). most people most of the time trust signatures/hashes served over TLS. in cases where they don't trust the upstream CA (which is commonly influenced/chosen by the developers), there's also a good chance they shouldn't rationally trust the source of that code. i see your point technically, but in practice you're almost describing a "trusting trust" attack, which i don't think is the most common threat model for app downloads. and i agree re: signed executables, but we live in a TOFU world and vanishingly few people personally verify fingerprints OOB with devs, so for many people not looking at trust paths and such, you're practically talking about trusting EFF's CA vs. mozilla's. perhaps that's a significant distinction to some, but i'd probably characterize it differently than you have.
  - morsch 12y ago
    It's trivial for a MITM to patch all binaries he's intercepting, it's less trivial[0] to modify the corresponding hashes on web pages. I'm talking about automation here, not messing with a single web page, which a human can do in a couple of minutes. [0] You've probably already sent out the unmodified hashes before you sent out the patched file. First idea: Keep a set of known binaries and a set of hash replacements. When you encounter a binary, first check if you've seen it before. If you haven't, deliver it unmodified, then compute hashes in a couple of known formats, and the hashes of the corresponding patched binary. When you deliver text content, check if any of the previously computed hashes match and replace them with your evil version. When you encounter the binary a second time, deliver the patched version, assuming that you either sent a modified hash or there was no hash in the first place.
    - kaoD 12y ago
      To further improve your idea: you could detect links to binaries and look for hash-like strings next to them. Then you can pre-download, detect hash algorithm, patch, hash the patched binary and then serve the page with the modified hash (and cache everything for later use). It might make the first visit slower, but for small binaries (or large bandwith) it's probably not that noticeable (especially in Tor, where slowness is expected). Lots of times the hash is in a separate file, which makes the attack even less noticeable since you can patch and hash the binary as you detect it but before the hash file is downloaded by the user (and you can even delay the hash download as much as you need). As I see it, downloading hashes through unsafe channels delivers a false sense of security, which is even worse than no security at all.
4ad 12y ago
The hash is not for cryptographic purposes, it's to detect corrupt downloads. Ideally you should provide both the hash and you should pgp sign your releases.
- colanderman 12y ago
  That is bizarre. Like "I thought XMODEM went out of style in 1992" bizarre. Shouldn't checksumming the payload be performed at the application layer? (A quick Google shows that indeed it can be: the Content-MD5 header <http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html> http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html>. Wonder how widely supported it is by HTTP software used by people who like to check hashes of things they download.)
  - sp332 12y ago
    Well what if you don't get the binary from the same source as the hash, or it's been sitting on a disk for a while and you want to double-check its integrity?
    - colanderman 12y ago
      The post to which I replied specifically focused on the case where the binary came from the same source as the hash, and was being used to confirm the integrity of the download. I was not addressing other cases. (Although I will note that file systems, like application protocols, should maintain their own integrity; however most do not. Which also seems bizarre given that it's 2014.)
      - sp332 12y ago
        Other programs could, accidentally or maliciously, step on the contents of a file. Filesystem checksums are awesome (I use btrfs), but it doesn't protect your file from everything. Hm... maybe a versioned filesystem where you can go back to the "original" version of a file and verify its checksum? :)
yourad_io 12y ago
Still better than seeing this as "installation instructions": curl http://not-ssl.somesite.ru/ba.sh | bash Often without an https:// https:// option available at all.
- emillon 12y ago
  I've made a list here: http://curlpipesh.tumblr.com/ http://curlpipesh.tumblr.com/
  - yourad_io 12y ago
    My personal beef is with curling HTTP (no -s) URLs - getting an installation shell script over HTTPS is not unreasonable (unless instructed to provide a no-SSL-confirmation flag, such as -k for curl, in which case it is more or less the same thing). By default, `curl https://blah.blah/` https://blah.blah/` will only work if the TLS certs are proper & validated. This isn't about trusting the author (you'll be running their code anyway, one way or another) but the transport medium (HTTP!s). If I had your show-and-shame tumblr, I'd only include http:// http:// links.
  - zxcdw 12y ago
    Please add the case of Rust package manager Cargo, found at http://crates.io http://crates.io
    - emillon 12y ago
      Done. Thanks!
  - yonran 12y ago
    The vast majority of your weblog postings describe https downloads, so it is mostly tangential to grandparent’s complaint and to the article. A .sh script may be a messier and less idiomatic file format than .pkg/.deb/.rpm/.msi, but a pkg from an unknown server is just as dangerous as a shell script from an unknown server.
    - deleted 12y ago
peterwwillis 12y ago
The only people who provide either a checksum or a PGP signature for downloads are geeks. Those geeks usually distribute their files via mirrors, who all download from either the main source or other sources. Any of those downloads could have been corrupted mid-download (for a variety of real world reasons), so the checksum verifies the file integrity. The PGP key is there for the 1% of people who want to verify the file came from the author; it's up to you to figure out how to get the author's PGP key in a secure fashion (typically HTTPS from their website, or via e-mail).

by billpg 12y ago

icelancer 12y ago

No, but it's another step in the process for sure. It's not useless.

kaoD 12y ago
> It's not useless. How not?
- justcommenting 12y ago
  two examples: if the hashes are served over an HTTPS connection that you believe trustworthy, and/or signed by a key you trust and can obtain securely
  - colanderman 12y ago
    And the server's not compromised.
    - justcommenting 12y ago
      this a valid concern but if that happens, significantly more people are screwed than those accessing the server via a bad tor exit
  - marcosdumay 12y ago
    The hash is comming from the same channel so, no it can not improve anything. If it is an https connection where you trust the CA, you don't need the hash. The binary coming from that connection is just as safe. If it is signed by a key you can obtain securely, well, sign the executable for once, forget about the hash.
    - justcommenting 12y ago
      not necessarily--in terms of hashes, i was referring to the practice of serving hashes via TLS and larger binaries via http. i believe virtualbox does this, for example. in such cases as long as you trust the TLS connection used to obtain the hashes, you can reasonably trust the binary obtained insecurely as much as you trust your TLS connection and your hashing algorithm. and indeed, hashes could/should be replaced with signatures in those circumstances, and sometimes are (e.g. tor project, but they also serve tor browser binaries via tls). most people most of the time trust signatures/hashes served over TLS. in cases where they don't trust the upstream CA (which is commonly influenced/chosen by the developers), there's also a good chance they shouldn't rationally trust the source of that code. i see your point technically, but in practice you're almost describing a "trusting trust" attack, which i don't think is the most common threat model for app downloads. and i agree re: signed executables, but we live in a TOFU world and vanishingly few people personally verify fingerprints OOB with devs, so for many people not looking at trust paths and such, you're practically talking about trusting EFF's CA vs. mozilla's. perhaps that's a significant distinction to some, but i'd probably characterize it differently than you have.
- morsch 12y ago
  It's trivial for a MITM to patch all binaries he's intercepting, it's less trivial[0] to modify the corresponding hashes on web pages. I'm talking about automation here, not messing with a single web page, which a human can do in a couple of minutes. [0] You've probably already sent out the unmodified hashes before you sent out the patched file. First idea: Keep a set of known binaries and a set of hash replacements. When you encounter a binary, first check if you've seen it before. If you haven't, deliver it unmodified, then compute hashes in a couple of known formats, and the hashes of the corresponding patched binary. When you deliver text content, check if any of the previously computed hashes match and replace them with your evil version. When you encounter the binary a second time, deliver the patched version, assuming that you either sent a modified hash or there was no hash in the first place.
  - kaoD 12y ago
    To further improve your idea: you could detect links to binaries and look for hash-like strings next to them. Then you can pre-download, detect hash algorithm, patch, hash the patched binary and then serve the page with the modified hash (and cache everything for later use). It might make the first visit slower, but for small binaries (or large bandwith) it's probably not that noticeable (especially in Tor, where slowness is expected). Lots of times the hash is in a separate file, which makes the attack even less noticeable since you can patch and hash the binary as you detect it but before the hash file is downloaded by the user (and you can even delay the hash download as much as you need). As I see it, downloading hashes through unsafe channels delivers a false sense of security, which is even worse than no security at all.

4ad 12y ago

The hash is not for cryptographic purposes, it's to detect corrupt downloads. Ideally you should provide both the hash and you should pgp sign your releases.

colanderman 12y ago
That is bizarre. Like "I thought XMODEM went out of style in 1992" bizarre. Shouldn't checksumming the payload be performed at the application layer? (A quick Google shows that indeed it can be: the Content-MD5 header <http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html> http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html>. Wonder how widely supported it is by HTTP software used by people who like to check hashes of things they download.)
- sp332 12y ago
  Well what if you don't get the binary from the same source as the hash, or it's been sitting on a disk for a while and you want to double-check its integrity?
  - colanderman 12y ago
    The post to which I replied specifically focused on the case where the binary came from the same source as the hash, and was being used to confirm the integrity of the download. I was not addressing other cases. (Although I will note that file systems, like application protocols, should maintain their own integrity; however most do not. Which also seems bizarre given that it's 2014.)
    - sp332 12y ago
      Other programs could, accidentally or maliciously, step on the contents of a file. Filesystem checksums are awesome (I use btrfs), but it doesn't protect your file from everything. Hm... maybe a versioned filesystem where you can go back to the "original" version of a file and verify its checksum? :)

yourad_io 12y ago

Still better than seeing this as "installation instructions": curl http://not-ssl.somesite.ru/ba.sh | bash Often without an https:// https:// option available at all.

emillon 12y ago
I've made a list here: http://curlpipesh.tumblr.com/ http://curlpipesh.tumblr.com/
- yourad_io 12y ago
  My personal beef is with curling HTTP (no -s) URLs - getting an installation shell script over HTTPS is not unreasonable (unless instructed to provide a no-SSL-confirmation flag, such as -k for curl, in which case it is more or less the same thing). By default, `curl https://blah.blah/` https://blah.blah/` will only work if the TLS certs are proper & validated. This isn't about trusting the author (you'll be running their code anyway, one way or another) but the transport medium (HTTP!s). If I had your show-and-shame tumblr, I'd only include http:// http:// links.
- zxcdw 12y ago
  Please add the case of Rust package manager Cargo, found at http://crates.io http://crates.io
  - emillon 12y ago
    Done. Thanks!
- yonran 12y ago
  The vast majority of your weblog postings describe https downloads, so it is mostly tangential to grandparent’s complaint and to the article. A .sh script may be a messier and less idiomatic file format than .pkg/.deb/.rpm/.msi, but a pkg from an unknown server is just as dangerous as a shell script from an unknown server.
  - deleted 12y ago

peterwwillis 12y ago

The only people who provide either a checksum or a PGP signature for downloads are geeks. Those geeks usually distribute their files via mirrors, who all download from either the main source or other sources. Any of those downloads could have been corrupted mid-download (for a variety of real world reasons), so the checksum verifies the file integrity. The PGP key is there for the 1% of people who want to verify the file came from the author; it's up to you to figure out how to get the author's PGP key in a secure fashion (typically HTTPS from their website, or via e-mail).