7 ms·

Show HN: Hash Archive helps you verify the hashes of insecure downloads

52 points by btrask 10y ago

tekacs 10y ago
The author's other project, https://github.com/btrask/stronglink https://github.com/btrask/stronglink looks fairly interesting too - 'A searchable, syncable, content-addressable notetaking system'.
- btrask 10y ago
  Thanks! Right now we're sort of in a chicken-and-egg situation where everything uses (mutable) URLs, so knowing the hash of a file isn't very useful. The Hash Archive is part of my strategy for promoting content addressing, to hopefully raise demand for systems like StrongLink (and others).
  - tekacs 10y ago
    The more people pushing for awareness the better! Hopefully with other work like IPFS (which you do reference), use of this approach for /more/ consumer systems is around the corner. :)
    - btrask 10y ago
      Yeah, IPFS is a big one. I plan on adding support for computing IPFS hashes to Hash Archive at some point too.
ikeboy 10y ago
Pretty cool. Could this be made into a browser extension that alerts you if there's a difference?
- btrask 10y ago
  Yes, this has been discussed. It has to be done carefully so that it doesn't leak every URL you download, but if done well I think it would be useful.
  - ikeboy 10y ago
    Double hash the url and check if it's in the database: if not, prompt the user if you want to upload the actual url.
    - btrask 10y ago
      I believe that double-hashing is "equivalent" to single-hashing. See this issue: https://github.com/ipfs/refs-denylists-dmca/issues/2 https://github.com/ipfs/refs-denylists-dmca/issues/2 There are some ways of doing it, depending on exactly what your threat model is, but I think it's risky in general. Right now this is just in the planning stages, but I want to have submitting URLs be a manual button click for each download, and also provide an option to use a local copy of the database (so that all lookups would be completely private).
- toyg 10y ago
  I was trying to do something exactly like this a few months ago, but I found that browsers' extension APIs have no way to access files after download. You'd have to intercept a download and manage it entirely in the extension, which will likely break so many JS-driven downloads as to be very bothersome. Alternatively you would have to install a local helper process outside the browser, and at that point, you're basically an antivirus. In fact, I think AVs are the ones better placed to add this as a feature, as well as already having the significant resources needed to maintain a secure archive of hashes for files (although I had thought up a signature-based scheme that software vendors/distributors could adopt for a small fee).
geofft 10y ago
PowerShell 4.0, shipped with Windows 8 and above, has a pretty simple Get-FileHash command built in, so you don't need a separate download on those platforms. (On Windows 7 you still need either the download you link to, or a manual upgrade to PowerShell 4.0.) https://technet.microsoft.com/en-us/library/dn520872.aspx https://technet.microsoft.com/en-us/library/dn520872.aspx I believe the syntax is just Get-FileHash [file] -Algorithm SHA256
- btrask 10y ago
  Thanks, I've made a note to add this to the FAQ.
  - zokier 10y ago
    For previous versions of Windows there is certutil included which can used to compute SHA1 with -hashfile option: https://technet.microsoft.com/en-us/library/cc732443%28v=ws.11%29.aspx#BKMK_hashfile https://technet.microsoft.com/en-us/library/cc732443%28v=ws....
- theandrewbailey 10y ago
  I use HashTab. It makes a tab in the file properties dialog that will automatically compute several hashes at once. It even has copy-paste-verify functionality. http://implbits.com/products/hashtab/ http://implbits.com/products/hashtab/
Buge 10y ago
It would be nice if there was an easy way to copy the hashes for example to diff it against what you computed. As it is now, the page is laid out so that it is hard to copy just the hash.
- btrask 10y ago
  Agreed. I've thought about adding a JS/Flash clipboard button. For now, if you click a hash, it's easier to copy from the hash's own page. I think there's also potential for better tooling, for example CLI tools that integrate with Hash Archive (or other sites/databases) directly.
  - kpcyrd 10y ago
    sha256sum -c <(curl -sS https://hash-archive.org/history/http://openwall.com/signatures/openwall-signatures.asc | sed -nr 's/.*sha256/([a-f0-9]*).*/\1 openwall-signatures.asc/ p')
amjo324 10y ago
Great idea for a project. Nice job. It says in the About section on the home page "Unless someone can intercept your local traffic and our traffic to a site, you'll be able to spot MITM attacks". I'd argue that this is not entirely true. If an attacker operating as a MITM can intercept all local traffic (e.g. via some form of DNS attack), they do not need to control the traffic from hash-archive.org to 3rd party sites. They simply need to control how hash-archive.org is presented to the victim. In theory, the attacker could serve up a bogus version of hash-archive.org that appears to be legitimate but is returning falsified hashes that match the malicious downloads they have intercepted elsewhere. You might claim this is not possible because hash-archive.org runs over HTTPS so an attacker would also have to somehow generate a valid SSL certificate signed by a trusted CA. This is true but if someone types hash-archive.org into their browser URL bar, the initial request is made over HTTP. The legitimate hash-archive.org redirects the client to HTTPS seamlessly but a fraudulent hash-archive.org could just keep the victim on HTTP. To provide some mitigation against this type of attack, you could do a couple things: * Only allow hash-archive.org to be accessed over HTTPS (port 443). Close port 80. [EDIT: in fact, this doesn't really help all that much because the MITM can still try serve their bogus version of hash-archive.org over HTTP] * Set the HTTP Strict Transport Security header (HSTS) [1]. After the first visit to the legitimate hash-archive.org, compliant browsers will only ever allow future visits to be made over HTTPS. For good measure, you could also set up HTTP Public Key Pinning (HPKP). HPKP is a 'security feature that tells a web client to associate a specific cryptographic public key with a certain web server to prevent MITM attacks with forged certificates.' [2] [1] https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_strict_transport_security https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_s... [2] https://developer.mozilla.org/en/docs/Web/Security/Public_Key_Pinning https://developer.mozilla.org/en/docs/Web/Security/Public_Ke...
- kefka 10y ago
  HPKP is also the technique used/abused by Microsoft to keep users from knowing what the Windows 10 system is telling to other servers about us.
  - userbinator 10y ago
sprin 10y ago
Fantastic! While not as bulletproof as receiving the hash out-of-band for a critical resource, this is better than verifying against a hash received from the same origin as the resource, and far better than no hash verification at all. And because this is FOSS, we can be gain some protection against the compromise or MITM of a single, central hash-archive server when many of them are deployed by distinct entities on different public domains. One request: there are lots of users who would be well-served by a way to compute hashes in-browser via the WebCryptoAPI [1]. Would you consider accepting this feature into hash-archive? For users who aren't able to install or have difficulty using a hash calculator locally, this would enable verification of downloaded files in a one-stop online workflow. [1] https://www.w3.org/TR/WebCryptoAPI/ https://www.w3.org/TR/WebCryptoAPI/ edit: I stood up an instance here, and I'll make an effort to keep it running and updated: https://hash-archive.probablybroken.com/ https://hash-archive.probablybroken.com/

52 points by btrask 10y ago

The author's other project, https://github.com/btrask/stronglink https://github.com/btrask/stronglink looks fairly interesting too - 'A searchable, syncable, content-addressable notetaking system'.

btrask 10y ago
Thanks! Right now we're sort of in a chicken-and-egg situation where everything uses (mutable) URLs, so knowing the hash of a file isn't very useful. The Hash Archive is part of my strategy for promoting content addressing, to hopefully raise demand for systems like StrongLink (and others).
- tekacs 10y ago
  The more people pushing for awareness the better! Hopefully with other work like IPFS (which you do reference), use of this approach for /more/ consumer systems is around the corner. :)
  - btrask 10y ago
    Yeah, IPFS is a big one. I plan on adding support for computing IPFS hashes to Hash Archive at some point too.

ikeboy 10y ago

Pretty cool. Could this be made into a browser extension that alerts you if there's a difference?

btrask 10y ago
Yes, this has been discussed. It has to be done carefully so that it doesn't leak every URL you download, but if done well I think it would be useful.
- ikeboy 10y ago
  Double hash the url and check if it's in the database: if not, prompt the user if you want to upload the actual url.
  - btrask 10y ago
    I believe that double-hashing is "equivalent" to single-hashing. See this issue: https://github.com/ipfs/refs-denylists-dmca/issues/2 https://github.com/ipfs/refs-denylists-dmca/issues/2 There are some ways of doing it, depending on exactly what your threat model is, but I think it's risky in general. Right now this is just in the planning stages, but I want to have submitting URLs be a manual button click for each download, and also provide an option to use a local copy of the database (so that all lookups would be completely private).
toyg 10y ago
I was trying to do something exactly like this a few months ago, but I found that browsers' extension APIs have no way to access files after download. You'd have to intercept a download and manage it entirely in the extension, which will likely break so many JS-driven downloads as to be very bothersome. Alternatively you would have to install a local helper process outside the browser, and at that point, you're basically an antivirus. In fact, I think AVs are the ones better placed to add this as a feature, as well as already having the significant resources needed to maintain a secure archive of hashes for files (although I had thought up a signature-based scheme that software vendors/distributors could adopt for a small fee).

geofft 10y ago

PowerShell 4.0, shipped with Windows 8 and above, has a pretty simple Get-FileHash command built in, so you don't need a separate download on those platforms. (On Windows 7 you still need either the download you link to, or a manual upgrade to PowerShell 4.0.) https://technet.microsoft.com/en-us/library/dn520872.aspx https://technet.microsoft.com/en-us/library/dn520872.aspx I believe the syntax is just Get-FileHash [file] -Algorithm SHA256

btrask 10y ago
Thanks, I've made a note to add this to the FAQ.
- zokier 10y ago
  For previous versions of Windows there is certutil included which can used to compute SHA1 with -hashfile option: https://technet.microsoft.com/en-us/library/cc732443%28v=ws.11%29.aspx#BKMK_hashfile https://technet.microsoft.com/en-us/library/cc732443%28v=ws....
theandrewbailey 10y ago
I use HashTab. It makes a tab in the file properties dialog that will automatically compute several hashes at once. It even has copy-paste-verify functionality. http://implbits.com/products/hashtab/ http://implbits.com/products/hashtab/

Buge 10y ago

It would be nice if there was an easy way to copy the hashes for example to diff it against what you computed. As it is now, the page is laid out so that it is hard to copy just the hash.

btrask 10y ago
Agreed. I've thought about adding a JS/Flash clipboard button. For now, if you click a hash, it's easier to copy from the hash's own page. I think there's also potential for better tooling, for example CLI tools that integrate with Hash Archive (or other sites/databases) directly.
- kpcyrd 10y ago
  sha256sum -c <(curl -sS https://hash-archive.org/history/http://openwall.com/signatures/openwall-signatures.asc | sed -nr 's/.*sha256/([a-f0-9]*).*/\1 openwall-signatures.asc/ p')

amjo324 10y ago

Great idea for a project. Nice job. It says in the About section on the home page "Unless someone can intercept your local traffic and our traffic to a site, you'll be able to spot MITM attacks". I'd argue that this is not entirely true. If an attacker operating as a MITM can intercept all local traffic (e.g. via some form of DNS attack), they do not need to control the traffic from hash-archive.org to 3rd party sites. They simply need to control how hash-archive.org is presented to the victim. In theory, the attacker could serve up a bogus version of hash-archive.org that appears to be legitimate but is returning falsified hashes that match the malicious downloads they have intercepted elsewhere. You might claim this is not possible because hash-archive.org runs over HTTPS so an attacker would also have to somehow generate a valid SSL certificate signed by a trusted CA. This is true but if someone types hash-archive.org into their browser URL bar, the initial request is made over HTTP. The legitimate hash-archive.org redirects the client to HTTPS seamlessly but a fraudulent hash-archive.org could just keep the victim on HTTP. To provide some mitigation against this type of attack, you could do a couple things: * Only allow hash-archive.org to be accessed over HTTPS (port 443). Close port 80. [EDIT: in fact, this doesn't really help all that much because the MITM can still try serve their bogus version of hash-archive.org over HTTP] * Set the HTTP Strict Transport Security header (HSTS) [1]. After the first visit to the legitimate hash-archive.org, compliant browsers will only ever allow future visits to be made over HTTPS. For good measure, you could also set up HTTP Public Key Pinning (HPKP). HPKP is a 'security feature that tells a web client to associate a specific cryptographic public key with a certain web server to prevent MITM attacks with forged certificates.' [2] [1] https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_strict_transport_security https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_s... [2] https://developer.mozilla.org/en/docs/Web/Security/Public_Key_Pinning https://developer.mozilla.org/en/docs/Web/Security/Public_Ke...

kefka 10y ago
HPKP is also the technique used/abused by Microsoft to keep users from knowing what the Windows 10 system is telling to other servers about us.
- userbinator 10y ago

sprin 10y ago

Fantastic! While not as bulletproof as receiving the hash out-of-band for a critical resource, this is better than verifying against a hash received from the same origin as the resource, and far better than no hash verification at all. And because this is FOSS, we can be gain some protection against the compromise or MITM of a single, central hash-archive server when many of them are deployed by distinct entities on different public domains. One request: there are lots of users who would be well-served by a way to compute hashes in-browser via the WebCryptoAPI [1]. Would you consider accepting this feature into hash-archive? For users who aren't able to install or have difficulty using a hash calculator locally, this would enable verification of downloaded files in a one-stop online workflow. [1] https://www.w3.org/TR/WebCryptoAPI/ https://www.w3.org/TR/WebCryptoAPI/ edit: I stood up an instance here, and I'll make an effort to keep it running and updated: https://hash-archive.probablybroken.com/ https://hash-archive.probablybroken.com/