3 ms·

Web browsers should support built-in checksum verification of downloads (when owners upload md5sum/signatures/etc with the files), instead of having to do it ma

by plasma 15y ago

Web browsers should support built-in checksum verification of downloads (when owners upload md5sum/signatures/etc with the files), instead of having to do it manually.

gcr 15y ago
How does the browser verify that the md5sum is correct? The server would have to send the md5sum to the browser through SSL, which is good enough to protect the download anyway.
- craz 15y ago
  I guess if the download is served from a CDN or mirror the checksum would still be helpful.
cookiecaper 15y ago
They should really use BitTorrent as the standard. BitTorrent is popular with people who download lots of large files because BitTorrent is an excellent way to download things. Summary and on-the-fly data verification, simple, reasonable ways to recover in case of corruption (download (approximately) only the corrupted portions, not the whole file again), and easy scaling and load distribution that makes things difficult to topple. BT is pretty awesome, I'd love to see it become a standard file distribution mechanism.
- dsl 15y ago
  You are confusing checksums with signatures. BitTorrent only verifies that I have a bit for bit copy of the file the torrent was created from, not that the file is an unaltered version of what the author originally released. Additionally, the modern internet no longer needs mechanisms to deal with download corruption beoynd TCP checksums, BitTorrent implements recovery from failed blocks to protect you from a "flaw" of the protocol itself (you can't always trust the origin of the bits). BitTorrent is also overly inefficient as a download protocol. It only makes sense in situations where you have extremely large flash crowds (WoW for example sending multiple GB to 14 million players in a few days) or the original source isn't in a position (legally or technologically) to source more than a few downloads. The final nail in the coffin is the fact that a few major CDNs tried peer-to-peer for content delivery. The major broadband ISPs were none too pleased with their last-mile infrastructure (the most expensive part to deploy mind you) being abused. Their networks were designed for consumption, not distribution.
  - cookiecaper 15y ago
    I didn't confuse checksums and signatures. I understand the difference between a checksum and a cryptographic signature. I'm sorry if "verification" was used outside its normal context, I just felt it was the best descriptor. Secondly, there's been multiple times where I've downloaded a large file and found it was corrupted. Under conventional protocols you can't easily identify the part that is damaged, so even if it's only a small part (because WiFi dropped or whatever), you have to redownload the whole thing. Instead of doing that, I have just created a torrent, uploaded to openbittorrent, and downloaded the few megabytes I still needed. I agree it may not be worth it to torrent small files (maybe <50MB), but I am not so pessimistic. As far as ISPs go, they just have to accept that their customers are going to be uploading much more than they have been and provide whatever infrastructure upgrades are required. I have no pity for that situation and I don't think it's a valid excuse to not see wider-spread usage of BitTorrent as a conventional download mechanism (i.e., baked into browsers, looks like a HTTP download). They can even default browsers to upload at a low rate, like 5Kb/s, or even 0Kb/s, and be fine; the benefits we're discussing now are inherent in the protocol, not necessarily the size of the swarm; we can still expect content providers to provide the machines that upload all of their content and allow seeding as opt-in only, and see big improvements from the usage of the BitTorrent protocol.
mike-cardwell 15y ago
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html 14.15 Content-MD5 The Content-MD5 entity-header field, as defined in RFC 1864 [23], is an MD5 digest of the entity-body for the purpose of providing an end-to-end message integrity check (MIC) of the entity-body. (Note: a MIC is good for detecting accidental modification of the entity-body in transit, but is not proof against malicious attacks.) Content-MD5 = "Content-MD5" ":" md5-digest md5-digest = <base64 of 128 bit MD5 digest as per RFC 1864> EDIT: I don't know if any browsers actually support this. I just configured an incorrect Content-MD5 header for a page and visited it in Chrome, Safari and Firefox and none of them complained about it.
- justincormack 15y ago
  Amazon S3 uses this, dont think any browsers do. You could file a bug report!
zobzu 15y ago
md5 only ensure data integrity from the download itself if the data has been tampered with, it ensures nothing. the md5 can be changed or even, the file can be made to match the md5 (its broken) gpg/pgp is a much better alternative as you can't create the signature either without having the original key (its not bullet proof as one could store the key in unsafe locations or be hacked of course, but its much better)
- mike-cardwell 15y ago
  Signing using PGP would be good, but it wouldn't prevent somebody using a replay attack to cause the browser to download an old vulnerable version of an application instead of a new patched version.
  - Tharkun 15y ago
    Signing moves the problem elsewhere. The problem then becomes the key exchange. How do I, as a first time downloader, know that the key listed on the website is the correct one? It's the same problem SSL has, sure, there's a key, and your browser trusts it, but how do I know that this key can really be trusted?

by plasma 15y ago

Web browsers should support built-in checksum verification of downloads (when owners upload md5sum/signatures/etc with the files), instead of having to do it manually.

gcr 15y ago

How does the browser verify that the md5sum is correct? The server would have to send the md5sum to the browser through SSL, which is good enough to protect the download anyway.

craz 15y ago
I guess if the download is served from a CDN or mirror the checksum would still be helpful.

cookiecaper 15y ago

They should really use BitTorrent as the standard. BitTorrent is popular with people who download lots of large files because BitTorrent is an excellent way to download things. Summary and on-the-fly data verification, simple, reasonable ways to recover in case of corruption (download (approximately) only the corrupted portions, not the whole file again), and easy scaling and load distribution that makes things difficult to topple. BT is pretty awesome, I'd love to see it become a standard file distribution mechanism.

dsl 15y ago
You are confusing checksums with signatures. BitTorrent only verifies that I have a bit for bit copy of the file the torrent was created from, not that the file is an unaltered version of what the author originally released. Additionally, the modern internet no longer needs mechanisms to deal with download corruption beoynd TCP checksums, BitTorrent implements recovery from failed blocks to protect you from a "flaw" of the protocol itself (you can't always trust the origin of the bits). BitTorrent is also overly inefficient as a download protocol. It only makes sense in situations where you have extremely large flash crowds (WoW for example sending multiple GB to 14 million players in a few days) or the original source isn't in a position (legally or technologically) to source more than a few downloads. The final nail in the coffin is the fact that a few major CDNs tried peer-to-peer for content delivery. The major broadband ISPs were none too pleased with their last-mile infrastructure (the most expensive part to deploy mind you) being abused. Their networks were designed for consumption, not distribution.
- cookiecaper 15y ago
  I didn't confuse checksums and signatures. I understand the difference between a checksum and a cryptographic signature. I'm sorry if "verification" was used outside its normal context, I just felt it was the best descriptor. Secondly, there's been multiple times where I've downloaded a large file and found it was corrupted. Under conventional protocols you can't easily identify the part that is damaged, so even if it's only a small part (because WiFi dropped or whatever), you have to redownload the whole thing. Instead of doing that, I have just created a torrent, uploaded to openbittorrent, and downloaded the few megabytes I still needed. I agree it may not be worth it to torrent small files (maybe <50MB), but I am not so pessimistic. As far as ISPs go, they just have to accept that their customers are going to be uploading much more than they have been and provide whatever infrastructure upgrades are required. I have no pity for that situation and I don't think it's a valid excuse to not see wider-spread usage of BitTorrent as a conventional download mechanism (i.e., baked into browsers, looks like a HTTP download). They can even default browsers to upload at a low rate, like 5Kb/s, or even 0Kb/s, and be fine; the benefits we're discussing now are inherent in the protocol, not necessarily the size of the swarm; we can still expect content providers to provide the machines that upload all of their content and allow seeding as opt-in only, and see big improvements from the usage of the BitTorrent protocol.

mike-cardwell 15y ago

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html 14.15 Content-MD5 The Content-MD5 entity-header field, as defined in RFC 1864 [23], is an MD5 digest of the entity-body for the purpose of providing an end-to-end message integrity check (MIC) of the entity-body. (Note: a MIC is good for detecting accidental modification of the entity-body in transit, but is not proof against malicious attacks.) Content-MD5 = "Content-MD5" ":" md5-digest md5-digest = <base64 of 128 bit MD5 digest as per RFC 1864> EDIT: I don't know if any browsers actually support this. I just configured an incorrect Content-MD5 header for a page and visited it in Chrome, Safari and Firefox and none of them complained about it.

justincormack 15y ago
Amazon S3 uses this, dont think any browsers do. You could file a bug report!

zobzu 15y ago

md5 only ensure data integrity from the download itself if the data has been tampered with, it ensures nothing. the md5 can be changed or even, the file can be made to match the md5 (its broken) gpg/pgp is a much better alternative as you can't create the signature either without having the original key (its not bullet proof as one could store the key in unsafe locations or be hacked of course, but its much better)

mike-cardwell 15y ago
Signing using PGP would be good, but it wouldn't prevent somebody using a replay attack to cause the browser to download an old vulnerable version of an application instead of a new patched version.
- Tharkun 15y ago
  Signing moves the problem elsewhere. The problem then becomes the key exchange. How do I, as a first time downloader, know that the key listed on the website is the correct one? It's the same problem SSL has, sure, there's a key, and your browser trusts it, but how do I know that this key can really be trusted?