4 ms·

Why wasn't this functionality ever baked into the web? HEAD requests could return a SHA hash of the content, and browsers would check peers before issuing a GET

by sowbug 4y ago

Why wasn't this functionality ever baked into the web? HEAD requests could return a SHA hash of the content, and browsers would check peers before issuing a GET for the remote resource.

fbrchps 4y ago
The security implications of "find me another device on this network who has gone to a specific page" are immense. Not to mention accessing account-related information due to improper no-cache headers on the website.
- ruined 4y ago
  there are plenty of more plausible misconfiguration risks that we accept, or consider the operators responsible. i'm not sure why you would take issue with this one. additionally, content-addressing provides another layer of security beyond location addressing. even improperly cached information is as secure as your hashing algorithm.
- deleted 4y ago
  [deleted]
- 908B64B197 4y ago
  Only applications where I could see the tradeoffs be worth it is over satellite links. Either that or game/code assets on large college or corporate campuses.
  - sowbug 4y ago
    There are lots of grade schools with slow internet, and when OS updates come out, all the kids open their school laptops and get prompted to update, which basically ends the class immediately because the slow pipe can't service 25 identical hundred-megabyte downloads quickly.
amaccuish 4y ago
Because that would effectively broadcast your browsing history to the LAN.
WJW 4y ago
Seems like it would only work in a very benign network environment. The first thing 4chan would do is to write a "peer" that answers yes for every SHA it gets and then sends over porn instead. Hope all your network software re-verifies the hash for every file you get! There is also the massive privacy leak of asking for the SHA of a certain file only found on specific websites and then seeing who has it. HTTP is just inherently server-client, decreasing load on servers by sharing between clients was not a design goal.
_dain_ 4y ago
isn't that just bittorrent?
7steps2much 4y ago
I mean ... At that point you could just include a caching/proxy layer at your router.
Ekaros 4y ago
Which peers? Connected how? Sounds simple, but number of cases where this would happen is likely pretty low.
capableweb 4y ago
If you read some of the original documents about the web, you'll see that it was one of the ideas. It never made it further than being an idea though. Although IPFS seems to want to implement something like that, and probably much easier to do in a content-addressable way (like IPFS). So content has a hash, and as long as you trust the source of the hash, you can download the content of a hash from anywhere, even your drunk and slightly annoying neighbor that you don't actually trust, because you can verify that you got the right bytes after all. Bittorrent does it for transfers already, and it works alright, so why not for the web too?
kevin_thibedeau 4y ago
That doesn't work with dynamic pages. A simple timestamp change would invalidate the hash between HEAD and GET.
- sowbug 4y ago
  For sure. Content addressing doesn't help if the content is dynamic.
cesarb 4y ago
It's not exactly the same, but back in the days before "SSL all the things!" became the norm, it was common to have transparent Squid caching proxies on local networks. If a peer on your network had recently requested a web resource, it was very likely that it was still in the Squid cache, and it would be loaded from there, instead of going to the origin server. Yes, it was a pain whenever the origin server had misconfigured caching headers. The "force reload bypassing caches" shortcut (Shift-Control-R, or Shift-Click-Reload-Button) was always handy. The move to TLS everywhere, which killed these "transparent" (when they worked) caching proxies, made the web much more stable and predictable.
- snvzz 4y ago
  LANs would usually have non-transparent proxies, which prevent the issues of transparent proxies and can also work with HTTPS. Some ISPs, on the other hand, did run "transparent" proxies in their networks, causing a range of issues.
digitalsushi 4y ago
if you have enough people (2, with similar interests at minimum, or a satellite office of workers, etc) a perimiter web proxy is a great way to help get a lot of this behavior. so long as you dont have secrets, it works great
deleted 4y ago
[deleted]

by sowbug 4y ago

Why wasn't this functionality ever baked into the web? HEAD requests could return a SHA hash of the content, and browsers would check peers before issuing a GET for the remote resource.

fbrchps 4y ago

The security implications of "find me another device on this network who has gone to a specific page" are immense. Not to mention accessing account-related information due to improper no-cache headers on the website.

ruined 4y ago
there are plenty of more plausible misconfiguration risks that we accept, or consider the operators responsible. i'm not sure why you would take issue with this one. additionally, content-addressing provides another layer of security beyond location addressing. even improperly cached information is as secure as your hashing algorithm.
deleted 4y ago
[deleted]
908B64B197 4y ago
Only applications where I could see the tradeoffs be worth it is over satellite links. Either that or game/code assets on large college or corporate campuses.
- sowbug 4y ago
  There are lots of grade schools with slow internet, and when OS updates come out, all the kids open their school laptops and get prompted to update, which basically ends the class immediately because the slow pipe can't service 25 identical hundred-megabyte downloads quickly.

amaccuish 4y ago

Because that would effectively broadcast your browsing history to the LAN.

WJW 4y ago

Seems like it would only work in a very benign network environment. The first thing 4chan would do is to write a "peer" that answers yes for every SHA it gets and then sends over porn instead. Hope all your network software re-verifies the hash for every file you get! There is also the massive privacy leak of asking for the SHA of a certain file only found on specific websites and then seeing who has it. HTTP is just inherently server-client, decreasing load on servers by sharing between clients was not a design goal.

_dain_ 4y ago

isn't that just bittorrent?

7steps2much 4y ago

I mean ... At that point you could just include a caching/proxy layer at your router.

Ekaros 4y ago

Which peers? Connected how? Sounds simple, but number of cases where this would happen is likely pretty low.

capableweb 4y ago

If you read some of the original documents about the web, you'll see that it was one of the ideas. It never made it further than being an idea though. Although IPFS seems to want to implement something like that, and probably much easier to do in a content-addressable way (like IPFS). So content has a hash, and as long as you trust the source of the hash, you can download the content of a hash from anywhere, even your drunk and slightly annoying neighbor that you don't actually trust, because you can verify that you got the right bytes after all. Bittorrent does it for transfers already, and it works alright, so why not for the web too?

kevin_thibedeau 4y ago

That doesn't work with dynamic pages. A simple timestamp change would invalidate the hash between HEAD and GET.

sowbug 4y ago
For sure. Content addressing doesn't help if the content is dynamic.

cesarb 4y ago

It's not exactly the same, but back in the days before "SSL all the things!" became the norm, it was common to have transparent Squid caching proxies on local networks. If a peer on your network had recently requested a web resource, it was very likely that it was still in the Squid cache, and it would be loaded from there, instead of going to the origin server. Yes, it was a pain whenever the origin server had misconfigured caching headers. The "force reload bypassing caches" shortcut (Shift-Control-R, or Shift-Click-Reload-Button) was always handy. The move to TLS everywhere, which killed these "transparent" (when they worked) caching proxies, made the web much more stable and predictable.

snvzz 4y ago
LANs would usually have non-transparent proxies, which prevent the issues of transparent proxies and can also work with HTTPS. Some ISPs, on the other hand, did run "transparent" proxies in their networks, causing a range of issues.

digitalsushi 4y ago

if you have enough people (2, with similar interests at minimum, or a satellite office of workers, etc) a perimiter web proxy is a great way to help get a lot of this behavior. so long as you dont have secrets, it works great

deleted 4y ago

[deleted]