5 ms·

It also sounds like a cool caching technique : since people already have the resource cached on their local system, why not allow them to distribute it? HTTP h

It also sounds like a cool caching technique : since people already have the resource cached on their local system, why not allow them to distribute it? HTTP headers can allow to know when to flush cache (just like it's currently done) and provide last known md5/sha1/whatever digest to make sure page is not tempered with (let's say it's checked when the download is complete, and retry a download if the signature does not match: it should not happen often anyway). It obviously won't work for pages which distribute auth related content, but it would be great for assets. I guess a problem could be that page load will be slower (depends on the ability to parallelize and to contact geographically close peers, I suppose), but it would mean way less heavy load on servers.

NTripleOne 10y ago
>It also sounds like a cool caching technique : since people already have the resource cached on their local system, why not allow them to distribute it? Sounds like a cool way to deter people from using it if you ask me.
- puddintane 10y ago
  Especially with Comcast now introducing a 1 tb cap to its non-business users.
  - oelmekki 10y ago
    I'm not from USA, am I correct to think Comcast is a mobile ISP? If so, this is a non issue : mobile apps can already decide not to do big update when not on wifi, browsers can do the same.
    - bespoke_engnr 10y ago
      Unfortunately Comcast is one of the two companies (along with Verizon) that have monopolized the ISP market in the US. Together they own most of the market, and have agreements to stay out of each others' territories. This affects something like 40% of home Internet customers in the US :(.
    - theandrewbailey 10y ago
      That is incorrect. Comcast is the (second?[0]) largest cable ISP in the USA. About 10 years ago, they were calling people saying that they were using too much data, but that was always nebulous and varied greatly. Then they defined a 250 GB cap in 2008, and later, 300 GB. They only recently expanded it to 1TB. Go over it for three months, and Comcast will get very angry. I'm not sure if the cap is in effect for places with fiber internet (like Verizon FIOS). It wouldn't surprise me, since they've admitted caps are unnecessary.[1] [0] Charter + Time Warner Cable (after merge) is larger, I think. [1] https://consumerist.com/2015/11/06/leaked-comcast-doc-admits-data-caps-have-nothing-to-do-with-congestion/ https://consumerist.com/2015/11/06/leaked-comcast-doc-admits...
      - oelmekki 10y ago
        I see, thanks. This sounds quite terrible actually : if internet has any historical impact, the last thing you want is for people to think about "not using too much data". I hope this won't last.
        theandrewbailey 10y ago
        Many big American ISPs are too greedy. Even worse is that most people have no clue how much a gigabyte of data is. About the time Chairman Wheeler was aiming for net neutrality, he was also looking at data caps and if they were bad. Within a month, Comcast increased theirs to 1 TB. Coincidence? There's going to be a lot about how 'competitive' ISPs are in the next few years in America now that everything's Republican. But most people have two options: expensive cable ISP that's fast when it wants to be, and slow DSL that's also expensive and not getting better.
  - sametmax 10y ago
    Even when I download non stop torrent movies I don't reach this in one month. I doubt it would be a problem, especially since the load would be balanced across all users.
    - lacksconfidence 10y ago
      Try a different quality? Best quality on amazon video is 7GB per hour, works out to about 150 hours a month. Sure this is a lot of video, but spread across a household with say 4 people with their own screens and viewing habits is under 1.5 hours per day per person
    - deleted 10y ago
      [deleted]
    - xj9 10y ago
      i nearly reached my cap in five days copying data from my NAS to a cloud server last week. i only have ~100GB left for the rest of the month.
    - stordoff 10y ago
      If you download games, you can cut into a 1TB cap pretty quickly. Last I checked, Halo 5 on Xbox One was >90GB.
    - puddintane 10y ago
      Streaming Netflix alone can cap your bandwidth quickly given the statistics from the Netflix page [1]. I have recently experienced this actually (I do not run torrents from home and during the month of December I did not transfer anything from the remote box). Month of December I used 1022 GB out of 1024 GB over 900 GB was from NetFlix. Using: 1 terabyte = 8,388,608 megabits Ultra HD: 25 mbps ~= 93.2 hours HD: 5 mbps ~= 19.5 days SD: Would be good but hey I bought an HD TV for a reason so that would not be fair to skimp out on quality just because they don't want to introduce cheaper bandwidth caps. side note It is $10 per 50 GB once you go over the cap with Comcast which is pretty huge for overhead if a user wasn't even aware of the cap. Thankfully they cap it at $200 over your bill. side note2 this number will only increase - so really that 1 TB cap already needs to see a lift to 2 TB just to start to fulfill the requirements of the newer age web. [1] https://help.netflix.com/en/node/306 https://help.netflix.com/en/node/306
  - ReverseCold 10y ago
    ATT doesn't have a cap for gigabit customers. Then again I also have a choice of one other ISP, so I will just switch again if they introduce a cap.
    - tracker1 10y ago
      For me the choice is cable or dsl... and dsl is pretty painfully slow, and the carrier is much less responsive to issues. Fortunately my cable provider doesn't charge much more for a business connection ($140 vs $90, compared to comcast charging 3x as much). Cox tends to respond to technical issues within hours (onsite) and not days, and I don't have any cap issues. Anything gray (tv torrents) I now use a seedbox or vpn for.
rakoo 10y ago
It'd be a huge downgrade in privacy. As much as we don't like it, when I'm watching cat videos only Google knows; with bittorrent, everybody knows what you're watching. As much as I love bittorrent (I really do), this is an aspect for which I don't see easy solutions.
- oelmekki 10y ago
  Indeed. Maybe it should include Tor like features as well :)
  - oelmekki 10y ago
    I realize that this would not work as well : tor is routing traffic through exit nodes, so it would basically cancel the advantages of p2p distribution. The privacy concern makes me realize something else : there's no incentive for users here. They lose privacy, what do they win? It does not prevent totally the idea of using p2p to distribute content as a low level implementation, but since we (the ones who manage servers) are the only ones to benefit from it, we must first find a way for it to have no impact on users. The biggest privacy problem is because of how p2p works currently : we have a list of IPs associated to a resource. How can we obfuscate this without going through proxies?
    - sametmax 10y ago
      Not "tor". "tor-like". Onion routing. Something that would make sure only your closest neighbor knows what you request, and won't know if it's for you or your closest neighbor.
      - oelmekki 10y ago
        Indeed, local disclosure is a cool idea to mitigate history leak. This makes me think that an other feature could be to not disclose all peers available, but randomly select some. This would force someone wanting to look someone up to download a possible big amount of times the same list to check for an ip, instead of just pulling it once per ressource to know as a fact. Both ideas are not exactly privacy shields, but steps to mitigate the problem.
digi_owl 10y ago
The likes of IPFS and Freenet work on this principle.
takeda 10y ago
I like https://named-data.net/ https://named-data.net/ approach better, although that requires infrastructure. In short their approach is that instead of connecting point to point and using addresses of hosts, what if we could address based on the data we want. This suddenly makes routers aware of what data they are forwarding, that knowledge allows them to start caching and reuse the same data packets when multiple people request the same data. Things like multicasting or multi path forwarding are simpler. Interestingly, the more people are viewing the same video for example the better, essentially CDN is no longer needed. Most of use cases how we currently use the Internet, are actually easier to do this way. Things that are harder though (but not impossible to do) is point to point communication, such as SSH.
- oelmekki 10y ago
  This sounds super cool, thanks! (note to others: the description of project can be found here https://named-data.net/project/ https://named-data.net/project/ ) Actually, what could be done without "rebooting" the whole internet is to use caching relays, which would operate just like DNS servers do, but caching content instead of IPs (maybe it's what they're suggesting, it was unclear from general description). EDIT : but I wonder if I'm really ready to deal, as a webdev, with content propagation based on TTL, like we do with dns zones :D EDIT 2 : actually, this is a non issue. TTL is there because when caching something as short as a IP string, you don't want to have to issue a request to parent node. But when caching assets, it won't be that a problem to issue a HEAD http request to asset owner to ask if the cached content should be invalidated.

NTripleOne 10y ago

>It also sounds like a cool caching technique : since people already have the resource cached on their local system, why not allow them to distribute it? Sounds like a cool way to deter people from using it if you ask me.

puddintane 10y ago
Especially with Comcast now introducing a 1 tb cap to its non-business users.
- oelmekki 10y ago
  I'm not from USA, am I correct to think Comcast is a mobile ISP? If so, this is a non issue : mobile apps can already decide not to do big update when not on wifi, browsers can do the same.
  - bespoke_engnr 10y ago
    Unfortunately Comcast is one of the two companies (along with Verizon) that have monopolized the ISP market in the US. Together they own most of the market, and have agreements to stay out of each others' territories. This affects something like 40% of home Internet customers in the US :(.
  - theandrewbailey 10y ago
    That is incorrect. Comcast is the (second?[0]) largest cable ISP in the USA. About 10 years ago, they were calling people saying that they were using too much data, but that was always nebulous and varied greatly. Then they defined a 250 GB cap in 2008, and later, 300 GB. They only recently expanded it to 1TB. Go over it for three months, and Comcast will get very angry. I'm not sure if the cap is in effect for places with fiber internet (like Verizon FIOS). It wouldn't surprise me, since they've admitted caps are unnecessary.[1] [0] Charter + Time Warner Cable (after merge) is larger, I think. [1] https://consumerist.com/2015/11/06/leaked-comcast-doc-admits-data-caps-have-nothing-to-do-with-congestion/ https://consumerist.com/2015/11/06/leaked-comcast-doc-admits...
    - oelmekki 10y ago
      I see, thanks. This sounds quite terrible actually : if internet has any historical impact, the last thing you want is for people to think about "not using too much data". I hope this won't last.
      - theandrewbailey 10y ago
        Many big American ISPs are too greedy. Even worse is that most people have no clue how much a gigabyte of data is. About the time Chairman Wheeler was aiming for net neutrality, he was also looking at data caps and if they were bad. Within a month, Comcast increased theirs to 1 TB. Coincidence? There's going to be a lot about how 'competitive' ISPs are in the next few years in America now that everything's Republican. But most people have two options: expensive cable ISP that's fast when it wants to be, and slow DSL that's also expensive and not getting better.
- sametmax 10y ago
  Even when I download non stop torrent movies I don't reach this in one month. I doubt it would be a problem, especially since the load would be balanced across all users.
  - lacksconfidence 10y ago
    Try a different quality? Best quality on amazon video is 7GB per hour, works out to about 150 hours a month. Sure this is a lot of video, but spread across a household with say 4 people with their own screens and viewing habits is under 1.5 hours per day per person
  - deleted 10y ago
    [deleted]
  - xj9 10y ago
    i nearly reached my cap in five days copying data from my NAS to a cloud server last week. i only have ~100GB left for the rest of the month.
  - stordoff 10y ago
    If you download games, you can cut into a 1TB cap pretty quickly. Last I checked, Halo 5 on Xbox One was >90GB.
  - puddintane 10y ago
    Streaming Netflix alone can cap your bandwidth quickly given the statistics from the Netflix page [1]. I have recently experienced this actually (I do not run torrents from home and during the month of December I did not transfer anything from the remote box). Month of December I used 1022 GB out of 1024 GB over 900 GB was from NetFlix. Using: 1 terabyte = 8,388,608 megabits Ultra HD: 25 mbps ~= 93.2 hours HD: 5 mbps ~= 19.5 days SD: Would be good but hey I bought an HD TV for a reason so that would not be fair to skimp out on quality just because they don't want to introduce cheaper bandwidth caps. side note It is $10 per 50 GB once you go over the cap with Comcast which is pretty huge for overhead if a user wasn't even aware of the cap. Thankfully they cap it at $200 over your bill. side note2 this number will only increase - so really that 1 TB cap already needs to see a lift to 2 TB just to start to fulfill the requirements of the newer age web. [1] https://help.netflix.com/en/node/306 https://help.netflix.com/en/node/306
- ReverseCold 10y ago
  ATT doesn't have a cap for gigabit customers. Then again I also have a choice of one other ISP, so I will just switch again if they introduce a cap.
  - tracker1 10y ago
    For me the choice is cable or dsl... and dsl is pretty painfully slow, and the carrier is much less responsive to issues. Fortunately my cable provider doesn't charge much more for a business connection ($140 vs $90, compared to comcast charging 3x as much). Cox tends to respond to technical issues within hours (onsite) and not days, and I don't have any cap issues. Anything gray (tv torrents) I now use a seedbox or vpn for.

rakoo 10y ago

It'd be a huge downgrade in privacy. As much as we don't like it, when I'm watching cat videos only Google knows; with bittorrent, everybody knows what you're watching. As much as I love bittorrent (I really do), this is an aspect for which I don't see easy solutions.

oelmekki 10y ago
Indeed. Maybe it should include Tor like features as well :)
- oelmekki 10y ago
  I realize that this would not work as well : tor is routing traffic through exit nodes, so it would basically cancel the advantages of p2p distribution. The privacy concern makes me realize something else : there's no incentive for users here. They lose privacy, what do they win? It does not prevent totally the idea of using p2p to distribute content as a low level implementation, but since we (the ones who manage servers) are the only ones to benefit from it, we must first find a way for it to have no impact on users. The biggest privacy problem is because of how p2p works currently : we have a list of IPs associated to a resource. How can we obfuscate this without going through proxies?
  - sametmax 10y ago
    Not "tor". "tor-like". Onion routing. Something that would make sure only your closest neighbor knows what you request, and won't know if it's for you or your closest neighbor.
    - oelmekki 10y ago
      Indeed, local disclosure is a cool idea to mitigate history leak. This makes me think that an other feature could be to not disclose all peers available, but randomly select some. This would force someone wanting to look someone up to download a possible big amount of times the same list to check for an ip, instead of just pulling it once per ressource to know as a fact. Both ideas are not exactly privacy shields, but steps to mitigate the problem.

digi_owl 10y ago

The likes of IPFS and Freenet work on this principle.

takeda 10y ago

I like https://named-data.net/ https://named-data.net/ approach better, although that requires infrastructure. In short their approach is that instead of connecting point to point and using addresses of hosts, what if we could address based on the data we want. This suddenly makes routers aware of what data they are forwarding, that knowledge allows them to start caching and reuse the same data packets when multiple people request the same data. Things like multicasting or multi path forwarding are simpler. Interestingly, the more people are viewing the same video for example the better, essentially CDN is no longer needed. Most of use cases how we currently use the Internet, are actually easier to do this way. Things that are harder though (but not impossible to do) is point to point communication, such as SSH.

oelmekki 10y ago
This sounds super cool, thanks! (note to others: the description of project can be found here https://named-data.net/project/ https://named-data.net/project/ ) Actually, what could be done without "rebooting" the whole internet is to use caching relays, which would operate just like DNS servers do, but caching content instead of IPs (maybe it's what they're suggesting, it was unclear from general description). EDIT : but I wonder if I'm really ready to deal, as a webdev, with content propagation based on TTL, like we do with dns zones :D EDIT 2 : actually, this is a non issue. TTL is there because when caching something as short as a IP string, you don't want to have to issue a request to parent node. But when caching assets, it won't be that a problem to issue a HEAD http request to asset owner to ask if the cached content should be invalidated.