5 ms·

While I agree that TCP/IP is decentralized, unfortunately due to NAT and corporate firewalls, vast swaths of users have been relegated to second class citizens

While I agree that TCP/IP is decentralized, unfortunately due to NAT and corporate firewalls, vast swaths of users have been relegated to second class citizens who can only act as data consumers. Unfortunately this same misguided approach to security, or as I call it suckerity, has been carried forward into IPv6. There is some hope that WebRTC and other p2p technologies might alleviate the problem but then the task of punching through firewalls with techniques like STUN falls on developers. I'm not seeing an industry-led campaign to standardize a true p2p protocol based on something like UDP, because it would undermine the profits of data producers. I could go into how the proper layering of TCP should have been IP->UDP->TCP instead of IP->TCP and a whole host of technical mistakes such as inability to set the number of checksum bits, use alternatives to TCP header compression, or even use jumbo frames over broadband, but what it all comes down to is that what we think of as “the internet” (connection-oriented reliable streams and the client/server model) is not really compatible with a distributed content-addressable connectionless internet that can work with high latencies and packet loss. I think if this network existed, then extending HTTP to operate in a p2p fashion wouldn’t be all that complicated. Probably the way it would work is that you’d request data by hash instead of by address, and local caches would send you the pieces that match like BitTorrent. Most of the complexity will center around security, but I think Bitcoin and other networks of trust point the way to being able to both prove one’s identity and surf anonymously. Tim Berners-Lee may not be thinking about the problem at this level but we need more people with his clout to get the ball rolling. I wasted a lot of time attempting to write a low-level stream API over UDP for networked games before zeromq or WebRTC existed and ended up failing because there are just too many factors that made it nondeterministic. It’s going to have to be a group effort and will probably require funding or at least a donation of talent from people familiar with the pitfalls to get it right. Otherwise I just don’t think a new standard (one that makes recommendations instead of solving the heart of the problem) is going to catch on.

jnbiche 13y ago
This is exactly the problem. We need a better transport layer. You know, the speed at which SPDY has caught on has given me hope. Admittedly, that's at the application layer, but I think if Google were to get behind an initiative to create a better TCP, it would actually have a chance at working. Historically, Google's incentives were properly aligned with a decentralized Internet, but Google is more and more a content producer. So I'm not sure. Until then, we'll have to rely on servers to bootstrap peer-to-peer networks, even Bitcoin, Bittorrent, and yes, WebRTC.
- rubiquity 13y ago
  > but I think if Google were to get behind an initiative to create a better TCP, it would actually have a chance at working. I don't think Google has much interest in a decentralized web. Google, along with Facebook, are two of the prime examples of a centralized web.
  - dclara 13y ago
    Yeah, you got the point. They are on the list of "reliant on big companies, and one big server". But "support geo-distributed clusters" remains an open task on Google's radar. See here: http://highscalability.com/google-architecture http://highscalability.com/google-architecture Unfortunately, with the existing architecture and infrastructure along with the team resources, they are focusing more on the algorithms of machine intelligence and AI.
- oscargrouch 13y ago
  >You know, the speed at which SPDY has caught on has given me hope. Admittedly, that's at the application layer, but I think if Google were to get behind an initiative to create a better TCP, it would actually have a chance at working. You might find QUIC interesting (being implemented on chrome): http://blog.chromium.org/2013/06/experimenting-with-quic.html http://blog.chromium.org/2013/06/experimenting-with-quic.htm... The "SPDY" application layer its implemented on top of it.. working at UDP.. so in theory, one can implement other sorts of applications layers on top of it..
  - jnbiche 13y ago
    Oh, thank you. I hadn't heard of QUIC. It's tunneled through UDP...does that mean it's NAT-friendly? Presumably so.
    - M2Ys4U 13y ago
      >It's tunneled through UDP...does that mean it's NAT-friendly? Presumably so. Not as NAT friendly as you might think. There's a fair bit of discussion in the QUIC design document[0] around how NATs screw up the protocol and how to mitigate the problem. [0] https://docs.google.com/document/d/1RNHkx_VvKWyWg6Lr8SZ-saqsQx7rFV-ev2jRFUoVD34/edit#heading=h.7mlx5id9bdry https://docs.google.com/document/d/1RNHkx_VvKWyWg6Lr8SZ-saqs...
zwegner 13y ago
As far as a decentralized content-addressable network goes, the swift protocol[0] seems pretty close to what you're talking about. I heard about it here a while back, and was quite interested in it, but it doesn't look like it's moved much lately (the page has a copyright date of 2010). [0] http://libswift.org/ http://libswift.org/
- gritzko 13y ago
  (Co)author here. The libswift effort got converted into IETF PPSP eventually https://datatracker.ietf.org/doc/draft-ietf-ppsp-peer-protocol/ https://datatracker.ietf.org/doc/draft-ietf-ppsp-peer-protoc... Also, folks in Sweden tinker with the codebase, mostly from the video distribution angle.
amirmc 13y ago
Or a bunch of University research. This is exactly the kind of complex problem that research groups work on but they tend not to get much attention until they're fully baked. The group I'm working with is considering what kind toolstack we need for a distributed future and then building it from the ground up. If you're interested, you can find out more from the link below. One of the pieces hit 1.0 late last year and we're working on the other components too. http://nymote.org/blog/2013/introducing-nymote/ http://nymote.org/blog/2013/introducing-nymote/

jnbiche 13y ago

This is exactly the problem. We need a better transport layer. You know, the speed at which SPDY has caught on has given me hope. Admittedly, that's at the application layer, but I think if Google were to get behind an initiative to create a better TCP, it would actually have a chance at working. Historically, Google's incentives were properly aligned with a decentralized Internet, but Google is more and more a content producer. So I'm not sure. Until then, we'll have to rely on servers to bootstrap peer-to-peer networks, even Bitcoin, Bittorrent, and yes, WebRTC.

rubiquity 13y ago
> but I think if Google were to get behind an initiative to create a better TCP, it would actually have a chance at working. I don't think Google has much interest in a decentralized web. Google, along with Facebook, are two of the prime examples of a centralized web.
- dclara 13y ago
  Yeah, you got the point. They are on the list of "reliant on big companies, and one big server". But "support geo-distributed clusters" remains an open task on Google's radar. See here: http://highscalability.com/google-architecture http://highscalability.com/google-architecture Unfortunately, with the existing architecture and infrastructure along with the team resources, they are focusing more on the algorithms of machine intelligence and AI.
oscargrouch 13y ago
>You know, the speed at which SPDY has caught on has given me hope. Admittedly, that's at the application layer, but I think if Google were to get behind an initiative to create a better TCP, it would actually have a chance at working. You might find QUIC interesting (being implemented on chrome): http://blog.chromium.org/2013/06/experimenting-with-quic.html http://blog.chromium.org/2013/06/experimenting-with-quic.htm... The "SPDY" application layer its implemented on top of it.. working at UDP.. so in theory, one can implement other sorts of applications layers on top of it..
- jnbiche 13y ago
  Oh, thank you. I hadn't heard of QUIC. It's tunneled through UDP...does that mean it's NAT-friendly? Presumably so.
  - M2Ys4U 13y ago
    >It's tunneled through UDP...does that mean it's NAT-friendly? Presumably so. Not as NAT friendly as you might think. There's a fair bit of discussion in the QUIC design document[0] around how NATs screw up the protocol and how to mitigate the problem. [0] https://docs.google.com/document/d/1RNHkx_VvKWyWg6Lr8SZ-saqsQx7rFV-ev2jRFUoVD34/edit#heading=h.7mlx5id9bdry https://docs.google.com/document/d/1RNHkx_VvKWyWg6Lr8SZ-saqs...

zwegner 13y ago

As far as a decentralized content-addressable network goes, the swift protocol[0] seems pretty close to what you're talking about. I heard about it here a while back, and was quite interested in it, but it doesn't look like it's moved much lately (the page has a copyright date of 2010). [0] http://libswift.org/ http://libswift.org/

gritzko 13y ago
(Co)author here. The libswift effort got converted into IETF PPSP eventually https://datatracker.ietf.org/doc/draft-ietf-ppsp-peer-protocol/ https://datatracker.ietf.org/doc/draft-ietf-ppsp-peer-protoc... Also, folks in Sweden tinker with the codebase, mostly from the video distribution angle.

amirmc 13y ago

Or a bunch of University research. This is exactly the kind of complex problem that research groups work on but they tend not to get much attention until they're fully baked. The group I'm working with is considering what kind toolstack we need for a distributed future and then building it from the ground up. If you're interested, you can find out more from the link below. One of the pieces hit 1.0 late last year and we're working on the other components too. http://nymote.org/blog/2013/introducing-nymote/ http://nymote.org/blog/2013/introducing-nymote/