5 ms·

Interestingly enough Van Jacobson has continued in this line of research, but the future of Internet networking looks radically different than what was proposed

Interestingly enough Van Jacobson has continued in this line of research, but the future of Internet networking looks radically different than what was proposed in these early days. A pivotal recognition is that the Internet nowadays is dominated by content, not by endpoints. The modern CDN is really a very complex kludge in order to deal with what we now believe the Internet's primary purpose to be: content delivery. The Internet was originally designed in a TCP/IP point-to-point structure, with the most important aspects being the communication of two parties. Now the most important part is that a given party (or really many parties) receive a specific piece of content, regardless of where that content comes from. In this light, it is wise to redesign the Internet's infrastructure against standard TCP/IP and instead build knowledge of content and caching layers directly into the transport protocols. The preliminary paper which deals with many of these issues is Jacobson's Networking Named Content[1]. The general idea is that one requests content instead of communication with a specific party. ISPs and other intermediate layers cache content blocks (because its in their own financial best interests by saving money in peering costs and pure networking hardware costs) and your request for content is delivered via the nearest available hot cache. Many other papers have been written with their own tweaks, but this is the general idea for what the next generation of the Internet should look like (from the distributed systems researcher perspective). In my Distributed Systems in Challenging Environments class one of the things my partners and I were working on was building a prototype of CCN (the protocol described in Jacobson's paper) on top of the current TCP/IP layer to examine its real-world characteristics and swarm behavior. We never completely finished the prototype for Contemplating Content Centric Protocols, but it was an interesting project nonetheless. [1] http://conferences.sigcomm.org/co-next/2009/papers/Jacobson.pdf http://conferences.sigcomm.org/co-next/2009/papers/Jacobson....

tptacek 15y ago
This is such a great thread. I've been hoping to see something tangible from VJ on this for awhile now. He was doing content-based networking even back when he was at Cisco. To an extent I think recognized by a lot of networking researchers, including IIRC YC's Robert Morris at MIT, the future of the Internet is probably not IPv6 and a network consisting of operating systems addressed by scalar integers. Instead, IP-type protocols are going to assume the same role that Ethernet and 802.11 (and ATM) play today, as a substrate for connectivity on top of which real applications will be built. The wide area discovery and "session" (OSI style) management roles played by IP and TCP will be played by overlay networks. We have primitive overlays now (DHTs, BitTorrent, Skype) and proprietary ones (CDNs like Akamai, whatever the cable content providers are using to push VOD to the edge), but eventually we'll get a general-purpose open one and the game will be on. I think this follows straightforwardly from Reed's end-to-end argument. Intelligence belongs to the edges, not the core of the network; intelligence at the core is necessarily a lowest-common-denominator affair, and is hard to scale. That's a key reason why we don't have multicast today. Routers are so bad at scale that tier 1 NSPs filtered BGP announcements by prefix length, because they couldn't handle /32-granular routing for even the tiny subset of machines that actually wanted it (begging the question of how we ever expect IPv6 to do the things the typical HN reader hopes they'll do, like provide them with their own portable /16-equivalents). Multicast effectively asked those same overtaxed routers to address web pages, individually. That's obviously not going to work when the service model involves random people demanding that AT&T and Level3 add routing table entries for their podcasts. But it scales just fine when the thousands of machines interested in those podcasts share a protocol and an infrastructure for arranging a fan-out overlay. So many unsolved problems here, from "what is the most reasonable kernel of overlay assembly, group management, and routing to deploy to end-systems" to "how do we avoid congestion collapse in arbitrary group messaging protocols" (a study of multicast reliability algorithms --- at least, from the late 90s when I last did this stuff --- will probably horrify you). But the great thing about it is, nobody has to ask permission to make this stuff work; we can evolve to the real next Internet without getting Comcast's permission, or for that matter (more perniciously) the IETF's.
- Locke1689 15y ago
  Yes, there are a huge number of advances to be made in this area. IP multicast was certainly the first draft in making this kind of networking possible but the fundamental problem is the incompatibility with the rest of the way the Internet is run. The problem we were trying to address in the graduate class is that Jacobson's paper simply assumes that the Internet suddenly "switches," like a lightbulb, to this new method of routing. I think everyone agrees that something that looks like CCN will become the future, but we were interested in the road to get there. We were trying to examine the problems associated with porting to this kind of network without preexisting infrastructure in place. One thing that you may find interesting is that the network tends to look more like a very layered Bittorrent network in current IP infrastructure, so it may end up that the most effective research ground for an intermediate protocol would be in adapting Bittorrent. The second area that I think is really interesting is secure and (perhaps more importantly) authenticated communication in this protocol. There's an entire other paper cited at the bottom of the CCN paper that I linked to that details the cryptographic authentication used in CCN but there are a number of things that I think either need more detail to be worked through or put me on the edge a little.
  - tptacek 15y ago
    We got funded for this idea, by Sony, during the first bubble. Before BitTorrent, when FEC-based multicast file transfer was either FLID/DL (an IETF standard that never went anywhere) or the startup by the guy who invented Tornado codes. We had a centralized tree-structured directory for discovery, and then (at first) deployable nodes running group messaging and running a weighted link-state routing protocol (more or less cribbed from OSPF), then later a small kernel message forwarding scheme with a programmable control plane so we could build arbitrary routing protocols in Tcl. Our initial application was chat (we overengineered, like, a little) and we pivoted to streaming video. We died in part because we hadn't the slightest clue what we were doing, and in part because the VCs replaced our management with a team that decided our best bet was to take our superior technology and go head-to-head with Akamai with our own streaming video CDN. We'd have been better off just open-sourcing. Anyhow: I'm obviously biased, but the way I think this is going to happen is, some open source project to build arbitrary overlays is going to catch on (the overlay will be a means-to-an-end for some killer app people actually care about; my bet is, telepresence).

tptacek 15y ago

This is such a great thread. I've been hoping to see something tangible from VJ on this for awhile now. He was doing content-based networking even back when he was at Cisco. To an extent I think recognized by a lot of networking researchers, including IIRC YC's Robert Morris at MIT, the future of the Internet is probably not IPv6 and a network consisting of operating systems addressed by scalar integers. Instead, IP-type protocols are going to assume the same role that Ethernet and 802.11 (and ATM) play today, as a substrate for connectivity on top of which real applications will be built. The wide area discovery and "session" (OSI style) management roles played by IP and TCP will be played by overlay networks. We have primitive overlays now (DHTs, BitTorrent, Skype) and proprietary ones (CDNs like Akamai, whatever the cable content providers are using to push VOD to the edge), but eventually we'll get a general-purpose open one and the game will be on. I think this follows straightforwardly from Reed's end-to-end argument. Intelligence belongs to the edges, not the core of the network; intelligence at the core is necessarily a lowest-common-denominator affair, and is hard to scale. That's a key reason why we don't have multicast today. Routers are so bad at scale that tier 1 NSPs filtered BGP announcements by prefix length, because they couldn't handle /32-granular routing for even the tiny subset of machines that actually wanted it (begging the question of how we ever expect IPv6 to do the things the typical HN reader hopes they'll do, like provide them with their own portable /16-equivalents). Multicast effectively asked those same overtaxed routers to address web pages, individually. That's obviously not going to work when the service model involves random people demanding that AT&T and Level3 add routing table entries for their podcasts. But it scales just fine when the thousands of machines interested in those podcasts share a protocol and an infrastructure for arranging a fan-out overlay. So many unsolved problems here, from "what is the most reasonable kernel of overlay assembly, group management, and routing to deploy to end-systems" to "how do we avoid congestion collapse in arbitrary group messaging protocols" (a study of multicast reliability algorithms --- at least, from the late 90s when I last did this stuff --- will probably horrify you). But the great thing about it is, nobody has to ask permission to make this stuff work; we can evolve to the real next Internet without getting Comcast's permission, or for that matter (more perniciously) the IETF's.

Locke1689 15y ago
Yes, there are a huge number of advances to be made in this area. IP multicast was certainly the first draft in making this kind of networking possible but the fundamental problem is the incompatibility with the rest of the way the Internet is run. The problem we were trying to address in the graduate class is that Jacobson's paper simply assumes that the Internet suddenly "switches," like a lightbulb, to this new method of routing. I think everyone agrees that something that looks like CCN will become the future, but we were interested in the road to get there. We were trying to examine the problems associated with porting to this kind of network without preexisting infrastructure in place. One thing that you may find interesting is that the network tends to look more like a very layered Bittorrent network in current IP infrastructure, so it may end up that the most effective research ground for an intermediate protocol would be in adapting Bittorrent. The second area that I think is really interesting is secure and (perhaps more importantly) authenticated communication in this protocol. There's an entire other paper cited at the bottom of the CCN paper that I linked to that details the cryptographic authentication used in CCN but there are a number of things that I think either need more detail to be worked through or put me on the edge a little.
- tptacek 15y ago
  We got funded for this idea, by Sony, during the first bubble. Before BitTorrent, when FEC-based multicast file transfer was either FLID/DL (an IETF standard that never went anywhere) or the startup by the guy who invented Tornado codes. We had a centralized tree-structured directory for discovery, and then (at first) deployable nodes running group messaging and running a weighted link-state routing protocol (more or less cribbed from OSPF), then later a small kernel message forwarding scheme with a programmable control plane so we could build arbitrary routing protocols in Tcl. Our initial application was chat (we overengineered, like, a little) and we pivoted to streaming video. We died in part because we hadn't the slightest clue what we were doing, and in part because the VCs replaced our management with a team that decided our best bet was to take our superior technology and go head-to-head with Akamai with our own streaming video CDN. We'd have been better off just open-sourcing. Anyhow: I'm obviously biased, but the way I think this is going to happen is, some open source project to build arbitrary overlays is going to catch on (the overlay will be a means-to-an-end for some killer app people actually care about; my bet is, telepresence).