12 ms·
Vint Cerf on mistakes he made in TCP/IP
- ksec 3y agoHad TCP/IP been 64bit to begin with things would be a lot different.
- m348e912 3y agoFor the time 32bit was more than adequate, not to mention routers only had so much memory and processing power.
- elihu 3y agoFunny enough, one of the arguments for longer key lengths now is to save memory. With IPV4 the way it is now, the address space is very fragmented so you need a lot of router table entries. More available address space means more contiguous address blocks and therefore fewer router table entries. That wasn't an issue back when IPv4 was being defined though.
- convolvatron 3y agoThe issue here isn’t the size of the address, but the way it’s allocated and managed. Since the provider block routing scheme was carried forward while cloth - realistically the most were going to get is a deprecation of the really tiny blocks - ‘the swamp’.
- em3rgent0rdr 3y agoIt is possible 64-bits might have discouraged adopters who didn't want to waste expensive unnecessary extra bits at the time, so TCP/IP might not have taken off.
- kragen 3y agothe ipv4+tcpv4 header is minimally 320 bits long so i don't think so
- jandrese 3y agoRouters don't care about the TCP header though, they only care about the 20 bytes of IP header. The interfaces on the router also care about the 14 byte MAC header, but that's a separate step. That said I agree that the world likely would have sucked it up and just gone with the 64 bit addresses, but there would have been a whole lot of grumbling for decades about the memory use. It's hard to imagine these days, but back then memory was outrageously expensive and enormous amounts of engineering went into minimizing memory use wherever possible. Having these addresses where we wouldn't even touch half of the bits for decades would have been unpopular with a lot of people, even if he would have been praised for being so forward thinking in the end. On the other hand, the transition to IPv6 does give us a chance to fix some of the longstanding defects in IPv4 that would otherwise be baked into the protocol until the end of time, so it's not all bad.
- imtringued 3y agoWe live in a NAT and CGNAT world. Routers do care about the TCP header.
- detaro 3y agonow. Not at the time were "adopting TCP or not" was a question.
- FullyFunctional 3y ago"Having these addresses where we wouldn't even touch half of the bits for decades would have been unpopular with a lot of people, even if he would have been praised for being so forward thinking in the end." Worse perhaps, people, algorithms, hardware, ... will all start to assume those bits are zero and start optimizing for that. We have seen this happen repeatedly throughout history (eg. Lisp pointers would stuff tags in the "spare" bits in addresses, making implementations non-portable). These are all hard problems, but the lesson I take away is to 1. Always have a plan for evolving the design/protocol/API/... 2. Aim for a watertight design that supports 1. (Example: TLS 1.3 puts random stuff into reserved fields so nobody can assume they are zero). 3. Really stretch your imagination about future usages (assume 50+ years) 4. Have a ridiculously comprehensive test suite for both clients and servers It seems clear that in terms of lifetimes, hardware gets used much longer than expected, software even more so (Y2K anyone), but protocols/formats/specs/... literally never dies
- bushbaba 3y agoI really wish we just did ipv4 on 64 but addresses. IPv6 is nearly a totally different paradigm, and has lead to adoption pain. Biggest being usability taking a back seat. People are being forced into using ipv6 not cause they want to, but from ip space exhaustion.
- merlyn 3y agoIPv4 today is a totally different paradigm than what IPv4 was in the 1980's and early 1990's. IPng (IPv6) was designed when the original paradigm existed, not what we have now-a-days. So, if we actually had been on track to do IPv6 in the decade it was defined, we could have gone a totally different way. Instead due to the rapid growth of the Internet in the mid 90's, IPv4 was morphed into something different and IPv6 had its own model until we started to need it again.
- astrange 3y agoIPv6 itself isn't the right thing for mobile internet either; people expect their IP address to not change when they migrate towers, meaning the entire approach to routing doesn't work.
- hughesjj 3y agoIsn't it like literally a data structure issue though? Like you have to route on prefixes? Like how else would you know where to go? Ip routing is kind of like a tree (not a graph) in the ip regards, how would you get around it? I admittedly don't know as much as I'd like about network, hence why I'm posing this question ;-) If theres a way to efficiently model hierarchical domains for stuff like DNS/IP i'd love to know. Like, consistent hash rings maybe, but then we'd be reintroducing ring topologies... Oh wait duh though. A cell tower is owned by a single company. Yeah that's prolly doable, the company can always buy beefier routers if they need the memory. And realistically speaking the virtual tree could be closely aligned to the physical one given how cell towers work.
- 3y ago
- oh_sigh 3y agoThings also could have been a lot different if IPv6 was just IPv4 with 128 bit addresses.
- ikiris 3y agoIt basically is.
- alfons_foobar 3y agoNo.
- ikiris 3y agoYes...
- amoss 3y agoPeople hate kludges and backward compatibility, but IPv4.1 with 5-byte addresses would have been the dominant standard by now.
- XorNot 3y agoNo, it wouldn't have been. Where are you going to put that extra byte? How is an intermediate box going to know what to do with it? How is a routing ASIC going to handle it? This is thinking which appeals to people who don't understand the problem because it feels like a compromise, but you can't negotiate with silicon. If it expects a 4 byte address payload, then that's all it will ever handle. Adding more bytes is exactly the same amount of redesign and replacement work as adding more. If the problem was solely software related, it would've been done by now.
- amoss 3y agoNah, hardware people just don't negotiate hard enough. Add a one-byte option into the header, give it an unassigned value (e.g. 6) and assume that any box that does not understand it will pass it through unchanged. Use it as the most significant byte of the address, and obsolete all routing equipment that doesn't understand it from the backbone so that they can be used to handle routing in one of the new 32-bit super-A class networks. Sure it's ugly as hell, but it is just backward compatible enough to work.
- _8j50 3y agoOr... hear me out: variable length, ideally with support for ascii or other encodings allowing human readable addresses ( no dns!) and routing done using top level addresses. "AAA.CA" becomes 4143.414141, routers will find the best route for 4143 and anycast ti 414141.<more specific address>, the apex AAA in this case would also be the ASN equivalent for bgp type routing and the TLD CA would also be the PKI certificate authority used to validate both routing updates and address ownership by the applications on either end. No port numbers either, the full address should describe the layer4+ addresses, so the full address user types instead of https://AAA.CA:8443 https://AAA.CA:8443, it becomes HTTPS.8443..AAA.CA (.. meaning anycast to the closest endpoint). A good protocol is the easy part, the hard part is getting all the big networking companies and their engineers to agree on something.
- jandrese 3y agoThe hard part is handling variable length addresses fast in hardware. Historically there has always been a sizeable speed penalty for making hardware deal with variable length anything.
- _8j50 3y agoThat's why you make addressing hierarchial. With IP it was one address, 16bitx2 addressing will be fixed for the purpose of routing in that it either looks at the top level 16bit and decides the route to choose or for core routers, the destination keeps the other end of the address space in the routing table,again 16x2 on core/transit and just 16 for LAN/Host routing (higher order 16bit addresses each get their own RT, scaling as needed). Routing becomes a fast two step process ("i have a table for that, then let me lookup the route" or "I don't have the table, default route or drop"). For that small cost of additional RT lookup and memory you get faster lookups and comparisons (16 vs 32bit) and you can get small routers that support as little as 65k routes cheal for small sites and stack/scale that to meet demand. So R1 gets the packet, looks at the first 16bit and decides which R3 has the RT for that and forwards it, R3 looks at only the second 16bit due to it's role in the stack and forwards it elsewhere. Internet routers will do the most work since they will do multiple (but predictable and limited since it scales exponentially) lookups then downstream looks up less and less until the actual application gets it. 16bit to support utf16. Arbitrary length but fixed hierarchial address partitioning.
- senectus1 3y agoi dont think so really. He made the right choice. the world will adapt, eventually. I'd be willing to bet IPv6 wont be the last network upheaval we'll have as a civilization. (assuming we dont wipe ourselves out first) What do you think will happen if we generate some really really big AI, and they realize that they'll be super reliable if we made every synapse have its own IP, then we network all the AI's together as each AI to behave as its own synapse node :-D IPv256?
- Quekid5 3y agoI imagine a bigger problem with your hypothetical might be the sheer mass of such a thing. Plus, you underestimate just how mind-bogglingly huge 2^64 is. (Granted, the IPv6 address space is going to very quite sparsely populated, but still.) I mean, the estimate on the number of atoms in the observable universe is "only" 10^80, for comparison.
- clouddrover 3y ago> “I didn’t really appreciate the implications of the World Wide Web.” That's not a mistake in TCP/IP.
- astrange 3y agoQUIC doesn't run on TCP for this reason; it wants to avoid the head of line blocking issues. The earlier attempts to replace it like SCTP didn't work out because too much middleware blocks it.
- em3rgent0rdr 3y agoCerf: "I'm serious, the decision to put a 32-bit address space on there was the result of a year's battle among a bunch of engineers who couldn't make up their minds about 32, 128 or variable length. And after a year of fighting I said -- I'm now at ARPA, I'm running the program, I'm paying for this stuff and using American tax dollars -- and I wanted some progress because we didn't know if this is going to work. So I said 32 bits, it is enough for an experiment, it is 4.3 billion terminations -- even the defense department doesn't need 4.3 billion of anything and it couldn't afford to buy 4.3 billion edge devices to do a test anyway. So at the time I thought we were doing a experiment to prove the technology and that if it worked we'd have an opportunity to do a production version of it. Well -- [laughter] -- it just escaped! -- it got out and people started to use it and then it became a commercial thing. So, this [IPv6] is the production attempt at making the network scalable. Only 30 years later." [1] https://youtu.be/mZo69JQoLb8?t=815 https://youtu.be/mZo69JQoLb8?t=815 "Google IPv6 Conference 2008: What will the IPv6 Internet look like?"
- checkyoursudo 3y agoThis is a good lesson. I've done a lot of pilot studies, prototypes, proof-of-concept experiments. It is scary^1 how many times "let's just get this tested and see if it even works" turns into "welp, that worked and we don't have time or funding to do another one, so that's the final version." The constraint is often time. Gotta get a product out, or run an follow-on experiment that depended on the first, or submit a paper or funding application, or whatever. Test the thing you want to test! 1. Not scary like fear, just like, I don't know, astonishing.
- Dalewyn 3y agoAn old Russian proverb goes: There is nothing more permanent than a temporary solution.
- contingencies 3y agoNice. Added to https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
- deleted 3y ago[deleted]
- bawolff 3y ago> Before public-key cryptography came around, key distribution was a really messy manual process,” Cerf says. “It was awful, and it didn’t scale. So that’s why I didn’t try to push that into the Internet. I dont really think this is a mistake. Its not like there are great options even now for end2end security on the transport layer (like we have vpns but they mostly require having preshared secrets). It would have been pretty impossible back in the day and added a lot of friction
- unwind 3y agoI was going to post that quote too, it was really fun (and kind of jarring in a good way) to see someone use "the Internet" and really not [1] be another marketing/management/whatever person who just doesn't know the difference between the Internet and the web, so on. Loved that! [1]: I mean like really not. At all.
- tptacek 3y agoLook at the first round of IPSEC work --- a decade and a half or so later, in the late 1990s --- to see what a disaster native cryptography would have been for IPv4.
- Animats 3y agoXNS, which predates TCP, had 96-bit addresses. "IDP uses Ethernet's 48-bit address as the basis for its own network addressing, generally using the machine's MAC address as the primary unique identifier. To this is added another 48-bit address section provided by the networking equipment; 32 bits are provided by routers to identify the network number in the internetwork, and another 16 bits define a socket number for service selection within a single host." Think of the network number as corresponding to today's autonomous system number. This was Xerox's approach to scaling. It could have worked. Routinely looking up things in hashes was not common back then. Hashing was computationally expensive when routers ran on little machines with well under 1 MIPS. A huge flat address space was not feasible with memory costs at the time. Nor was keeping huge routing tables in sync well understood. There had to be some cheap way to make routing decisions based on the IP address.
- tssva 3y agoThe fixed 48-bit network field of XNS and derivative protocols such as IPX and VIP limited scalability since it didn't allow for hierarchical routing where as the subnet mask of IP effectively provides a variable length network field and greater scaling through hierarchical routing.
- captainkrtek 3y ago#4: IP Fragmentation
- p1mrx 3y agoIt probably would've been better for routers to set a "truncated" bit and chop off the rest of the packet. UDP-based applications would have to do MTU discovery, but that's much easier when you're receiving truncated packets.
- cyberax 3y agoTo be precise, MTU. The MTU should have been reified to the IP layer. And it would have been so easy! When a system sends a packet, it sets the "forward MTU" field in the IP packet to be equal to the MTU of the outgoing link. Each router along the way inspects the packet and if its outgoing link has a lower MTU, it replaces the "forward MTU" with its own. Once the packet reaches the destination, the target host copies the "forward MTU" field (which now has the lowest MTU seen along the path) to the "reflected MTU" and sets it on the next reply packet. So the originating system can discover the limiting MTU by the time it gets the ACK packet. Sigh. Instead, we have a Ruby Goldberg machine with fragmentation, ICMP and all the related unreliable crap that has forever cemented max MTU as 1500.
- unnah 3y agoThat would work only as long as the route does not change over time, and does not depend on which direction the packet is going. They probably did not want to make that assumption in the protocol, even if it turned out to hold most of the time in practice.
- cyberax 3y agoThis process should happen continuously, with every packet sent. Then the intermediary router just needs to chop the packet to its MTU and let it travel to the destination. The destination will detect that the packet has been chopped, send the updated reflected MTU and everything would be fine. Of course, the minimal allowed MTU (say, 512 bytes) should provide enough information for the target to find the correct TCP flow to which the packet belongs. No need for out-of-band ICMP signalling or fragmentation. And in practice, topology changes happen rarely, so this should be very infrequent.
- jiggawatts 3y agoIn my experience, the biggest issue with TCP is that it assumes that the traffic is a single continuous byte stream, whereas the majority of applications send messages. Vint mentioned the lack of encryption as a mistake, but even on private networks, the 16-bit TCP checksums are too weak to protect against bit-flips. Many orgs noticed that enforcing encryption reduces "random" crashes and errors, because it also enforces a strong checksum, typically 128 or 256 bit. However, even a 32-bit checksum would have sufficed for ordinary use. Almost all "real" protocols built on top of TCP eventually end up reinventing the same wheels: message boundaries, logical sub-streams with priorities, and strong checksums or encryption. Examples include Citrix's ICA, HTTP 3.0, gRPC, etc...
- zubnix 3y agoHttp 3.0 is UDP, ICA these days is also UDP.
- jayd16 3y agoYes, but by necessity and only after http/2 attempted everything listed over TCP.
- klabb3 3y agoTechnically true, but parent’s point still stands (even if it should have said “on top of IP”). Byte streams are not ideal to deal with for protocols. As soon as you have encryption, like tls, you have discrete messages anyway (records) and the stream is largely an illusion. That said, I still like TCP as a compromise between simplicity and versatility. It’s been holding up incredibly well for a huge variety of use cases.
- jandrese 3y agoIMHO TCP made the correct call here. TCP is a transport, not an application protocol. You are supposed to run your protocol over top of it, including things like blocking, substreams, etc... It's easy to add features to a simple transport, it's not so easy to work around features when you don't want them, plus they end up complicating the stack and become a possible source of bugs. Even "simple" TCP proved to be quite a challenge to implement back in the 90s, with many many buggy stacks out on the Internet. By being just a stream of bytes you are free to do whatever you want with the protocol. Sure that may mean implementing your own form of blocking but that's better than being stuck in a protocol where you are forced to work around the message oriented features even though you are just streaming data. The checksum does hail from an earlier era where 16 bit machines were commonplace and the total amount of data sent is minimal. Also, there is some assumption that the underlying transport is going to have its own checksums. You will note that IPv6 ditches the checksum entirely.
- helsinkiandrew 3y ago> “I thought 32 bits ought to be enough for Internet addresses”. “I didn’t pay enough attention to security”. “I didn’t really appreciate the implications of the World Wide Web.” Most experiments, prototypes, and even full blown systems designed for the long term die well before 30-40 years and those extra addresses and features would ever be needed. Much more human effort is wasted prematurely 'future proofing' systems and trying to predict the future. The brilliance of TCP/IP is its simplicity - if it had got more complex perhaps it would have been replaced in its entirety. We could be using Gopher over OpenDECNet.
- hughesjj 3y agoSo many network admins still prefer to configure smaller networks with IPv4 instead of ipv6 due to simplicity, then just nat peer it out Hell even AWS didn't start seriously supporting it until the mid 2010s... And heck I don't know if they support ipv6 across the board or not
- peterfirefly 3y agoIs IPv6 on his list?
- garbagecoder 3y agoI don't know, I think he did a good job. We're just using the tech, longer than almost anything else has survived. It's always good to catalog mistakes and learn from them, but I think there's probably more to learn about what he got right to make it that durable.
- zoobab 3y ago"I didn’t pay enough attention to security" Crypto is too hard, it should be offloaded to AI.
- AnimalMuppet 3y agoAn AI smart enough to build secure crypto is also smart enough to backdoor it.
- 1letterunixname 3y agoI don't see all of the criticisms for security, but there are some fair points. 0. Gating mechanisms should live at lower and higher levels in the stack. We have 801.1x for wifi/port security and TLS between client and server. 1. DHCPv4 (and conflation with BOOTP) is messy and doesn't have a trust model. There ought to be a mechanism to validate that the DHCP server authentic, its data is valid, and it's supposed to be there. (Rogue DHCP servers can be thwarted with participation by the network gear, but requires more engineering than client simply being able to ignore bogus DHCP servers. Networks still must police and shutdown rogue endpoints nonetheless.) 2. DHCPv6 (stateful -SLAAC, stateless +SLAAC, DHCPv6-PD), SLAAC, and M and O flags make it more painful to deploy than it ought to be. 3. DNS DNSSEC/DANE hasn't really caught on (yet?). DNS needs a top-down trust model or at least make it user-friendly enough (tools) to deploy. 4. EBGP security. (Let's not talk about that.) In general: i. Optional features don't get used. ii. Complexity is a barrier to adoption and a source of security issues. iii. Over-simplicity is another source of security issues and a source of lacking functionality.
- tptacek 3y agoOr, DNS doesn't need any security model at all, because the rest of the stack is designed to assume it's not secure, which is a model that has worked for the last 30 years.
- manv1 3y ago1974 was a long time ago. 32 bits was a big number. The 8080 had 8 bit words and ran at 2Mhz. For people today, it's hard to understand how expensive everything was back then. 1K of RAM in 1974 cost about $307 dollars. Yeah, you're not going to be putting that into your router. So those extra bits have a super-high hard dollar cost. We're lucky he didn't go to 16 bits.
- zackmorris 3y agoIt's easy to poke holes in any protocol, so I usually just stick to conceptual flaws (oversights that are obvious to someone who hasn't seen the implementation). These are the biggest for me: * TCP should have been a layer above UDP, not beside it. Or UDP and the urgent/out-of-band option in TCP should have been equivalent. This would prevent the blocking of UDP in corporate networks and countries trying to stop P2P networks. * Both the network address and checksum should have been variable-length. * NAT should not exist, because variable-length addresses would have negated most of its usefulness (other than for censorship). * TCP should have been designed for networks with high latency (minutes/hours/days) to be ready for use in space. Optimistic delivery (is this the word?) should have been used instead of handshakes, but this might have required encryption from the start, to prevent sending sensitive information to the wrong recipient before it's verified. * Address negotiation should have been built-in, so that peers IDs stay connected if the network changes, regardless of their IP addresses. TCP is a connected protocol (unlike UDP which is connectionless) so this was never really considered, but connected protocols simply don't work on the mobile web without yak shaving or embedding the stream in a tunnel that handles reconnection. These issues are all severe enough that we probably shouldn't be using TCP directly. I know that they would haunt me had I designed it. It would be nice if the web provided a WebSocket that wasn't terrible, that handled everything mentioned above. Also I wonder if we scrapped all NAT workarounds, what it would take to provide something mathematically equivalent to direct connections, perhaps with homomorphic encryption, through open matchmaking servers kind of like Tor exit nodes. Edit: I forget to add why these discussions are important. There's a tendency today to drink the Kool-Aid and assume that standards are perfect, when in reality they are often highly-opinionated, which creates a heavy burden on people who think differently. Flawed standards are a form of injustice.
- blep_ 3y agoI can't decide how I feel about NAT. It's a ridiculous hack that shouldn't exist, but it also probably had the side effect of saving us from a world where ISPs charge per internet-connected device, because there's no obvious way to do that in a world where they have no visibility into your LAN routing.
- 3y ago
- cryptonector 3y ago> “I didn’t pay enough attention to security.” In the late 70s / early 80s there was a ton that the public didn't yet know about cryptography. Few, if any private citizens could have gotten security remotely right at that point. Remember, Diffie-Hellman was invented in 1976, and RSA in 1977, and PKI came some years after that, and DES was also from 1976/1977. As well the public's understanding of the need to do authenticated encryption came later, and the industry didn't really start catching on until the 90s, and AEAD cipher modes didn't really become a thing till the 00s, and didn't start to get used widely for another decade, and so on and on.
- mncharity 3y agoAround 1990, IETF had a big fight over where to go from "not enough bits" IPv4. Should we IPvNext with an incremental "mostly just add more bits"? Or attempt a "second-system syndrome" "let's punt this time on rough consensus and working code" kitchen-sink monstrosity of a... err, IPv6. Followed by years and years of "ok, IPv6 was arguably the wrong call, but here we are now, things are turning around, and it will replace IPv4 RSN". I don't recall what Vint Cerf's position was. But attributing the last few decades of IP dysfunction to 1970's choices seems... incomplete.
- commandersaki 3y ago> I don't recall what Vint Cerf's position was. But attributing the last few decades of IP dysfunction to 1970's choices seems... incomplete. Yeah, I don't really think Cerf's decision really applies to the blunder that is IPv6. There were other options available to the IPng team that were foregone for SIPP knowing there was absolutely no transition/migration plan whereas the others had at least given this area some consideration. I'm sort of expecting in year 40 of this attempt to transition to IPv6, we'll collectively realise it's a losing battle and look at future Internet architectures as salvation.
- tomcam 3y agoWhat’s amazing is not TCP/IP’s warts, but how well conceived it was. His humility overshadows the fact this has been a magnificently flexible and robust technology. It’s one of the most impressive creations of the 21st century.
- ralferoo 3y agoOne thing that might have worked better is if the concept of separate address and port had been scrapped, and there was just a 64-bit "service" instead. Naive early implementations could have been nearly the same, just using the top 32-bits to route to a host, and let the host handle all lower 32-bits (effectively a big port number). But later revisions could have split it into e.g. /48 for host and 16-bits for service selection, or take to an extreme maybe even /56 for host and 8-bits for service. The stupid thing is that the vast majority of IPv4 addresses are only used for a handful of public services, almost always HTTP and HTTPS. So even though we have extreme pressure on the 32-bits of IP address, the 16-bits of port number is mostly wasted and underused.
- ralferoo 3y agoMaybe bad form to follow up on your own post, but just thinking about this more... At the time, before IPv4 was introduced, IP addressing was 8-bit, and this was mostly a single machine per site. Moving to a 64-bit service address, early implementations could have kept their 8-bit address and had 56-bits for service ID, most of which could just be 0 and unused. Some sites might then have gradually move to /8 for site and /16 for site+machine and 48-bits for service ID, most of which would still be unused. Later on, we might see bigger networks taking a /8, /16 for network+site, and /24 for network+site+machine, etc... Up to the present-day, we could easily see how /48 for network+customer, /56 for network+customer+machine and 8-bits for service would still satisfy almost every need. Given that IPv6 /48 are being given out like candy and a /40 is trivial to get for a larger organisation, there's not really a massive difference. So, that largely covers inbound services, but there's also outgoing connections to consider. Given that IPv4/IPv6 already has to consider address+port on both sides to identify a connection, we could still have something like a /56 for network+customer+machine leaving 8-bits for discriminating between connections to a /64 remote service. It's possibly interesting to consider that this is actually kind of how the original IP stack works - a connection is just the pair of 32-bit addresses plus the protocol and then UDP and TCP bodge in some extra port numbers as extra data within their protocol-specific data. If we'd instead had an IPv5 that extended the addresses to a pair of 64-bit addresses plus protocol, that TCPv5 and UDPv5 might not have even needed port numbers. This would then have shifted the burden of supporting TCP and UDP from the kernel (because it needs to be there for IPv4 to check access control to different ports) to potentially allowing user-space protocols to be implemented because the OS can assign the socket its 64-bit address and wouldn't really need to care what happens to it after that.