5 ms·

Everyone forgets that the Internet Architecture Board took a religious view on "Internet transparency and the end-to-end principle" which was counter to the rea

by nyrikki 6mo ago

Everyone forgets that the Internet Architecture Board took a religious view on "Internet transparency and the end-to-end principle" which was counter to the realities of limited tooling and actual site maintainers needs. [0] There were many of us who, even when it was still IPng (IP Next Generation) in the mid 1990's, tried to get it working and spent significant amount of effort to do so, only to be hit with unrealistic ideological ideals that blocked our ability to deploy it, especially with the limitations of the security tools back in the day. Remember when IPng started, even large regional ISPs like xmission had finger servers, many people used telnet and actually slackware enabled telnet with no root password by default!!! I used both to get wall a coworker who was late to work because he was playing tw2000. Back then we had really bad application firewalls like Altavista and PIX was just being invented, and the large surveillance capitalism market simply didn't exist then. The IAB hampered deployment by choosing hills to die on without providing real alternatives, and didn't relent until IPv4 exhaustion became a problem, and they had lost their battle because everyone was forced into CGNAT etc...because of the IETF, not in spite of it. The IAB and IETF was living in a MIT ITS mindset when the real world was making that model hazardous and impossible. End to end transparency may be 'pretty' to some people, but it wasn't what customers needed. When they wrote the RFCs to make other services simply fail and time out if you enabled IPv6 locally, but didn't have ISP support they burned a lot of good will and everyone just started ripping out the IPv6 stack and running IPv4 only. IMHO, Like almost all tech failures, it didn't flail based on technical merits, it flailed based on ignorance of the users needs and a refusal to consider them, insisting that adopters just had to drink their particular flavor of Kool-aid or stick to IPv4, and until forced most people chose the latter. [0] https://www.rfc-editor.org/rfc/rfc5902.txt https://www.rfc-editor.org/rfc/rfc5902.txt

nyrikki 6mo ago
In fact, 30 years later, I just had to add a IPv6 block on Ubuntu’s apt mirrors this week, because the aaaa record query has higher priority and was timing out on my CI, killing build times. That behavior is due to the same politics mentioned above. A few more pragmatic decisions, or at least empathetic guidance would have dramatically changed the acceptance of ipv6.
- Dagger2 6mo ago
  AAAA records have lower priority than A records if you don't have a v6 address assigned on your system. (Link-locals don't count for this). You would only see a timeout to an AAAA record if the connection attempt to the A record already failed. Some software (looking at you, apt-get) will only print the last connection failure instead of all of them, so you don't see the failure to connect to the A record. I've seen people blame v6 for this even though they don't have v6 and it's 100% caused by their v4 breaking. Run `getent ahosts example.com` to see the order your system sorts addresses into. `wget example.com` (wget 1.x only though) is also nice, because it prints the addresses and tries to connect to them in turn, printing every error. I mean... adding v6 is the right thing to do either way, but "AAAA is higher priority than A" isn't the reason.
  - nyrikki 6mo ago
    > AAAA records have lower priority than A records if you don't have a v6 address assigned on your system. (Link-locals don't count for this). There is an expired 6man draft that explains some of the issues here. https://www.ietf.org/archive/id/draft-buraglio-6man-rfc6724-update-03.html https://www.ietf.org/archive/id/draft-buraglio-6man-rfc6724-... To be clear, I go and clean out the temporary fixes for dual stack problems, but you want some more info so here it is. $ grep 'apt.systemd.daily' /var/log/syslog.1 | grep '^2026-04-16T01:09' | wc -l 86375 $ grep 'apt.systemd.daily' /var/log/syslog.1 | grep '^2026-04-16T01:09' | head -n 1 2026-04-16T01:09:15.276295-06:00 MrBig apt.systemd.daily[45660]: /usr/bin/unattended-upgrade:2736: Warning: W:Tried to start delayed item http://us.archive.ubuntu.com/ubuntu questing-updates/main amd64 bpftool amd64 <snip> $ grep 'apt.systemd.daily' /var/log/syslog.1 | grep '^2026-04-16T01:09' | head -n 1 | wc -c 8116 IPv6 aaaa timeout was shown to be the problem, adding `Acquire::ForceIPv4 "true";` fixed the problem on several hosts. $ getent ahosts us.archive.ubuntu.com 91.189.91.81 STREAM us.archive.ubuntu.com 91.189.91.81 DGRAM 91.189.91.81 RAW 91.189.91.82 STREAM 91.189.91.82 DGRAM 91.189.91.82 RAW 91.189.91.83 STREAM 91.189.91.83 DGRAM 91.189.91.83 RAW 2620:2d:4002:1::101 STREAM 2620:2d:4002:1::101 DGRAM 2620:2d:4002:1::101 RAW 2620:2d:4002:1::102 STREAM 2620:2d:4002:1::102 DGRAM 2620:2d:4002:1::102 RAW 2620:2d:4002:1::103 STREAM 2620:2d:4002:1::103 DGRAM 2620:2d:4002:1::103 RAW There are no non `fe80::` (link local addresses) on the host. $ ip a | grep inet6 inet6 ::1/128 scope host noprefixroute inet6 fe80::786a:e338:3957:b331/64 scope link noprefixroute inet6 fe80::a10c:eae9:9a49:c94d/64 scope link noprefixroute So to be clear, I removed my temporary ipv4 only apt config, but there are a million places for this to be brittle and you see people doing so with sysctl net.ipv6.conf.* netplan, systemd-networkd, NetworkManager, etc... plus the individual client etc.... Note: https://datatracker.ietf.org/doc/html/rfc6724#section-2.1 https://datatracker.ietf.org/doc/html/rfc6724#section-2.1 And how "::/0" > "::ffff:0:0/96" And the preceding text: > If an implementation is not configurable or has not been configured, then it SHOULD operate according to the algorithms specified here in conjunction with the following default policy table: One could argue that GUA's without a non-link-local IPv6 address should just be ignored...and in a perfect world they would. But as covered int the first link in this post this is not as easy or clear as expected and people tend to error towards following rfc6724 which states just below the above refrence: > Another effect of the default policy table is to prefer communication using IPv6 addresses to communication using IPv4 addresses, if matching source addresses are available. I am not an IPv6 hater...just giving observations that when you introduce a breaking change, and add additional friction, it dramatically reduces adoption. Many companies I have been at basically just implement enough to meet Federal Government requirements and often intentionally strip it out of the backend to avoid the brittleness it caused. I am old enough to remember when I could just ask for an ASN and a portable class-c and how nice that was, in theory IPv6 should have allowed for that in some form...I am just frustrated with how it has devolved into an intractable 'wicked problem' when there was a path. The fact that people don't acknowledge the pain for users, often due to situations beyond their control is a symptom of that problem. Ubuntu should never have even requested an IPv6 aaaa in the above system, and yes it only does because of politics and RFC requirements.
convolvatron 6mo ago
you repeat several times that IAB was too ivory tower and refused to address the critical issues of the day, but don't really go into much detail. I wrote an early implementation of v6, before ratification (and even won the UNH interop prize!). and I struggle to understand exactly what blame you are placing at their feet. just that maybe they took the e2e principle too seriously and should have backed the awful bodge that was NAT?
- perennialmind 6mo ago
  CLNP had existing implementations and was fundamentally sound. On its technical merits, RFC1347 TCP and UDP with Bigger Addresses (TUBA) wins hands-down. But it took too long for the ISO to agree to a hand-off (the IETF wanted to be able _fork_ it, which seems nuts to me) and the IAB required ownership. But aside from that, I actually do think we could have baked address extensions into the existing packet format's option fields and had a gradual upgrade that relied on that awful bodge that was (and is) NAT. And had a successful transition wherein it died a well-deserved death by now. :-)
  - convolvatron 6mo ago
    I didn't have too much visibility in the CLNP world, although we did have a test network where I worked. My personal issue was that I just couldn't read the massively overwrought ISO specs. My admittedly biased viewpoint there wasn't anything really wrong with Ipv6, but the providers were quite happy with the way things were and actually kind of liked the internet-as-television model that we ended up with. I do think that the IETF didn't realize that they were losing their agency, so its very likely that TUBA would have made the difference. not for any technical reason, but that it would have been a few years earlier when people were still listening.
    - perennialmind 6mo ago
      I only read up on CLNP based on a fascination with counterfactuals. I will say there is a fair bit to IS-IS and ES-IS that's directly relevant to the original articles points on the circuits-to-bus-to-circuits physical evolution. There was no blanket assumption that the underlying layer look like Ethernet. The subnet equivalent was at a higher level and the assumptions were that there would be an actual network of links to manage. The fact that IS-IS survived as a relevant IP routing protocol says a lot on its own.

by nyrikki 6mo ago

nyrikki 6mo ago

In fact, 30 years later, I just had to add a IPv6 block on Ubuntu’s apt mirrors this week, because the aaaa record query has higher priority and was timing out on my CI, killing build times. That behavior is due to the same politics mentioned above. A few more pragmatic decisions, or at least empathetic guidance would have dramatically changed the acceptance of ipv6.

Dagger2 6mo ago
AAAA records have lower priority than A records if you don't have a v6 address assigned on your system. (Link-locals don't count for this). You would only see a timeout to an AAAA record if the connection attempt to the A record already failed. Some software (looking at you, apt-get) will only print the last connection failure instead of all of them, so you don't see the failure to connect to the A record. I've seen people blame v6 for this even though they don't have v6 and it's 100% caused by their v4 breaking. Run `getent ahosts example.com` to see the order your system sorts addresses into. `wget example.com` (wget 1.x only though) is also nice, because it prints the addresses and tries to connect to them in turn, printing every error. I mean... adding v6 is the right thing to do either way, but "AAAA is higher priority than A" isn't the reason.
- nyrikki 6mo ago
  > AAAA records have lower priority than A records if you don't have a v6 address assigned on your system. (Link-locals don't count for this). There is an expired 6man draft that explains some of the issues here. https://www.ietf.org/archive/id/draft-buraglio-6man-rfc6724-update-03.html https://www.ietf.org/archive/id/draft-buraglio-6man-rfc6724-... To be clear, I go and clean out the temporary fixes for dual stack problems, but you want some more info so here it is. $ grep 'apt.systemd.daily' /var/log/syslog.1 | grep '^2026-04-16T01:09' | wc -l 86375 $ grep 'apt.systemd.daily' /var/log/syslog.1 | grep '^2026-04-16T01:09' | head -n 1 2026-04-16T01:09:15.276295-06:00 MrBig apt.systemd.daily[45660]: /usr/bin/unattended-upgrade:2736: Warning: W:Tried to start delayed item http://us.archive.ubuntu.com/ubuntu questing-updates/main amd64 bpftool amd64 <snip> $ grep 'apt.systemd.daily' /var/log/syslog.1 | grep '^2026-04-16T01:09' | head -n 1 | wc -c 8116 IPv6 aaaa timeout was shown to be the problem, adding `Acquire::ForceIPv4 "true";` fixed the problem on several hosts. $ getent ahosts us.archive.ubuntu.com 91.189.91.81 STREAM us.archive.ubuntu.com 91.189.91.81 DGRAM 91.189.91.81 RAW 91.189.91.82 STREAM 91.189.91.82 DGRAM 91.189.91.82 RAW 91.189.91.83 STREAM 91.189.91.83 DGRAM 91.189.91.83 RAW 2620:2d:4002:1::101 STREAM 2620:2d:4002:1::101 DGRAM 2620:2d:4002:1::101 RAW 2620:2d:4002:1::102 STREAM 2620:2d:4002:1::102 DGRAM 2620:2d:4002:1::102 RAW 2620:2d:4002:1::103 STREAM 2620:2d:4002:1::103 DGRAM 2620:2d:4002:1::103 RAW There are no non `fe80::` (link local addresses) on the host. $ ip a | grep inet6 inet6 ::1/128 scope host noprefixroute inet6 fe80::786a:e338:3957:b331/64 scope link noprefixroute inet6 fe80::a10c:eae9:9a49:c94d/64 scope link noprefixroute So to be clear, I removed my temporary ipv4 only apt config, but there are a million places for this to be brittle and you see people doing so with sysctl net.ipv6.conf.* netplan, systemd-networkd, NetworkManager, etc... plus the individual client etc.... Note: https://datatracker.ietf.org/doc/html/rfc6724#section-2.1 https://datatracker.ietf.org/doc/html/rfc6724#section-2.1 And how "::/0" > "::ffff:0:0/96" And the preceding text: > If an implementation is not configurable or has not been configured, then it SHOULD operate according to the algorithms specified here in conjunction with the following default policy table: One could argue that GUA's without a non-link-local IPv6 address should just be ignored...and in a perfect world they would. But as covered int the first link in this post this is not as easy or clear as expected and people tend to error towards following rfc6724 which states just below the above refrence: > Another effect of the default policy table is to prefer communication using IPv6 addresses to communication using IPv4 addresses, if matching source addresses are available. I am not an IPv6 hater...just giving observations that when you introduce a breaking change, and add additional friction, it dramatically reduces adoption. Many companies I have been at basically just implement enough to meet Federal Government requirements and often intentionally strip it out of the backend to avoid the brittleness it caused. I am old enough to remember when I could just ask for an ASN and a portable class-c and how nice that was, in theory IPv6 should have allowed for that in some form...I am just frustrated with how it has devolved into an intractable 'wicked problem' when there was a path. The fact that people don't acknowledge the pain for users, often due to situations beyond their control is a symptom of that problem. Ubuntu should never have even requested an IPv6 aaaa in the above system, and yes it only does because of politics and RFC requirements.

convolvatron 6mo ago

you repeat several times that IAB was too ivory tower and refused to address the critical issues of the day, but don't really go into much detail. I wrote an early implementation of v6, before ratification (and even won the UNH interop prize!). and I struggle to understand exactly what blame you are placing at their feet. just that maybe they took the e2e principle too seriously and should have backed the awful bodge that was NAT?

perennialmind 6mo ago
CLNP had existing implementations and was fundamentally sound. On its technical merits, RFC1347 TCP and UDP with Bigger Addresses (TUBA) wins hands-down. But it took too long for the ISO to agree to a hand-off (the IETF wanted to be able _fork_ it, which seems nuts to me) and the IAB required ownership. But aside from that, I actually do think we could have baked address extensions into the existing packet format's option fields and had a gradual upgrade that relied on that awful bodge that was (and is) NAT. And had a successful transition wherein it died a well-deserved death by now. :-)
- convolvatron 6mo ago
  I didn't have too much visibility in the CLNP world, although we did have a test network where I worked. My personal issue was that I just couldn't read the massively overwrought ISO specs. My admittedly biased viewpoint there wasn't anything really wrong with Ipv6, but the providers were quite happy with the way things were and actually kind of liked the internet-as-television model that we ended up with. I do think that the IETF didn't realize that they were losing their agency, so its very likely that TUBA would have made the difference. not for any technical reason, but that it would have been a few years earlier when people were still listening.
  - perennialmind 6mo ago
    I only read up on CLNP based on a fascination with counterfactuals. I will say there is a fair bit to IS-IS and ES-IS that's directly relevant to the original articles points on the circuits-to-bus-to-circuits physical evolution. There was no blanket assumption that the underlying layer look like Ethernet. The subnet equivalent was at a higher level and the assumptions were that there would be an actual network of links to manage. The fact that IS-IS survived as a relevant IP routing protocol says a lot on its own.