6 ms·

How the Hourglass Won

117 points by zdw 2y ago

__MatrixMan__ 2y ago
I'm not saying that a non-hourglass approach would've been better, but I often find myself dreaming of a world that's more hospitable to alternatives to IP (in both senses of the acronym, though I'm talking about Internet Protocol here), and its IP's position at the thin waist of the hourglass that makes it resist change. My problem with IP is that it's not partition friendly. You have to keep your stuff powered on and connected just in case two parties want to have an instantaneous conversation right now, otherwise they can't talk at all. That's not too heavy a burden if you're an ISP, but it's sort of a problem in cases where there are fewer resources available. Like maybe the bits will get there, but they're going to have to wait for the sun to come up and shine on a certain solar panel, or they're going to hitch a ride on a certain delivery truck that's heading that way anyway. If our software could tolerate reductions in service like this, if its had more modes than up-entirely-and-instantaneous and down-entirely, then there would be fewer incentives for misbehaving governments to attempt to shut it off every now and then, and it would be more likely to be helpful in the event of some kind of infrastructure-damaging disaster.
- regentbowerbird 2y ago
  Couldn't this be implemented on top of IP? It seems to me this is already kind of how mail exchanges work, they hold mail until the recipient (or next relay in the chain) downloads it.
  - ryukoposting 2y ago
    I was thinking this, as well. The brilliance of the IP hourglass lies in the fact that IP makes very few assumptions about anything above or below it, thus you can put just about anything above or below it. IP has no notion of connection, and thus no notion of the reliability thereof. That's TCP's job (or what-have-you). I understand the top comment's complaint, but I don't think IP is the problem. Another case: satellite internet. You can't assume a reliable connection to anything when "beam it into space" is an integral step in the chain of communication. Yet, it works! Whether or not a particular service is reliable is an issue with that service, not the addressing scheme. IP is an envelope with a to-address and from-address. The upper layer protocols are whatever's inside the envelope - a birthday card, a bill. The lower-layer protocols are USPS, FedEx, the Royal Mail, etc. Blaming IP for partitioning problems is like blaming the envelope for not making it to its destination.
    - __MatrixMan__ 2y ago
      But the addresses, both from and to... they're both transient names for computers. Not for people, not for data, and they must remain unchanged for the duration of the interaction. That biases everything above that layer in significant ways. You can't say "get me this data from wherever it might be found" because you have no means of talking about data except as anchored to some custodian machine, despite the fact that the same bits might be cached nearby (perhaps even on your device). You also can't gossip with nearby devices about locally relevant things like restaurant menus, open/closing hours, weather, etc... you have to instead uniquely identify a machine (which may not be local) and have it re-interpret your locality and tell you who your neighbors are. You end up depending on connectivity to a server in another state maintained by people who you don't know or necessarily trust just to determine whether the grocery store down the street has what you want. It creates unnecessary choke points, which end up failing haphazardly or being exploited by bad actors.
      - tsimionescu 2y ago
        It's true that IP itself has a significant problem with devices that roam between networks. I believe that there were some attempts to get a solution into IPv6, but they were abandoned (sadly - that could have perhaps been the killer feature that would have made adoption a much easier sell). I don't think you're right about neighbors though. IP does support broadcast to allow you to communicate with nearby machines. Of course, in real networks, this is often disabled beyond some small perimeters because of concerns for overwhelming the bandwidth.
      - Veserv 2y ago
        The things you are complaining about are things that are handled by higher level human-scale protocols. They can and are layered on top of the existing low-level hardware-scale protocols. You might think that those layers suck because they are layered on top of low-level protocols. If we just baked everything in from the start, then everything would work more cleanly. That is almost never the case. Those layers usually suck because it is just really hard to do human-level context-dependent whatever. To the extent that they suck for outside reasons, it is usually because the low-level protocols expose a abstraction that mismatches your desired functionality and is too high-level, not one that is too low-level. A lower-level abstraction would give you more flexibility to implement the high-level abstraction with fewer non-essential mismatches. Baking in these high-level human-scale abstractions down at the very heart of things is how we get complex horrible nonsense like leap seconds which we then need to add even more horrible nonsense like leap second smearing on top to attempt to poorly undo it. It is how you get Microsoft Excel rewriting numbers to dates and spell correcting gene names with no way to turn it off because they baked it in all the way at the bottom.
- timrice 2y ago
  The magic of the existing hourglass model, and the entire premise of the end-to-end principle [1], is that you can build all those features on top of the infrastructure provided by IP. IP as the skinny neck allows service providers to focus on delivering one thing very well, and enables anyone who wants to use that infrastructure to build anything they can think of on top of it. [1] https://en.wikipedia.org/wiki/End-to-end_principle https://en.wikipedia.org/wiki/End-to-end_principle
- Tor3 2y ago
  A big difference between now and earlier is the timeouts.. they weren't there before. When I worked at a customer's place back in 1997-1998 I used a CVS server in my home country as a code repository. I stayed at the customer's place for months, updating code etc. Back then with no timeouts I could simply start an up- or download at any time, and because the net was pretty slow - I used a 19200 bps connection via a mobile phone - it would take its pretty time. At day's end I simply closed my laptop, shut down the phone connection, and went back to my rented apartment. The next day I came back to the office, opened the laptop, connected the phone and re-connected to my home office, and the CVS update continued as if nothing happened. Now, of course, if anything is idle for two minutes it's basically dead, and if anything disconnects the connection itself is aborted. So now everything is indeed forced to be instantaneous. (Of course there are kind of reasons for that.. with no timeouts any malicious actor could tie up all a server's port, for example, but still..)
  - quesera 2y ago
    That might be fixable -- what kind of timeouts are you hitting, and for what application protocols? There's no concept of "idle" at the networking (IP, TCP, UDP) level. There are packet response timeouts at the kernel level, but that has always been true. Some applications (e.g. sshd) also have session idle/timeout settings, but they are just administrative hygiene decisions and usually easy to override. Definitely easy if you control the server, and the ssh client has a KeepAlive config setting if you do not. > with no timeouts any malicious actor could tie up all a server's port, for example This might be a misunderstanding. Ports do not get tied up. A server can be overwhelmed with traffic, but that requires packet volume (e.g. DoS attack) and it doesn't matter what ports are used.
    - pwg 2y ago
      > what kind of timeouts are you hitting Best guess: lack of activity timeouts in a NAT layer somewhere between the two end points that removes the NAT mapping for that given ip/port pairing.
      - quesera 2y ago
- zaphar 2y ago
  I think the issue there isn't IP but TCP. IP is purely an addressing and routing solution. It doesn't care about partitions or outages or intermittent connectivity. TCP does care about that as a protocol but you could envision and in fact there are other protocols that do handle these things over the top of IP.
- wpietri 2y ago
  Is that really a problem with IP? IP doesn't require connection state. If you want resilience in the face of loss you need to add more, like holding the data until a transmission has been acknowledged as received. TCP does that fine for modest packet loss and short outages. For longer timescales, I don't think you want some sort of store-and-forward baked into the normal network protocols, because meaning decays with time. Imagine I want to watch a movie, but the service is down for a week, so I request it every night. The network stores those 7 requests and delivers them when the movie service comes up again. Do I really want to get 7 copies of a movie I may have given up on? Or even one? That question isn't resolvable at the network layer; it requires application level knowledge. Better to let application logic handle it. I agree with your concern about government control, but I don't think the IP layer is a place to address it. I think the work around ad-hoc and mesh networks is much more interesting there. That would also drive more resilient applications, but I think that can be built on top of IP just fine.
  - barrkel 2y ago
    Consider a society that has left Earth. If light takes minutes to travel between locations, is a chatty protocol what you want? Or do you need a whole host of different application-level solutions to this problem? There's a different lens to look at it. There's an old tech talk that got me out of the IP-shaped box a lot of us are trapped in: https://www.youtube.com/watch?v=gqGEMQveoqg https://www.youtube.com/watch?v=gqGEMQveoqg I got Gemini 1.5 Pro to summarize the transcript, which I butchered a bit to fit in a text box: The Problem: The Internet is treated as a binary concept: You're either connected or not, leading to issues with mobile devices and seamless connectivity. The focus on connections (TCP/IP conversations) hinders security: Security is tied to the pipe, not the data, making it difficult to address issues like spam and content integrity. Inefficient use of resources: Broadcasting and multi-point communication are inefficient as the network is unaware of content, leading to redundant data transmission. The Proposed Solution: Data-centric architecture: Data is identified by name, not location. This enables: Trust and integrity based on data itself, not the source. Efficient multi-point communication and broadcasting. Seamless mobility and intermittent connectivity. Improved security against spam, phishing, and content manipulation. Key principles: Data integrity and trust are derived from the data itself through cryptographic signatures and verification. Immutable content with versioning and updates through supersession. Data segmentation for efficient transmission and user control. Challenges: Designing robust incentive structures for content sharing. Mitigating risks of malicious content and freeloaders. Addressing legal and liability concerns.
- pjc50 2y ago
  > IP's position at the thin waist of the hourglass that makes it resist change Believe me, this is the least change-resistant option. You don't have to change all the boxes at one end or the other or in the middle to deploy new stuff! .. unless it's IPv6. Look how well that went. Because it's not compatible, it's effectively a separate hourglass over to the side of the first one. > IP is that it's not partition friendly It's simply not at the level to care about that. If there's no link, there's no internetworking. It could perhaps be more router-flap or roaming-network friendly, at some cost. The thing you want for intermittent operation might be UUCP, or USENET, from the days when links weren't up all the time. Fidonet had similar solutions for the days of expensive telephone links.
- cruffle_duffle 2y ago
  Sounds like a more document centric network. Isn’t this something like a torrent where you are after a specific document and it is served by whatever has it and is online at the moment?
- immibis 2y ago
  Have you ever looked into Usenet or bang-path email? These are obsolete technologies because we have IP, but you can consider bringing them back for specific purposes. The obvious problem with delayed store-and-forward delivery is that you may need very big buffers.
- ooterness 2y ago
  You may be interested in Delay Tolerant Networking (DTN). https://en.wikipedia.org/wiki/Delay-tolerant_networking https://en.wikipedia.org/wiki/Delay-tolerant_networking It started as a NASA project for deep-space networks where communications are intermittent, packets take minutes or hours to arrive, and bandwidth is slim. But it's seeing increasing interest for other applications as well.
- cryptonector 2y ago
  What we needed was a protocol where listeners are addressed by _name_ rather than by IP address. And actually we have that, it's called HTTP, though it's not quite at the right layer, though QUIC is close to the right layer because UDP is so thin. And a layer below that we should have had a protocol where things are addressed by {AS number, address}. Anyways, IP is what we have. The cost of changing all of this now is astronomical, so it won't happen.
  - __MatrixMan__ 2y ago
    Most of my gripes with IP addresses is that they're too much like names. Wherever you have anything like a name you need some authority of which name maps to which thing, and wherever you have that you get either a single point of failure and subsequent fragility or a single point of control which people will use to mistreat each other. I think we can get pretty far on just key pairs and hashes.
- jgeada 2y ago
  You can build that with UDP, TCP isn't the only option on IP
- ianburrell 2y ago
  I have been thinking about building distributed store-and-forward messaging system. I think you can't do normal packets with unreliable network, need to change the model and put the model in the applications. Basically, modern email designed for disconnected operation. It would work normally on Internet but keep going when disconnected. I was also thinking it would be useful for limited bandwidth like ham radio. I also think that model would be useful for communicating with other planets. The problem is that it only works for some applications, and applications would need to change to use the new model and deal with the high latency. Which is a big hurdle when mobile and satellite makes Internet more available.
  - __MatrixMan__ 2y ago
    I think we've been having similar thoughts. The way I'm imagining it, the applications can each provide a function matching some standardized interface to the network layer. So every time two nodes are near enough to synchronize, they loop through all of the applications they have in common and run the functions for those applications. Maybe some are maintaining a web of trust, others are verifying the content in some way before they decide to store it, who knows--the network layer doesn't care, it just has some node-operator-configured resources to allocate to that app: 1GB of storage and 15 seconds of sync time every 5 minutes or somesuch. It would be up to the apps to write their sync functions such that old or irrelevant data gets forgotten, new or venerated data gets propagated, etc. Also they'd have to figure out how to make it worth everyone's while to actually dedicate those resources. You probably wouldn't end up with millions of apps like this--it would only make sense if many people around you were also using the same app--but even just two or three useful ones would probably make it worthwhile to build the lower layers. E-mail strikes me as one of the more difficult ones to do because it's sort of fundamentally point-to-point, but I could imagine a sync function which gossips about which other nodes that node typically comes across such that whenever two nodes meet in an elevator or at a stoplight or whatever there's a sort of gradient indicating which of the two is more likely to carry a message to its destination.
    - ianburrell 2y ago
      I was thinking simple messages. The new part would be routing messages when areas are disconnected. Maybe devices would volunteer to carry messages if going the right way since storage is cheap. It is also easy to make ad-hoc networks. Data synchronization would be layer on top. I think it would be hard because of latency, and would need to use CRDT or similar. My feeling is that would only work for simple databases, and complicated databases wouldn't be supported.
- RiverCrochet 2y ago
  IP's job is super simple. A) Move this packet out of the interface connected to the destination IP, or B) move this packet to a router that claims to be its next hop, or C) drop it. Because it is so simple, we can add more interfaces and/or routers at any time and expand the network very quickly. By outsourcing responsibility to anything else to other layers, IP can focus on speed. It'd be really nice for something above IP but below HTTP to gain non-hobbyist traction that separates addressing "who" you want to talk to (an identity) from IP address. So then most applications can work with what they really want to work with - identities - and not network addresses. Existing solutions in typical software mostly rely on HTTP cookies, DNS, and TLS. A lot of existing work on decentralized P2P networks do try or actually solve this problem somewhat with various cryptographic and schemes like DHTs. > You have to keep your stuff powered on and connected just in case two parties want to have an instantaneous conversation right now, otherwise they can't talk at all. I mean .. if I want to text you and your phone isn't on, you're physically not capable of receiving it. Something has to capture, save, and provide the message when the other side is available. ISP routers having that capability would be interesting, but not sure it would be a good idea.
- deleted 2y ago
  [deleted]
Timwi 2y ago
Can people please stop posting websites that pop up a “SUBSCRIBE!!” box when you scroll and make the rest of the page unreadable? It's driving me crazy. We should have a policy to post only clean websites where we can read the content unimpeded.
- Gupie 2y ago
  Just clicking the "continue reading" link worked for me, no email address needed to be entered
  - metabagel 2y ago
    It's still an interruption.
- deleted 2y ago
  [deleted]
- tveyben 2y ago
  The internet have unfortunately stopped being ‘clean’ years ago - ther’s so much ‘Dark Pattern’ going on that it’s scary - I amvannot imagine surfing without an adblocker…
- alanbernstein 2y ago
  Are there still websites that don't?
  - deleted 2y ago
    [deleted]
- coldpie 2y ago
  Install uBlock Origin and enable its Annoyances and Cookie Banner filters. This will be most effective in Firefox, but will work in other browsers as well. If you're unable to do that for some reason (e.g. iOS), check out the Kill Sticky bookmarklet, which at least lets you nuke these quickly and with consistent muscle memory. It also gets rid of those trashy sticky headers. https://www.smokingonabike.com/2024/01/20/take-back-your-web-browser-screen-space-with-kill-sticky/ https://www.smokingonabike.com/2024/01/20/take-back-your-web... NoScript also does a good job of preventing this garbage. It can be a bit of effort to use, but I find it worth the effort. If none of these work, then take that as a signal that the content is not worth viewing, close the tab, and move on with your life.
ryukoposting 2y ago
> don’t assume that today’s technology is the one true approach. Our forebears have much wisdom to give us, but this might be the most important advice of all.
aidenn0 2y ago
> One of the claims of the ATM camp was that it had inherently superior performance to IP due to the small cell size (and hence would better serve video and voice). This was arguably true in 1994 but turned out to be irrelevant as faster links became available and a combination of new router designs and Moore’s law enabled high-speed routing to flourish For those readers young enough to have never used a landline, know that this is bullshit. No VOIP (or cell) link I have ever used today achieves similar latency to a landline call in the US in the mid 90s. Ignoring latency, peak voice quality is far higher than the 8kHz uLaw that was used then, but the 95 percentile quality was much better back then as well. The flip side is that VOIP is nearly free by the standards of POTS costs in the 90s. This is true for local phone service and even more true as you compare to long-distance and international. Had the Bell heads won out, POTS probably would have come down in real-terms (also a good chance that the quality would have gone up as technology enabled it), but there's no way it would be as cheap as it is today.
newpavlov 2y ago
I think IP has two main issues: fragmentation and TCP. Possibility of fragmentation means that we can not rely on "atomicity" of IP packets. We have to limit ourselves to conservative MTU sizes if we want reasonable performance on possibly unreliable networks. And there are security considerations as well. It would've been great if IP supported a reliable way to do MTU discovery (e.g. by replying with a special "error" packet containing MTU size) or had specified a bigger guaranteed MTU size (ideally, 2^16 bytes). Finally, ossification of the TCP protocol is a well known problem. Ideally, we would have only IP packets with included source and destination ports, while TCP would be a QUIC-like protocol opaque for "dumb pipe" network hardware.
- vlovich123 2y ago
  > It would've been great if IP supported a reliable way to do MTU discovery (e.g. by replying with a special "error" packet containing MTU size) That’s literally how IP works. The problem is that some middleware boxes block the ICMP error packet that has the MTU so most people don’t see the don’t fragment IPv4 bit. IPv6 gets rid of the bit since there’s not even any fragmentation support (but the error packet remains). To combat this, DPLPMTUD was developed for QUIC where probe packets are used instead to discover the MTU at the cost of global efficiency.
  - newpavlov 2y ago
    Yes, you are right. I completely forgot about ICMP Packet Too Big messages since they are effectively not used in practice for application-level stuff. One minor difference between it and the behavior desired by me is that the error packet would be sent directly to an application port (one of the reasons why it would be nice for IP packets to contain source/destination ports), making it possible to conveniently process it. IIUC even assuming PTB messages are reliably delivered, we would need some kind of OS help (and additional APIs) to process them properly.
- cryptonector 2y ago
  MTU discovery is a solved problem (solved twice). The problem with IP is not the problem with IP, but a problem with middleware that makes deployment of new protocols above IP practically impossible. That's why QUIC rides over UDP rather than being a first class transport protocol.
tonymet 2y ago
The comments on IP winning are fine, but from the application experience, CableTV is far better than what we are converging on. The internet is converging on a few major players streaming low quality content , at a higher price, using 100x the resources of cable TV. We were streaming 500 HDTV channels for $50 / month around 2005. Now similar content would cost you $100 in internet service + $150-$200 in streaming fees. And you need much more sophisticated and numerous ICs for encode-decode. My point is , you can't judge a mature product against an evolving product . The entire content ecosystem is much worse off. We basically spent trillions of dollars to rebuild the shitty content delivery of cableTV , with worse content.
- sadtoot 2y ago
  but the non-TV part of the internet is, on its own, probably more valuable than TV ever was
  - cryptonector 2y ago
    Imagine having the Internet, and all you want it for is TV.
    - tonymet 2y ago
      That’s what the audience wants time spent is converging on social video and streaming video Full circle
  - tonymet 2y ago
    Web is practically dead . Social media a net loss. Messaging has some value . VC has some value . But we should compare apples to apples IP video consumption experience is lower quality and requires 100x the resources .
- deleted 2y ago
  [deleted]
jbandela1 2y ago
Apart from networking, the hourglass design is common throughout Electricity is an hourglass. Coal plants, solar panels, gas turbines, wind turbines, nuclear power plants all produce electricity. This is then consumed by electric cars, computers, washing machines, etc. LLVM IR is an hourglass. Many compilers and languages produce LLVM IR. This is then converted to many different instructions sets. I think if you want many-to-many non-coupled relationships, you will end up with some sort of hourglass design eventually.
- shadowgovt 2y ago
  POSIX is also an hourglass, right? It creates expectations on the part of apps of how the OS is interfaced to and expectations for how an OS is shaped for POSIX-compliant apps to interface to it. Details may vary, but that baseline makes it much easier to, for example, have emacs on Windows, Mac, and every flavor of Linux under the sun.
  - cryptonector 2y ago
    Interfaces are the narrow waists of potential hourglasses: many consumers, many producers. Any market is an hourglass too.
lexicality 2y ago
Shame, I was genuinely hoping for an essay on historical timekeeping devices when I read that title...
Terr_ 2y ago
IMO this is... just how technology works? It happens anywhere you have a one group A of things that are necessarily distinct (e.g. software tools) and another group B that is also necessarily distinct (e.g. buried wires and satellites) and people want to be able to use anything from category A with anything from category B. We could create a mechanism for every single A*B combination, but instead there's gonna be pressure to minimize the effort, and that means a smaller set of intermediate stuff in the middle. Another example might be that you have a bunch of people with different skills, and a bunch of people with different goods, and then the convergence-point involves money, rather than having a lot of distinct techniques like "how to use carpentry to acquire chickens."
shadowgovt 2y ago
HTTP(S) has also become an hourglass, in a sense. Given the choice to implement a brand-new socket-level protocol or figure out how to shoehorn a service into RESTful APIs accessible via HTTP, there are a lot of incentives to doing the latter (among others: corporate firewalls can tend to block other ports outbound by default, but frequently have to leave port 80 and port 22 traffic unmolested to not seriously break everyone's user experience).
pierrebai 2y ago
Isn't this whole description a lie, both ways? You can embed/tunnel any network transport into another. There is nothing magical about the internet and IP. It is actually being tunneled when you're using a cable modem. WiFi is a horrible hack that encapsulate IP in a very ugly way to make it onto it's wireless tech. You could have tunnel ATM over IP, I'm pretty sure of it. The depiction seems to me like a flattering extolment of IP.
aaroninsf 2y ago
From the title I was expecting this to be a charming dive into how the hourglass icon ended up as the standard cursor shown during blocking activities in visual desktops lol

117 points by zdw 2y ago

I'm not saying that a non-hourglass approach would've been better, but I often find myself dreaming of a world that's more hospitable to alternatives to IP (in both senses of the acronym, though I'm talking about Internet Protocol here), and its IP's position at the thin waist of the hourglass that makes it resist change. My problem with IP is that it's not partition friendly. You have to keep your stuff powered on and connected just in case two parties want to have an instantaneous conversation right now, otherwise they can't talk at all. That's not too heavy a burden if you're an ISP, but it's sort of a problem in cases where there are fewer resources available. Like maybe the bits will get there, but they're going to have to wait for the sun to come up and shine on a certain solar panel, or they're going to hitch a ride on a certain delivery truck that's heading that way anyway. If our software could tolerate reductions in service like this, if its had more modes than up-entirely-and-instantaneous and down-entirely, then there would be fewer incentives for misbehaving governments to attempt to shut it off every now and then, and it would be more likely to be helpful in the event of some kind of infrastructure-damaging disaster.

regentbowerbird 2y ago
Couldn't this be implemented on top of IP? It seems to me this is already kind of how mail exchanges work, they hold mail until the recipient (or next relay in the chain) downloads it.
- ryukoposting 2y ago
  I was thinking this, as well. The brilliance of the IP hourglass lies in the fact that IP makes very few assumptions about anything above or below it, thus you can put just about anything above or below it. IP has no notion of connection, and thus no notion of the reliability thereof. That's TCP's job (or what-have-you). I understand the top comment's complaint, but I don't think IP is the problem. Another case: satellite internet. You can't assume a reliable connection to anything when "beam it into space" is an integral step in the chain of communication. Yet, it works! Whether or not a particular service is reliable is an issue with that service, not the addressing scheme. IP is an envelope with a to-address and from-address. The upper layer protocols are whatever's inside the envelope - a birthday card, a bill. The lower-layer protocols are USPS, FedEx, the Royal Mail, etc. Blaming IP for partitioning problems is like blaming the envelope for not making it to its destination.
  - __MatrixMan__ 2y ago
    But the addresses, both from and to... they're both transient names for computers. Not for people, not for data, and they must remain unchanged for the duration of the interaction. That biases everything above that layer in significant ways. You can't say "get me this data from wherever it might be found" because you have no means of talking about data except as anchored to some custodian machine, despite the fact that the same bits might be cached nearby (perhaps even on your device). You also can't gossip with nearby devices about locally relevant things like restaurant menus, open/closing hours, weather, etc... you have to instead uniquely identify a machine (which may not be local) and have it re-interpret your locality and tell you who your neighbors are. You end up depending on connectivity to a server in another state maintained by people who you don't know or necessarily trust just to determine whether the grocery store down the street has what you want. It creates unnecessary choke points, which end up failing haphazardly or being exploited by bad actors.
    - tsimionescu 2y ago
      It's true that IP itself has a significant problem with devices that roam between networks. I believe that there were some attempts to get a solution into IPv6, but they were abandoned (sadly - that could have perhaps been the killer feature that would have made adoption a much easier sell). I don't think you're right about neighbors though. IP does support broadcast to allow you to communicate with nearby machines. Of course, in real networks, this is often disabled beyond some small perimeters because of concerns for overwhelming the bandwidth.
    - Veserv 2y ago
      The things you are complaining about are things that are handled by higher level human-scale protocols. They can and are layered on top of the existing low-level hardware-scale protocols. You might think that those layers suck because they are layered on top of low-level protocols. If we just baked everything in from the start, then everything would work more cleanly. That is almost never the case. Those layers usually suck because it is just really hard to do human-level context-dependent whatever. To the extent that they suck for outside reasons, it is usually because the low-level protocols expose a abstraction that mismatches your desired functionality and is too high-level, not one that is too low-level. A lower-level abstraction would give you more flexibility to implement the high-level abstraction with fewer non-essential mismatches. Baking in these high-level human-scale abstractions down at the very heart of things is how we get complex horrible nonsense like leap seconds which we then need to add even more horrible nonsense like leap second smearing on top to attempt to poorly undo it. It is how you get Microsoft Excel rewriting numbers to dates and spell correcting gene names with no way to turn it off because they baked it in all the way at the bottom.
timrice 2y ago
The magic of the existing hourglass model, and the entire premise of the end-to-end principle [1], is that you can build all those features on top of the infrastructure provided by IP. IP as the skinny neck allows service providers to focus on delivering one thing very well, and enables anyone who wants to use that infrastructure to build anything they can think of on top of it. [1] https://en.wikipedia.org/wiki/End-to-end_principle https://en.wikipedia.org/wiki/End-to-end_principle
Tor3 2y ago
A big difference between now and earlier is the timeouts.. they weren't there before. When I worked at a customer's place back in 1997-1998 I used a CVS server in my home country as a code repository. I stayed at the customer's place for months, updating code etc. Back then with no timeouts I could simply start an up- or download at any time, and because the net was pretty slow - I used a 19200 bps connection via a mobile phone - it would take its pretty time. At day's end I simply closed my laptop, shut down the phone connection, and went back to my rented apartment. The next day I came back to the office, opened the laptop, connected the phone and re-connected to my home office, and the CVS update continued as if nothing happened. Now, of course, if anything is idle for two minutes it's basically dead, and if anything disconnects the connection itself is aborted. So now everything is indeed forced to be instantaneous. (Of course there are kind of reasons for that.. with no timeouts any malicious actor could tie up all a server's port, for example, but still..)
- quesera 2y ago
  That might be fixable -- what kind of timeouts are you hitting, and for what application protocols? There's no concept of "idle" at the networking (IP, TCP, UDP) level. There are packet response timeouts at the kernel level, but that has always been true. Some applications (e.g. sshd) also have session idle/timeout settings, but they are just administrative hygiene decisions and usually easy to override. Definitely easy if you control the server, and the ssh client has a KeepAlive config setting if you do not. > with no timeouts any malicious actor could tie up all a server's port, for example This might be a misunderstanding. Ports do not get tied up. A server can be overwhelmed with traffic, but that requires packet volume (e.g. DoS attack) and it doesn't matter what ports are used.
  - pwg 2y ago
    > what kind of timeouts are you hitting Best guess: lack of activity timeouts in a NAT layer somewhere between the two end points that removes the NAT mapping for that given ip/port pairing.
    - quesera 2y ago
zaphar 2y ago
I think the issue there isn't IP but TCP. IP is purely an addressing and routing solution. It doesn't care about partitions or outages or intermittent connectivity. TCP does care about that as a protocol but you could envision and in fact there are other protocols that do handle these things over the top of IP.
wpietri 2y ago
Is that really a problem with IP? IP doesn't require connection state. If you want resilience in the face of loss you need to add more, like holding the data until a transmission has been acknowledged as received. TCP does that fine for modest packet loss and short outages. For longer timescales, I don't think you want some sort of store-and-forward baked into the normal network protocols, because meaning decays with time. Imagine I want to watch a movie, but the service is down for a week, so I request it every night. The network stores those 7 requests and delivers them when the movie service comes up again. Do I really want to get 7 copies of a movie I may have given up on? Or even one? That question isn't resolvable at the network layer; it requires application level knowledge. Better to let application logic handle it. I agree with your concern about government control, but I don't think the IP layer is a place to address it. I think the work around ad-hoc and mesh networks is much more interesting there. That would also drive more resilient applications, but I think that can be built on top of IP just fine.
- barrkel 2y ago
  Consider a society that has left Earth. If light takes minutes to travel between locations, is a chatty protocol what you want? Or do you need a whole host of different application-level solutions to this problem? There's a different lens to look at it. There's an old tech talk that got me out of the IP-shaped box a lot of us are trapped in: https://www.youtube.com/watch?v=gqGEMQveoqg https://www.youtube.com/watch?v=gqGEMQveoqg I got Gemini 1.5 Pro to summarize the transcript, which I butchered a bit to fit in a text box: The Problem: The Internet is treated as a binary concept: You're either connected or not, leading to issues with mobile devices and seamless connectivity. The focus on connections (TCP/IP conversations) hinders security: Security is tied to the pipe, not the data, making it difficult to address issues like spam and content integrity. Inefficient use of resources: Broadcasting and multi-point communication are inefficient as the network is unaware of content, leading to redundant data transmission. The Proposed Solution: Data-centric architecture: Data is identified by name, not location. This enables: Trust and integrity based on data itself, not the source. Efficient multi-point communication and broadcasting. Seamless mobility and intermittent connectivity. Improved security against spam, phishing, and content manipulation. Key principles: Data integrity and trust are derived from the data itself through cryptographic signatures and verification. Immutable content with versioning and updates through supersession. Data segmentation for efficient transmission and user control. Challenges: Designing robust incentive structures for content sharing. Mitigating risks of malicious content and freeloaders. Addressing legal and liability concerns.
pjc50 2y ago
> IP's position at the thin waist of the hourglass that makes it resist change Believe me, this is the least change-resistant option. You don't have to change all the boxes at one end or the other or in the middle to deploy new stuff! .. unless it's IPv6. Look how well that went. Because it's not compatible, it's effectively a separate hourglass over to the side of the first one. > IP is that it's not partition friendly It's simply not at the level to care about that. If there's no link, there's no internetworking. It could perhaps be more router-flap or roaming-network friendly, at some cost. The thing you want for intermittent operation might be UUCP, or USENET, from the days when links weren't up all the time. Fidonet had similar solutions for the days of expensive telephone links.
cruffle_duffle 2y ago
Sounds like a more document centric network. Isn’t this something like a torrent where you are after a specific document and it is served by whatever has it and is online at the moment?
immibis 2y ago
Have you ever looked into Usenet or bang-path email? These are obsolete technologies because we have IP, but you can consider bringing them back for specific purposes. The obvious problem with delayed store-and-forward delivery is that you may need very big buffers.
ooterness 2y ago
You may be interested in Delay Tolerant Networking (DTN). https://en.wikipedia.org/wiki/Delay-tolerant_networking https://en.wikipedia.org/wiki/Delay-tolerant_networking It started as a NASA project for deep-space networks where communications are intermittent, packets take minutes or hours to arrive, and bandwidth is slim. But it's seeing increasing interest for other applications as well.
cryptonector 2y ago
What we needed was a protocol where listeners are addressed by _name_ rather than by IP address. And actually we have that, it's called HTTP, though it's not quite at the right layer, though QUIC is close to the right layer because UDP is so thin. And a layer below that we should have had a protocol where things are addressed by {AS number, address}. Anyways, IP is what we have. The cost of changing all of this now is astronomical, so it won't happen.
- __MatrixMan__ 2y ago
  Most of my gripes with IP addresses is that they're too much like names. Wherever you have anything like a name you need some authority of which name maps to which thing, and wherever you have that you get either a single point of failure and subsequent fragility or a single point of control which people will use to mistreat each other. I think we can get pretty far on just key pairs and hashes.
jgeada 2y ago
You can build that with UDP, TCP isn't the only option on IP
ianburrell 2y ago
I have been thinking about building distributed store-and-forward messaging system. I think you can't do normal packets with unreliable network, need to change the model and put the model in the applications. Basically, modern email designed for disconnected operation. It would work normally on Internet but keep going when disconnected. I was also thinking it would be useful for limited bandwidth like ham radio. I also think that model would be useful for communicating with other planets. The problem is that it only works for some applications, and applications would need to change to use the new model and deal with the high latency. Which is a big hurdle when mobile and satellite makes Internet more available.
- __MatrixMan__ 2y ago
  I think we've been having similar thoughts. The way I'm imagining it, the applications can each provide a function matching some standardized interface to the network layer. So every time two nodes are near enough to synchronize, they loop through all of the applications they have in common and run the functions for those applications. Maybe some are maintaining a web of trust, others are verifying the content in some way before they decide to store it, who knows--the network layer doesn't care, it just has some node-operator-configured resources to allocate to that app: 1GB of storage and 15 seconds of sync time every 5 minutes or somesuch. It would be up to the apps to write their sync functions such that old or irrelevant data gets forgotten, new or venerated data gets propagated, etc. Also they'd have to figure out how to make it worth everyone's while to actually dedicate those resources. You probably wouldn't end up with millions of apps like this--it would only make sense if many people around you were also using the same app--but even just two or three useful ones would probably make it worthwhile to build the lower layers. E-mail strikes me as one of the more difficult ones to do because it's sort of fundamentally point-to-point, but I could imagine a sync function which gossips about which other nodes that node typically comes across such that whenever two nodes meet in an elevator or at a stoplight or whatever there's a sort of gradient indicating which of the two is more likely to carry a message to its destination.
  - ianburrell 2y ago
    I was thinking simple messages. The new part would be routing messages when areas are disconnected. Maybe devices would volunteer to carry messages if going the right way since storage is cheap. It is also easy to make ad-hoc networks. Data synchronization would be layer on top. I think it would be hard because of latency, and would need to use CRDT or similar. My feeling is that would only work for simple databases, and complicated databases wouldn't be supported.
RiverCrochet 2y ago
IP's job is super simple. A) Move this packet out of the interface connected to the destination IP, or B) move this packet to a router that claims to be its next hop, or C) drop it. Because it is so simple, we can add more interfaces and/or routers at any time and expand the network very quickly. By outsourcing responsibility to anything else to other layers, IP can focus on speed. It'd be really nice for something above IP but below HTTP to gain non-hobbyist traction that separates addressing "who" you want to talk to (an identity) from IP address. So then most applications can work with what they really want to work with - identities - and not network addresses. Existing solutions in typical software mostly rely on HTTP cookies, DNS, and TLS. A lot of existing work on decentralized P2P networks do try or actually solve this problem somewhat with various cryptographic and schemes like DHTs. > You have to keep your stuff powered on and connected just in case two parties want to have an instantaneous conversation right now, otherwise they can't talk at all. I mean .. if I want to text you and your phone isn't on, you're physically not capable of receiving it. Something has to capture, save, and provide the message when the other side is available. ISP routers having that capability would be interesting, but not sure it would be a good idea.
deleted 2y ago
[deleted]

Timwi 2y ago

Can people please stop posting websites that pop up a “SUBSCRIBE!!” box when you scroll and make the rest of the page unreadable? It's driving me crazy. We should have a policy to post only clean websites where we can read the content unimpeded.

Gupie 2y ago
Just clicking the "continue reading" link worked for me, no email address needed to be entered
- metabagel 2y ago
  It's still an interruption.
deleted 2y ago
[deleted]
tveyben 2y ago
The internet have unfortunately stopped being ‘clean’ years ago - ther’s so much ‘Dark Pattern’ going on that it’s scary - I amvannot imagine surfing without an adblocker…
alanbernstein 2y ago
Are there still websites that don't?
- deleted 2y ago
  [deleted]
coldpie 2y ago
Install uBlock Origin and enable its Annoyances and Cookie Banner filters. This will be most effective in Firefox, but will work in other browsers as well. If you're unable to do that for some reason (e.g. iOS), check out the Kill Sticky bookmarklet, which at least lets you nuke these quickly and with consistent muscle memory. It also gets rid of those trashy sticky headers. https://www.smokingonabike.com/2024/01/20/take-back-your-web-browser-screen-space-with-kill-sticky/ https://www.smokingonabike.com/2024/01/20/take-back-your-web... NoScript also does a good job of preventing this garbage. It can be a bit of effort to use, but I find it worth the effort. If none of these work, then take that as a signal that the content is not worth viewing, close the tab, and move on with your life.

ryukoposting 2y ago

> don’t assume that today’s technology is the one true approach. Our forebears have much wisdom to give us, but this might be the most important advice of all.

aidenn0 2y ago

> One of the claims of the ATM camp was that it had inherently superior performance to IP due to the small cell size (and hence would better serve video and voice). This was arguably true in 1994 but turned out to be irrelevant as faster links became available and a combination of new router designs and Moore’s law enabled high-speed routing to flourish For those readers young enough to have never used a landline, know that this is bullshit. No VOIP (or cell) link I have ever used today achieves similar latency to a landline call in the US in the mid 90s. Ignoring latency, peak voice quality is far higher than the 8kHz uLaw that was used then, but the 95 percentile quality was much better back then as well. The flip side is that VOIP is nearly free by the standards of POTS costs in the 90s. This is true for local phone service and even more true as you compare to long-distance and international. Had the Bell heads won out, POTS probably would have come down in real-terms (also a good chance that the quality would have gone up as technology enabled it), but there's no way it would be as cheap as it is today.

newpavlov 2y ago

I think IP has two main issues: fragmentation and TCP. Possibility of fragmentation means that we can not rely on "atomicity" of IP packets. We have to limit ourselves to conservative MTU sizes if we want reasonable performance on possibly unreliable networks. And there are security considerations as well. It would've been great if IP supported a reliable way to do MTU discovery (e.g. by replying with a special "error" packet containing MTU size) or had specified a bigger guaranteed MTU size (ideally, 2^16 bytes). Finally, ossification of the TCP protocol is a well known problem. Ideally, we would have only IP packets with included source and destination ports, while TCP would be a QUIC-like protocol opaque for "dumb pipe" network hardware.

vlovich123 2y ago
> It would've been great if IP supported a reliable way to do MTU discovery (e.g. by replying with a special "error" packet containing MTU size) That’s literally how IP works. The problem is that some middleware boxes block the ICMP error packet that has the MTU so most people don’t see the don’t fragment IPv4 bit. IPv6 gets rid of the bit since there’s not even any fragmentation support (but the error packet remains). To combat this, DPLPMTUD was developed for QUIC where probe packets are used instead to discover the MTU at the cost of global efficiency.
- newpavlov 2y ago
  Yes, you are right. I completely forgot about ICMP Packet Too Big messages since they are effectively not used in practice for application-level stuff. One minor difference between it and the behavior desired by me is that the error packet would be sent directly to an application port (one of the reasons why it would be nice for IP packets to contain source/destination ports), making it possible to conveniently process it. IIUC even assuming PTB messages are reliably delivered, we would need some kind of OS help (and additional APIs) to process them properly.
cryptonector 2y ago
MTU discovery is a solved problem (solved twice). The problem with IP is not the problem with IP, but a problem with middleware that makes deployment of new protocols above IP practically impossible. That's why QUIC rides over UDP rather than being a first class transport protocol.

tonymet 2y ago

The comments on IP winning are fine, but from the application experience, CableTV is far better than what we are converging on. The internet is converging on a few major players streaming low quality content , at a higher price, using 100x the resources of cable TV. We were streaming 500 HDTV channels for $50 / month around 2005. Now similar content would cost you $100 in internet service + $150-$200 in streaming fees. And you need much more sophisticated and numerous ICs for encode-decode. My point is , you can't judge a mature product against an evolving product . The entire content ecosystem is much worse off. We basically spent trillions of dollars to rebuild the shitty content delivery of cableTV , with worse content.

sadtoot 2y ago
but the non-TV part of the internet is, on its own, probably more valuable than TV ever was
- cryptonector 2y ago
  Imagine having the Internet, and all you want it for is TV.
  - tonymet 2y ago
    That’s what the audience wants time spent is converging on social video and streaming video Full circle
- tonymet 2y ago
  Web is practically dead . Social media a net loss. Messaging has some value . VC has some value . But we should compare apples to apples IP video consumption experience is lower quality and requires 100x the resources .
deleted 2y ago
[deleted]

jbandela1 2y ago

Apart from networking, the hourglass design is common throughout Electricity is an hourglass. Coal plants, solar panels, gas turbines, wind turbines, nuclear power plants all produce electricity. This is then consumed by electric cars, computers, washing machines, etc. LLVM IR is an hourglass. Many compilers and languages produce LLVM IR. This is then converted to many different instructions sets. I think if you want many-to-many non-coupled relationships, you will end up with some sort of hourglass design eventually.

shadowgovt 2y ago
POSIX is also an hourglass, right? It creates expectations on the part of apps of how the OS is interfaced to and expectations for how an OS is shaped for POSIX-compliant apps to interface to it. Details may vary, but that baseline makes it much easier to, for example, have emacs on Windows, Mac, and every flavor of Linux under the sun.
- cryptonector 2y ago
  Interfaces are the narrow waists of potential hourglasses: many consumers, many producers. Any market is an hourglass too.

lexicality 2y ago

Shame, I was genuinely hoping for an essay on historical timekeeping devices when I read that title...

Terr_ 2y ago

IMO this is... just how technology works? It happens anywhere you have a one group A of things that are necessarily distinct (e.g. software tools) and another group B that is also necessarily distinct (e.g. buried wires and satellites) and people want to be able to use anything from category A with anything from category B. We could create a mechanism for every single A*B combination, but instead there's gonna be pressure to minimize the effort, and that means a smaller set of intermediate stuff in the middle. Another example might be that you have a bunch of people with different skills, and a bunch of people with different goods, and then the convergence-point involves money, rather than having a lot of distinct techniques like "how to use carpentry to acquire chickens."

shadowgovt 2y ago

HTTP(S) has also become an hourglass, in a sense. Given the choice to implement a brand-new socket-level protocol or figure out how to shoehorn a service into RESTful APIs accessible via HTTP, there are a lot of incentives to doing the latter (among others: corporate firewalls can tend to block other ports outbound by default, but frequently have to leave port 80 and port 22 traffic unmolested to not seriously break everyone's user experience).

pierrebai 2y ago

Isn't this whole description a lie, both ways? You can embed/tunnel any network transport into another. There is nothing magical about the internet and IP. It is actually being tunneled when you're using a cable modem. WiFi is a horrible hack that encapsulate IP in a very ugly way to make it onto it's wireless tech. You could have tunnel ATM over IP, I'm pretty sure of it. The depiction seems to me like a flattering extolment of IP.

aaroninsf 2y ago

From the title I was expecting this to be a charming dive into how the hourglass icon ended up as the standard cursor shown during blocking activities in visual desktops lol