4 ms·

I've been thinking a lot about this in my own enterprise and I've increasingly come to the conclusion that IP itself is the wrong abstraction for how the majori

I've been thinking a lot about this in my own enterprise and I've increasingly come to the conclusion that IP itself is the wrong abstraction for how the majority of modern networked compute works. IPv6, as a (quite old itself) iteration on top of IPv4 with a bunch of byzantine processes and acronyms tacked on is solving the wrong problem. Originally IP was a way to allow discrete physical computers in different locations owned by different organizations to find each other and exchange information autonomously. These days most compute actually doesn't look like that. All my compute is in AWS. Rather than being autonomous it is controlled by a single global control plane and uniquely identified within that control plane. So when I want my services to connect to each-other within AWS why am I still dealing with these complex routing algorithms and obtuse numbering schemes? AWS knows exactly which physical hosts my processes are running on and could at a control plane level connect them directly. And I, as someone running a business, could focus on the higher level problem of 'service X is allowed to connect to service Y' rather than figuring out how to send IP packets across subnets/TGWs and where to configure which ports in NACLs and security groups to allow the connection. Similarly my ISP knows exactly where Amazon and CloudFlare's nearest front doors are so instead of 15 hops and DNS resolutions my laptop could just make a request to Service X on AWS. My ISP could drop the message in AWS' nearest front door and AWS could figure out how to drop the message on the right host however they want to. I know there's a lot of legacy cruft and also that there are benefits of the autonomous/decentralized model vs central control for the internet as a whole but given the centralized reality we're in, especially within the enterprise, I think it's worth reevaluating how we approach networking and whether the continuing focus on IP is the best use of of our time.

akira2501 4y ago
> Rather than being autonomous it is controlled by a single global control plane and uniquely identified within that control plane. By default, sure. You can easily bring your own IPs into AWS and use them instead, and I don't think it's hard to imagine the pertinent use cases and risk management this brings.
wpietri 4y ago
> my laptop could just make a request to Service X on AWS I was looking for the "just" that handwaves away the complexity and I was not disappointed. How do you imagine your laptop expressing a request in a way that it makes it through to the right machine? Doing a traceroute to amazon.com, I count 26 devices between me and it. How will those devices know which physical connection to pass the request over? Remember that some of them will be handling absurd amounts of traffic, so your scheme will need to work with custom silicon for routing as well as doing ok on the $40 Linksys home unit. What are you imagining that would be so much more efficient that it's worth the enormous switching costs? I also have questions about your notion of "centralization". Are you saying that Google, Microsoft, and other cloud vendors should just... give up and hand their business to AWS? Is that also true for anybody who does hosting, including me running a server at home? If so, I invite you to read up on the history of antitrust law, as there are good reasons to avoid a small number of people having total control over key economic sectors.
- dopylitty 4y ago
  > How do you imagine your laptop expressing a request in a way that it makes it through to the right machine? Doing a traceroute to amazon.com, I count 26 devices between me and it. How will those devices know which physical connection to pass the request over? That's my whole point. You're thinking of it from an IP perspective where there are individual devices in some chain and they all need to autonomously figure out a path from my laptop to AWS. The reality is every device between me and AWS is owned by my ISP. They know exactly which physical path ahead of time will get a message from my laptop to AWS. So why waste all the time on the IP abstraction? > I also have questions about your notion of "centralization". Are you saying that Google, Microsoft, and other cloud vendors should just... give up and hand their business to AWS? AWS is just an example. Realistically a huge amount of traffic on the internet is going to 6 places and my ISP already has direct physical connections to those places. Maintaining this complex and byzantine abstraction to figure out how to get a message from my laptop to compute in those companies' infrastructure should not be necessary. And in general the more important part is within AWS' (or Microsoft's or enterprise X's) network why waste time on IP when the network owner knows exactly which host every compute process is running on? Instead of thinking of an enterprise network as a set of autonomous hosts that need to figure out a path between each other think of it as a set of processes running on the same OS (the virtual infrastructure). Linux doesn't need to do BGP to figure out how to connect two processes so why does your network?
ec109685 4y ago
The IP addresses you see as an AWS customer aren’t the same used to route packets between hosts. That said, there’s a huge amount of commodity infrastructure built up that understands IP addresses and routing layers, so unless a new scheme offers tremendous benefits, it won’t get adoption. At least from a security perspective though ip acl’s are falling out of favor to service based identities, which is a good thing. You can see how AWS internally does networking here: https://m.youtube.com/watch?v=ii5XWpcYYnI https://m.youtube.com/watch?v=ii5XWpcYYnI
- ec109685 4y ago
  This service pretty much does what you were asking for: service name and it handles the routing under the covers: https://aws.amazon.com/blogs/aws/new-amazon-ecs-service-connect-enabling-easy-communication-between-microservices/ https://aws.amazon.com/blogs/aws/new-amazon-ecs-service-conn...
error503 4y ago
> So when I want my services to connect to each-other within AWS why am I still dealing with these complex routing algorithms and obtuse numbering schemes? > AWS knows exactly which physical hosts my processes are running on and could at a control plane level connect them directly. And I, as someone running a business, could focus on the higher level problem of 'service X is allowed to connect to service Y' rather than figuring out how to send IP packets across subnets/TGWs and where to configure which ports in NACLs and security groups to allow the connection. You shouldn't be? Doesn't AWS number your machines for you automatically and give you a unique ID you can use with DNS to reach it? And also provide a variety of 'ingress' services to abstract load balancing and security as well? I'm not a consumer of AWS services in my dayjob, but isn't this their entire raison d'etre? Otherwise you may as well just run much cheaper VMs elsewhere. > Similarly my ISP knows exactly where Amazon and CloudFlare's nearest front doors are so instead of 15 hops and DNS resolutions my laptop could just make a request to Service X on AWS. My ISP could drop the message in AWS' nearest front door and AWS could figure out how to drop the message on the right host however they want to. Uhm, aside from handwaving away how your ISP is going to give you a direct, no-hops connection to AWS, this is pretty much exactly what your ISP is doing. Hell, in some cases, your ISP has abstracted the underlying backbone hops too using something like MPLS, and this is completely invisible to you as an end user. You or your laptop don't have to think about the network part of things at all. You ask to connect to s3, your laptop looks up the service's IP address (unique ID) in DNS, sends some packets, and your ISP routes them to CloudFlare's nearest front doors. There are some good arguments to be made for a message-passing focused rather than connection focused protocol model, but that doesn't seem to be what you're talking about. What you seem to be talking about is doing away with routing altogether, and even in a relatively centralized internet, that just makes zero sense. We will continue to need the aggregation layer, we will continue to have multiple routes to a resource through multiple hops that need to be resolved into a path, and we'll continue to need a way to uniquely identify a service endpoint.

akira2501 4y ago

> Rather than being autonomous it is controlled by a single global control plane and uniquely identified within that control plane. By default, sure. You can easily bring your own IPs into AWS and use them instead, and I don't think it's hard to imagine the pertinent use cases and risk management this brings.

wpietri 4y ago

> my laptop could just make a request to Service X on AWS I was looking for the "just" that handwaves away the complexity and I was not disappointed. How do you imagine your laptop expressing a request in a way that it makes it through to the right machine? Doing a traceroute to amazon.com, I count 26 devices between me and it. How will those devices know which physical connection to pass the request over? Remember that some of them will be handling absurd amounts of traffic, so your scheme will need to work with custom silicon for routing as well as doing ok on the $40 Linksys home unit. What are you imagining that would be so much more efficient that it's worth the enormous switching costs? I also have questions about your notion of "centralization". Are you saying that Google, Microsoft, and other cloud vendors should just... give up and hand their business to AWS? Is that also true for anybody who does hosting, including me running a server at home? If so, I invite you to read up on the history of antitrust law, as there are good reasons to avoid a small number of people having total control over key economic sectors.

dopylitty 4y ago
> How do you imagine your laptop expressing a request in a way that it makes it through to the right machine? Doing a traceroute to amazon.com, I count 26 devices between me and it. How will those devices know which physical connection to pass the request over? That's my whole point. You're thinking of it from an IP perspective where there are individual devices in some chain and they all need to autonomously figure out a path from my laptop to AWS. The reality is every device between me and AWS is owned by my ISP. They know exactly which physical path ahead of time will get a message from my laptop to AWS. So why waste all the time on the IP abstraction? > I also have questions about your notion of "centralization". Are you saying that Google, Microsoft, and other cloud vendors should just... give up and hand their business to AWS? AWS is just an example. Realistically a huge amount of traffic on the internet is going to 6 places and my ISP already has direct physical connections to those places. Maintaining this complex and byzantine abstraction to figure out how to get a message from my laptop to compute in those companies' infrastructure should not be necessary. And in general the more important part is within AWS' (or Microsoft's or enterprise X's) network why waste time on IP when the network owner knows exactly which host every compute process is running on? Instead of thinking of an enterprise network as a set of autonomous hosts that need to figure out a path between each other think of it as a set of processes running on the same OS (the virtual infrastructure). Linux doesn't need to do BGP to figure out how to connect two processes so why does your network?

ec109685 4y ago

The IP addresses you see as an AWS customer aren’t the same used to route packets between hosts. That said, there’s a huge amount of commodity infrastructure built up that understands IP addresses and routing layers, so unless a new scheme offers tremendous benefits, it won’t get adoption. At least from a security perspective though ip acl’s are falling out of favor to service based identities, which is a good thing. You can see how AWS internally does networking here: https://m.youtube.com/watch?v=ii5XWpcYYnI https://m.youtube.com/watch?v=ii5XWpcYYnI

ec109685 4y ago
This service pretty much does what you were asking for: service name and it handles the routing under the covers: https://aws.amazon.com/blogs/aws/new-amazon-ecs-service-connect-enabling-easy-communication-between-microservices/ https://aws.amazon.com/blogs/aws/new-amazon-ecs-service-conn...

error503 4y ago

> So when I want my services to connect to each-other within AWS why am I still dealing with these complex routing algorithms and obtuse numbering schemes? > AWS knows exactly which physical hosts my processes are running on and could at a control plane level connect them directly. And I, as someone running a business, could focus on the higher level problem of 'service X is allowed to connect to service Y' rather than figuring out how to send IP packets across subnets/TGWs and where to configure which ports in NACLs and security groups to allow the connection. You shouldn't be? Doesn't AWS number your machines for you automatically and give you a unique ID you can use with DNS to reach it? And also provide a variety of 'ingress' services to abstract load balancing and security as well? I'm not a consumer of AWS services in my dayjob, but isn't this their entire raison d'etre? Otherwise you may as well just run much cheaper VMs elsewhere. > Similarly my ISP knows exactly where Amazon and CloudFlare's nearest front doors are so instead of 15 hops and DNS resolutions my laptop could just make a request to Service X on AWS. My ISP could drop the message in AWS' nearest front door and AWS could figure out how to drop the message on the right host however they want to. Uhm, aside from handwaving away how your ISP is going to give you a direct, no-hops connection to AWS, this is pretty much exactly what your ISP is doing. Hell, in some cases, your ISP has abstracted the underlying backbone hops too using something like MPLS, and this is completely invisible to you as an end user. You or your laptop don't have to think about the network part of things at all. You ask to connect to s3, your laptop looks up the service's IP address (unique ID) in DNS, sends some packets, and your ISP routes them to CloudFlare's nearest front doors. There are some good arguments to be made for a message-passing focused rather than connection focused protocol model, but that doesn't seem to be what you're talking about. What you seem to be talking about is doing away with routing altogether, and even in a relatively centralized internet, that just makes zero sense. We will continue to need the aggregation layer, we will continue to have multiple routes to a resource through multiple hops that need to be resolved into a path, and we'll continue to need a way to uniquely identify a service endpoint.