5 ms·

I would appreciate it if the industry would stop trying to shove everything into user space. Imagine it's 40 years in the future, and we still have to kludge ne

I would appreciate it if the industry would stop trying to shove everything into user space. Imagine it's 40 years in the future, and we still have to kludge new protocols over an 80 year old TCP/IP stack because nobody wants to add a new protocol to the standard. Every application ships with its own unique protocol stack implementation. Duplicate code abounds. Bugs are fixed haphazardly. Quality and stability varies by application. RCEs are more common because an increase in implementations increases the probability of security bugs. Compare that to finalizing a new protocol, putting it in the kernel, and letting all applications just connect() and send() and recv(). We lose the ability to constantly mutate the protocol whenever FAANG feels like it, but we also gain the ability for every application to use the new protocol just by changing a flag passed to a syscall.

xxpor 4y ago
Pushing it to the kernel means you have to upgrade your kernel to use it. Instead, you can just include a library that exposes a send() and recv() style API to the main app code, while the library just throws everything into a UDP socket. Seems better to me, IMO. The real issue is that we can't get away from UDP/TCP because of middleboxes (firewalls, load balancers, etc) and the fact that switches only know how to hash on 5 tuples of protocols they know about. If you're lucky, your router will be able to look inside a GRE header or something, but that's about it. When was the last time you used SCTP, for example? (If you've used it recently, I bet it was tunneled over UDP)
- kazen44 4y ago
  technically speaking, switches do not hash TCP information but hash based on data in the Ethernet frame. (this is mainly used for correctly balancing load across links in an lacp bundle). High end routers are able to do pretty conplex lookups into their asics, but this usually only extends to specific use cases. (as far as I am aware for instance, an juniper MX series is able to do a lookup of both vlan tags in a single asic pipeline step when using q-in-q. the same also applies to mpls stacked lable lookups on provider edge routers).
  - xxpor 4y ago
    The separation between a switch and a router is getting blurrier by the day, but any datacenter grade switch today will support L4 hashing for LACP. Even my el cheapo Mikrotik does.
  - mlyle 4y ago
    > technically speaking, switches do not hash TCP information but hash based on data in the Ethernet frame. I'm not sure the distinction you're drawing here. When my switch balances based on port numbers,-- yes, it's technically just peeking at "data within the ethernet frame". (is this IP? OK, let's add the address at this offset to the hash. Oh, it's TCP too? let's add the port numbers at this offset to the hash). L2 hashing puts everything from a single pair of hosts on the subnet on one wire, but this can suck when a lot of traffic comes in through a gateway. L3 hashing fixes the gateway problem. L4 hashing lets individual connections have entire links. And, of course, the more fine-grained the hashing, the better the balancing can be expected to be and the lower the odds that 80% of your traffic goes down 1 link in a 4 connection bundle.
- throwaway787544 4y ago
  vendors can't get away from middleboxes, because Enterprise customers want middleboxes. But if customers want a completely reinvented underlying technology, everyone is forced to either adopt it or become obsolete themselves. This whole middlebox obsession is some kind of Stockholm syndrome. Here you are telling me there's people who make shitty devices that don't respect network traffic and because of that you can't change network traffic. It's like saying there's someone breaking into houses to steal stuff, so we can't install stained glass windows because a thief might break them.
boardwaalk 4y ago
And it would be great if SCTP support was common across many platforms and supported over networks so we could have something (in ways) better than TCP or UDP. Unfortunately it's over 20 years old and people just shove it in user space and use it over UDP (WebRTC). Your sentiment is appreciated but it wouldn't be a way to get anything actually done :/.

xxpor 4y ago

Pushing it to the kernel means you have to upgrade your kernel to use it. Instead, you can just include a library that exposes a send() and recv() style API to the main app code, while the library just throws everything into a UDP socket. Seems better to me, IMO. The real issue is that we can't get away from UDP/TCP because of middleboxes (firewalls, load balancers, etc) and the fact that switches only know how to hash on 5 tuples of protocols they know about. If you're lucky, your router will be able to look inside a GRE header or something, but that's about it. When was the last time you used SCTP, for example? (If you've used it recently, I bet it was tunneled over UDP)

kazen44 4y ago
technically speaking, switches do not hash TCP information but hash based on data in the Ethernet frame. (this is mainly used for correctly balancing load across links in an lacp bundle). High end routers are able to do pretty conplex lookups into their asics, but this usually only extends to specific use cases. (as far as I am aware for instance, an juniper MX series is able to do a lookup of both vlan tags in a single asic pipeline step when using q-in-q. the same also applies to mpls stacked lable lookups on provider edge routers).
- xxpor 4y ago
  The separation between a switch and a router is getting blurrier by the day, but any datacenter grade switch today will support L4 hashing for LACP. Even my el cheapo Mikrotik does.
- mlyle 4y ago
  > technically speaking, switches do not hash TCP information but hash based on data in the Ethernet frame. I'm not sure the distinction you're drawing here. When my switch balances based on port numbers,-- yes, it's technically just peeking at "data within the ethernet frame". (is this IP? OK, let's add the address at this offset to the hash. Oh, it's TCP too? let's add the port numbers at this offset to the hash). L2 hashing puts everything from a single pair of hosts on the subnet on one wire, but this can suck when a lot of traffic comes in through a gateway. L3 hashing fixes the gateway problem. L4 hashing lets individual connections have entire links. And, of course, the more fine-grained the hashing, the better the balancing can be expected to be and the lower the odds that 80% of your traffic goes down 1 link in a 4 connection bundle.
throwaway787544 4y ago
vendors can't get away from middleboxes, because Enterprise customers want middleboxes. But if customers want a completely reinvented underlying technology, everyone is forced to either adopt it or become obsolete themselves. This whole middlebox obsession is some kind of Stockholm syndrome. Here you are telling me there's people who make shitty devices that don't respect network traffic and because of that you can't change network traffic. It's like saying there's someone breaking into houses to steal stuff, so we can't install stained glass windows because a thief might break them.

boardwaalk 4y ago

And it would be great if SCTP support was common across many platforms and supported over networks so we could have something (in ways) better than TCP or UDP. Unfortunately it's over 20 years old and people just shove it in user space and use it over UDP (WebRTC). Your sentiment is appreciated but it wouldn't be a way to get anything actually done :/.