4 ms·

We don't use the kernel TCP implementation because we decided to rethink the entire networking stack (from DNS to TCP/HTTP/TLS) as it applies to native mobile a

We don't use the kernel TCP implementation because we decided to rethink the entire networking stack (from DNS to TCP/HTTP/TLS) as it applies to native mobile apps. So we wrote this whole stack but using UDP instead of TCP, and of course implemented entirely in userspace both on client side and server side Today's mobile app ecosystem has the amazing advantage of rapidly updating binary apps running on a billion devices. This allows us to ship new client side code (with app updates) to millions of devices every few days. It's the first time in computing history that such rapid iteration on network protocols has been possible and the only way to take advantage is to ditch the kernel and ship your code with the app. EDIT: I wrote a thing about it in much more detail than the space here will afford : http://www.infoworld.com/article/3016733/application-development/in-search-of-a-cure-for-slow-mobile-downloads.html http://www.infoworld.com/article/3016733/application-develop...

khc 10y ago
Not quite relevant to the OP article, as the way you described would still need to traverse the kernel and not able to realize the performance gains that the article described.
- chetanahuja 10y ago
  It's not just at locking and context switches. There's a deeper principle at play here (referenced from Julia's original post.) "It is, he says, simply the end-to-end principle in action yet again. This principle, which says that all of the intelligence in the network belongs at the ends of the connections, doesn't stop at the kernel. It should continue, pushing as much work as possible out of the core kernel and toward the actual applications." You can take this principle further. The two ends can do much smarter things when they know a lot about each other. What is the app use case (one static file, lots of tiny files, short API call, long running connections with intermittent message exchange etc.)? What are the current networking conditions... mobile radio type/carrier/location/time/subnet etc.? Can the sending side do a quick db lookup to make intelligent decisions about window-sizes/retransmits etc.). What if you had billions of data points on network conditions from around the world from every conceivable network, what can you learn from that? An app session might make 10 concurrent requests but it might want to give higher priority (more bandwidth) to the first one. Or even, a strict priority queueing might be desired. Going further, you might want to cancel the requests that are queued up on the server side without losing the built up state of the connection. And this is just getting started. There's a lot more. Smarter server discovery, easier key-exchange for encryption, independence from IP address as the endpoint and resultant resilience against network discontinuities. Once you take control of both ends and narrow down the scope to "solve native mobile apps", the solution space becomes much broader. The article I posted in GF comment contains a lot more details. Some blogposts on our site contain even more. Now, I presented the above as "what ifs". The reality is that we're already doing most of those things for our customers. The remaining are in the pipeline Kernel TCP (or TCP in general) is a limited, one-size-fits-all approach codified (dare I say, ossified) into RFC's a long time ago. Formally there's a process to approve changes to the protocol but those have to go through at least years if not decades long process to take effect widely (google the sagas of I10 or TFO if you're interested). The ability to iterate over the protocol tweaks on both sides (client and server) within a matter of days from conception of the idea is amazingly powerful.
patmcguire 10y ago
Questions: (Disclaimer, I know only the surface of networking) "For example, PZP is able to identify the device, rather than its IP address, as the endpoint for data packets, allowing it to accommodate the intermittent nature of mobile connections in a fault-tolerant manner": In what way is this different from NAT? Is this somehow aware of the phone switching towers (topology changes)? Or is there another advantage? How do they get that information? "The PacketZoom protocol recovers from dropped packets “gracefully,” with minimal overhead above the amount of data lost in the dropped packets." I don't know signal processing well enough to know how this works, I'm guessing you don't need to do things in order like TCP, so you can ask to resend and then stich it in? Where is the beef? Is it in following devices rather than IPs? What does the interface look like?
- signa11 10y ago
  > ... I'm guessing you don't need to do things in order like TCP ... kind of tangentially relevant: the problem with tcp is that it conflates congestion loss with error loss. this causes backoff, ergo poor performance under conditions such as lossy links, handovers etc. etc. udp doesn't really suffer from any of this because, well you do everything yourself. another advantage with udp, imho, is that server side can be scaled quite easily :) e.g. you can have multiple processes terminating a udp control protocol, and demuxing connections onto session tasks which can be spread around. i am kind of _surprised_ that ip addresses for devices end up changing. this might _only_ be true if you are handing over from/to different radio access networks wifi -> lte -> wifi etc. etc. roaming within a network would (must ?) not end up changing your addresses at all. for example, in an LTE network, PGW being an ip anchor, assigns addresses to the devices, and there is only one such 'thing' per epc core (slightly convoluted when moving from one operator to another, but the basic idea is still there) from the article linked by GP, it seems that each device needs to register with a cloud based server, via which the device traffic is routed. this initial registration can be used to exchange things like IMEI or somesuch for device identification, which then remains the same as the device moves around.
  - chetanahuja 10y ago
    "this initial registration can be used to exchange things like IMEI or somesuch for device identificatio" PacketZoom stack does create our own randomly generated identifier per app install (so two different apps on the same device will have different ids). Nothing at all that would positively identify a device or a user (IMIE, UDID etc.). That's a strict no-no by app/play store policies.

khc 10y ago

Not quite relevant to the OP article, as the way you described would still need to traverse the kernel and not able to realize the performance gains that the article described.

chetanahuja 10y ago
It's not just at locking and context switches. There's a deeper principle at play here (referenced from Julia's original post.) "It is, he says, simply the end-to-end principle in action yet again. This principle, which says that all of the intelligence in the network belongs at the ends of the connections, doesn't stop at the kernel. It should continue, pushing as much work as possible out of the core kernel and toward the actual applications." You can take this principle further. The two ends can do much smarter things when they know a lot about each other. What is the app use case (one static file, lots of tiny files, short API call, long running connections with intermittent message exchange etc.)? What are the current networking conditions... mobile radio type/carrier/location/time/subnet etc.? Can the sending side do a quick db lookup to make intelligent decisions about window-sizes/retransmits etc.). What if you had billions of data points on network conditions from around the world from every conceivable network, what can you learn from that? An app session might make 10 concurrent requests but it might want to give higher priority (more bandwidth) to the first one. Or even, a strict priority queueing might be desired. Going further, you might want to cancel the requests that are queued up on the server side without losing the built up state of the connection. And this is just getting started. There's a lot more. Smarter server discovery, easier key-exchange for encryption, independence from IP address as the endpoint and resultant resilience against network discontinuities. Once you take control of both ends and narrow down the scope to "solve native mobile apps", the solution space becomes much broader. The article I posted in GF comment contains a lot more details. Some blogposts on our site contain even more. Now, I presented the above as "what ifs". The reality is that we're already doing most of those things for our customers. The remaining are in the pipeline Kernel TCP (or TCP in general) is a limited, one-size-fits-all approach codified (dare I say, ossified) into RFC's a long time ago. Formally there's a process to approve changes to the protocol but those have to go through at least years if not decades long process to take effect widely (google the sagas of I10 or TFO if you're interested). The ability to iterate over the protocol tweaks on both sides (client and server) within a matter of days from conception of the idea is amazingly powerful.

patmcguire 10y ago

Questions: (Disclaimer, I know only the surface of networking) "For example, PZP is able to identify the device, rather than its IP address, as the endpoint for data packets, allowing it to accommodate the intermittent nature of mobile connections in a fault-tolerant manner": In what way is this different from NAT? Is this somehow aware of the phone switching towers (topology changes)? Or is there another advantage? How do they get that information? "The PacketZoom protocol recovers from dropped packets “gracefully,” with minimal overhead above the amount of data lost in the dropped packets." I don't know signal processing well enough to know how this works, I'm guessing you don't need to do things in order like TCP, so you can ask to resend and then stich it in? Where is the beef? Is it in following devices rather than IPs? What does the interface look like?

signa11 10y ago
> ... I'm guessing you don't need to do things in order like TCP ... kind of tangentially relevant: the problem with tcp is that it conflates congestion loss with error loss. this causes backoff, ergo poor performance under conditions such as lossy links, handovers etc. etc. udp doesn't really suffer from any of this because, well you do everything yourself. another advantage with udp, imho, is that server side can be scaled quite easily :) e.g. you can have multiple processes terminating a udp control protocol, and demuxing connections onto session tasks which can be spread around. i am kind of _surprised_ that ip addresses for devices end up changing. this might _only_ be true if you are handing over from/to different radio access networks wifi -> lte -> wifi etc. etc. roaming within a network would (must ?) not end up changing your addresses at all. for example, in an LTE network, PGW being an ip anchor, assigns addresses to the devices, and there is only one such 'thing' per epc core (slightly convoluted when moving from one operator to another, but the basic idea is still there) from the article linked by GP, it seems that each device needs to register with a cloud based server, via which the device traffic is routed. this initial registration can be used to exchange things like IMEI or somesuch for device identification, which then remains the same as the device moves around.
- chetanahuja 10y ago
  "this initial registration can be used to exchange things like IMEI or somesuch for device identificatio" PacketZoom stack does create our own randomly generated identifier per app install (so two different apps on the same device will have different ids). Nothing at all that would positively identify a device or a user (IMIE, UDID etc.). That's a strict no-no by app/play store policies.