3 ms·

Let me answer your latter questions first: In-protocol multiplexing must be pretty high up the list of necessary features Yes :-) but how about transparent m

Let me answer your latter questions first: In-protocol multiplexing must be pretty high up the list of necessary features Yes :-) but how about transparent multi-network roaming Yes :-) I'm talking about hopping between different transports. Yes :-) (I was thinking of writing a much more detailed answer to above, but it's so detailed that it would be far better as a blog post... I'll post the link here for a more inclusive discussion soon) Now going back to your first question: "what do you answer when the obvious comparison to (and lack of industry adoption of) SCTP is put forward?" Oh... where to start.. OK. First off the comparison is not all that obvious. SCTP was never built for the mobile use case (4 way handshake??). There are some improvements over TCP (reliable, out-of-order message based delivery with multihoming are all good things), but there's a fatal flaw that prevents any serious use of this protocol. It's not TCP or UDP!! And there are umpteen middle-boxes on the internet who won't let it through. That's all you need to know to explain the lack of industry adoption of SCTP. So we built our protocol on top of UDP (which we basically treat as a proxy for raw IP). Luckily for the mobile users, UDP based custom protocols are now well baked into various standard applications (DNS, VOIP etc.). A vast majority of middleboxes support it and the trend is in the right direction -- every new middle box has to support UDP passthrough/NAT by default barring exceptional circumstances. Now, how do we expect our new protocol to get adoption. The answer is, we don't. We're not just building a protocol. We're building a turnkey service for mobile app developers. Our business case doesn't involve convincing anyone of the technical superiority of the protocol. We simply sell a superior user experience with minimal developer effort. And you'll learn as we get more open in the coming few weeks (wink, wink), it's not at all a hard-sell :-) Feel free to connect with me directly or just continue the discussion here if you have more questions. I'm always happy to engage skeptical observers ;-)

bostik 12y ago
Let's start with the encouragement: please, please put the technical details into a blog post and publish it on HN. This is the kind of stuff I've always[tm] been interested about. > umpteen middle-boxes on the internet who won't let it through. That's all you need to know to explain the lack of industry adoption of SCTP. Heh, I've dealt with IPSec. You don't have to convince me on the futility of getting new protocols ("the magic number on IP header") approved for end to end delivery. :) For better or worse, UDP encapsulation is the sensible thing. I used SCTP as a fairly well known example of a protocol that has many of the modern technical requirements built in. As far as protocol features go, it is a pretty good yardstick. Now, as for "turnkey service"... I would guess you're positing yourself as the platform provider. You need to control the server endpoints to work with multi-homing; you'll have to provide and maintain the mobile app libraries, along with best practices documentation and aids for debugging; you probably want to provide a patch for wireshark (dissector); and then there's the security to think about. That's just off the top of my head. I am genuinely interested how you have solved the problems and where you have looked for inspiration. Because this is aimed at mobile developers, unreliable latency will be a major factor. That's something you do not have control over - maybe the protocol library can help a bit, and multi-homing already mandates graceful handling of concurrent retransmits. But mobile networks are not just unreliable, they are all too often outright crappy. Can one ever work around that?
- chetanahuja 12y ago
  Now, as for "turnkey service"... I would guess you're positing yourself as the platform provider. You need to control the server endpoints to work with multi-homing; you'll have to provide and maintain the mobile app libraries, along with best practices documentation and aids for debugging; you probably want to provide a patch for wireshark (dissector); and then there's the security to think about. That's just off the top of my head. That's a good list of issues. And yes, we've thought about each of those issues and have had enough success that a few pretty rigorous customers are currently using us in live apps. Because this is aimed at mobile developers, unreliable latency will be a major factor. That's something you do not have control over - maybe the protocol library can help a bit, and multi-homing already mandates graceful handling of concurrent retransmits. But mobile networks are not just unreliable, they are all too often outright crappy. Well the general issue of lossy connections, unpredictable latency and outright disconnections and reconnections (often with a different IP address) is the main impetus behind designing a brand new protocol. TCP does a woeful job here and any new approaches that limit themselves to just tweaking server side TCP are DOA when it comes to solving those issues. We'll be going more public with some of this info soon (in a couple of weeks). Btw, checked out Cricket WC odds on Smarkets. I think you guys are spot-on on the second Semi Finals ;-)
  - bostik 12y ago
    > TCP does a woeful job Not just TCP. Network operators are guilty too. There are the network side megalith buffers that are trying hard to "optimise" TCP transfers and just murder latency. Then, precisely because TCP is a stateful protocol and clients have the retransmission logic built in, the operators also have the option of applying something like RED at the earliest sign of network congestion. False positive or not. (Note: I have a friend who works on the other side of the table and runs a team of engineers responsible for installing new mobile network equipment in the field. The use of mobile networks grows faster than the operators and their contractors can add capacity. Which means the situation is unlikely to improve soon.) As you mentioned earlier, UDP at least has the distinction of being used in real-time and streaming protocols => operators and network gear are more reluctanct to mess with it. If you think you can get away with it, you could also assign multiple ports to any single session and apply at-least-once logic to the transmission - or to put it in layman's terms: blast and spam the network, in order to get even a single datagram out without excess delay. Do that often enough and you can expect the operators to crack down on it, though... As for work: thanks :-)

bostik 12y ago

Let's start with the encouragement: please, please put the technical details into a blog post and publish it on HN. This is the kind of stuff I've always[tm] been interested about. > umpteen middle-boxes on the internet who won't let it through. That's all you need to know to explain the lack of industry adoption of SCTP. Heh, I've dealt with IPSec. You don't have to convince me on the futility of getting new protocols ("the magic number on IP header") approved for end to end delivery. :) For better or worse, UDP encapsulation is the sensible thing. I used SCTP as a fairly well known example of a protocol that has many of the modern technical requirements built in. As far as protocol features go, it is a pretty good yardstick. Now, as for "turnkey service"... I would guess you're positing yourself as the platform provider. You need to control the server endpoints to work with multi-homing; you'll have to provide and maintain the mobile app libraries, along with best practices documentation and aids for debugging; you probably want to provide a patch for wireshark (dissector); and then there's the security to think about. That's just off the top of my head. I am genuinely interested how you have solved the problems and where you have looked for inspiration. Because this is aimed at mobile developers, unreliable latency will be a major factor. That's something you do not have control over - maybe the protocol library can help a bit, and multi-homing already mandates graceful handling of concurrent retransmits. But mobile networks are not just unreliable, they are all too often outright crappy. Can one ever work around that?

chetanahuja 12y ago
Now, as for "turnkey service"... I would guess you're positing yourself as the platform provider. You need to control the server endpoints to work with multi-homing; you'll have to provide and maintain the mobile app libraries, along with best practices documentation and aids for debugging; you probably want to provide a patch for wireshark (dissector); and then there's the security to think about. That's just off the top of my head. That's a good list of issues. And yes, we've thought about each of those issues and have had enough success that a few pretty rigorous customers are currently using us in live apps. Because this is aimed at mobile developers, unreliable latency will be a major factor. That's something you do not have control over - maybe the protocol library can help a bit, and multi-homing already mandates graceful handling of concurrent retransmits. But mobile networks are not just unreliable, they are all too often outright crappy. Well the general issue of lossy connections, unpredictable latency and outright disconnections and reconnections (often with a different IP address) is the main impetus behind designing a brand new protocol. TCP does a woeful job here and any new approaches that limit themselves to just tweaking server side TCP are DOA when it comes to solving those issues. We'll be going more public with some of this info soon (in a couple of weeks). Btw, checked out Cricket WC odds on Smarkets. I think you guys are spot-on on the second Semi Finals ;-)
- bostik 12y ago
  > TCP does a woeful job Not just TCP. Network operators are guilty too. There are the network side megalith buffers that are trying hard to "optimise" TCP transfers and just murder latency. Then, precisely because TCP is a stateful protocol and clients have the retransmission logic built in, the operators also have the option of applying something like RED at the earliest sign of network congestion. False positive or not. (Note: I have a friend who works on the other side of the table and runs a team of engineers responsible for installing new mobile network equipment in the field. The use of mobile networks grows faster than the operators and their contractors can add capacity. Which means the situation is unlikely to improve soon.) As you mentioned earlier, UDP at least has the distinction of being used in real-time and streaming protocols => operators and network gear are more reluctanct to mess with it. If you think you can get away with it, you could also assign multiple ports to any single session and apply at-least-once logic to the transmission - or to put it in layman's terms: blast and spam the network, in order to get even a single datagram out without excess delay. Do that often enough and you can expect the operators to crack down on it, though... As for work: thanks :-)