5 ms·

Isn't latency a major issue for connectivity with satellites in geosynchronous orbit? I'm curious what their approach is here, either technically or what custom

by nappy 9y ago

Isn't latency a major issue for connectivity with satellites in geosynchronous orbit? I'm curious what their approach is here, either technically or what customer use cases they are targeting.

gedmark 9y ago
The distance to GEO does add some latency due to the distance, that’s the main trade off. What we found in studying this problem for a long time was that 95% of internet traffic isn’t latency sensitive— CDN traffic, video streaming, audio streaming, file downloads, social media posts, etc. The bandwidth crunch is a huge problem to solve but we realized there is low hanging fruit here that we can go after by putting satellites one at a time in GEO and putting a dent in it immediately. vs LEO constellations where you have to put up hundred of satellites (at a cost of billions of dollars) just to get started.
- p1necone 9y ago
  95% of traffic != 95% of value.
  - shaklee3 9y ago
    You're right. It's higher. Close to 100% of the valuable traffic is not latency sensitive.
- paulsutter 9y ago
  How do you plan to compete with SpaceX once they start offering low latency service everywhere? All interactive traffic is latency sensitive (have built middleboxes to improve latency issues with tcp and other protocols)
  - jandrese 9y ago
    I'd take SpaceX's latency claims with a big grain of salt. They'll only be true if you're in close geographic proximity to one of their ground stations. Once you're routing packets between satellites to get to a ground station all bets are off.
    - bryananderson 9y ago
      The SpaceX ground stations are meant to be consumer owned (about the size of a pizza box).
      - dfox 9y ago
        There is a difference between the CPE (what you use to get the service) and what SpaceX uses to sink your traffic into the global internet. And in fact there is exactly zero incentive for somebody other than SpaceX to run the ground stations. On the other hand it is completely possible that SpaceX's ground stations will use HW that is similar to their CPE box and thus they can build ridiculous amount of such ground stations (ie. at every IXP or so, althought the "lightweith non-IP transport" proclamation somehow precludes doing that)
        jandrese 9y ago
        That would be an interesting model for the ground stations. Everybody else builds large and capable ground stations with big dishes. Iridium for example provides global phone and slow data coverage using only one public ground station, but also uses a completely different band to talk with the ground stations vs. the handhelds.
        planteen 9y ago
        Iridium used to have multiple gateways. I think Arizona, Italy, Middle East, and Asia. Do they only use Arizona now?
        haldora 9y ago
        According to this Motherboard article [1] from Jan 27, 2018, only the Tempe, AZ gateway handles non-military traffic, with the US DOD and Russia operating the two other (military) gateways. I can't remember if the other gateways were shutdown during Iridium's bankruptcy. [1] https://motherboard.vice.com/en_us/article/d34bmk/iridium-next-satellite-constellation-spacex-desch https://motherboard.vice.com/en_us/article/d34bmk/iridium-ne...
- jandrese 9y ago
  Audio streaming not including voice calls I assume, because those are absolutely latency sensitive. Are your terminals going to include TCP Performance Enhancing Proxies (PEPs)? The TCP three way handshake followed by the TLS handshake makes it kind of painful to browse on an unaccelerated network with a GEO hop. And this is before you get to the latency from the ground station to the website. The problem gets even worse in Web2.0 world where everything is running off of tiny queries that the web app developers assume will be back in 50ms or so. TCP PEPs can only do so much too (cutting down on the roundtrips and ramping up the window size more quickly), eventually you hit the hard limits of physics.
  - dfox 9y ago
    From my experience it is not so hard to build completely terrestrial links that have latency that is large enough such that TCP works (althought it should not), but TLS handshake will timeout. For GEO links you will get exactly this behavior by using BGAN terminal over Bluetooth DUN profile (BT adds enough latency for this to happen).
    - jandrese 9y ago
      The only time I've seen a terrestrial link with a latency so bad that TCP couldn't be established it also required you to install a poorly behaved app that reduced Window's TCP retransmit limit to 1 (from 3). TLS setups however as you note are a whole new ball of wax, especially if the link has a bit of loss to go along with the latency.
- PaulHoule 9y ago
  I am skeptical of that analysis. I have worked over ssh over internet connections with satellite-class latency and I can say it is painful. (Emacs shell mode can help) You probably think that 'the web' is a high latency application. It probably should be, and maybe it was in 2000. Since then, web developers have gotten into the habit of using AJAX indiscriminately, plus they feel pressured to add features such as customized fonts, advertising, third party tracking, etc. I am not sure if CDN is really a net positive when a web site might need to do 30 DNS lookups because it uses 30 CDNs. It just takes one of those lookups to be slow to obliterate the savings from the CDN. CDNs might help with the median, even the average load time, but I am not sure they help the 95% load time which is what causes customer pain. Add up all those round trips and the overhead of access control (maybe those patents on slotted ALOHA for satellite applications have expired by now) and you are talking upwards of 0.5 sec and it doesn't take many round trips for that 0.5 sec to turn into 5 to 10 seconds. Worst thing is that people who are developing locally or from places with fast connections to the data center will think these apps are really fast.
  - gedmark 9y ago
    Agree on the AJAX calls. Modern web apps are a mess. Thankfully mobile apps are built quite differently, so that's an improvement. And the biggest potential market out there to serve is new users that will be coming online for the first time mobile-first.
  - shaklee3 9y ago
    SSH is not a typical use case. He said 90-95% of traffic isn't latency sensitive, and yours isn't in the majority.
- deleted 9y ago
  [deleted]
- mirimir 9y ago
  That was my first question too, once I saw that you'll be using geostationary satellites. But yes, I get your argument about acceptable latency vs cost. Sometimes I use Tor via nested VPN chains, for decent anonymity, and total rtt can be hundreds of ms or more. Web browsing, chats, downloads, and even ssh sessions, are all workable. And if push to talk is acceptable, even VoIP is workable at 1-2 sec rtt.

by nappy 9y ago

Isn't latency a major issue for connectivity with satellites in geosynchronous orbit? I'm curious what their approach is here, either technically or what customer use cases they are targeting.

gedmark 9y ago

The distance to GEO does add some latency due to the distance, that’s the main trade off. What we found in studying this problem for a long time was that 95% of internet traffic isn’t latency sensitive— CDN traffic, video streaming, audio streaming, file downloads, social media posts, etc. The bandwidth crunch is a huge problem to solve but we realized there is low hanging fruit here that we can go after by putting satellites one at a time in GEO and putting a dent in it immediately. vs LEO constellations where you have to put up hundred of satellites (at a cost of billions of dollars) just to get started.

p1necone 9y ago
95% of traffic != 95% of value.
- shaklee3 9y ago
  You're right. It's higher. Close to 100% of the valuable traffic is not latency sensitive.
paulsutter 9y ago
How do you plan to compete with SpaceX once they start offering low latency service everywhere? All interactive traffic is latency sensitive (have built middleboxes to improve latency issues with tcp and other protocols)
- jandrese 9y ago
  I'd take SpaceX's latency claims with a big grain of salt. They'll only be true if you're in close geographic proximity to one of their ground stations. Once you're routing packets between satellites to get to a ground station all bets are off.
  - bryananderson 9y ago
    The SpaceX ground stations are meant to be consumer owned (about the size of a pizza box).
    - dfox 9y ago
      There is a difference between the CPE (what you use to get the service) and what SpaceX uses to sink your traffic into the global internet. And in fact there is exactly zero incentive for somebody other than SpaceX to run the ground stations. On the other hand it is completely possible that SpaceX's ground stations will use HW that is similar to their CPE box and thus they can build ridiculous amount of such ground stations (ie. at every IXP or so, althought the "lightweith non-IP transport" proclamation somehow precludes doing that)
      - jandrese 9y ago
        That would be an interesting model for the ground stations. Everybody else builds large and capable ground stations with big dishes. Iridium for example provides global phone and slow data coverage using only one public ground station, but also uses a completely different band to talk with the ground stations vs. the handhelds.
        planteen 9y ago
        Iridium used to have multiple gateways. I think Arizona, Italy, Middle East, and Asia. Do they only use Arizona now?
        haldora 9y ago
        According to this Motherboard article [1] from Jan 27, 2018, only the Tempe, AZ gateway handles non-military traffic, with the US DOD and Russia operating the two other (military) gateways. I can't remember if the other gateways were shutdown during Iridium's bankruptcy. [1] https://motherboard.vice.com/en_us/article/d34bmk/iridium-next-satellite-constellation-spacex-desch https://motherboard.vice.com/en_us/article/d34bmk/iridium-ne...
jandrese 9y ago
Audio streaming not including voice calls I assume, because those are absolutely latency sensitive. Are your terminals going to include TCP Performance Enhancing Proxies (PEPs)? The TCP three way handshake followed by the TLS handshake makes it kind of painful to browse on an unaccelerated network with a GEO hop. And this is before you get to the latency from the ground station to the website. The problem gets even worse in Web2.0 world where everything is running off of tiny queries that the web app developers assume will be back in 50ms or so. TCP PEPs can only do so much too (cutting down on the roundtrips and ramping up the window size more quickly), eventually you hit the hard limits of physics.
- dfox 9y ago
  From my experience it is not so hard to build completely terrestrial links that have latency that is large enough such that TCP works (althought it should not), but TLS handshake will timeout. For GEO links you will get exactly this behavior by using BGAN terminal over Bluetooth DUN profile (BT adds enough latency for this to happen).
  - jandrese 9y ago
    The only time I've seen a terrestrial link with a latency so bad that TCP couldn't be established it also required you to install a poorly behaved app that reduced Window's TCP retransmit limit to 1 (from 3). TLS setups however as you note are a whole new ball of wax, especially if the link has a bit of loss to go along with the latency.
PaulHoule 9y ago
I am skeptical of that analysis. I have worked over ssh over internet connections with satellite-class latency and I can say it is painful. (Emacs shell mode can help) You probably think that 'the web' is a high latency application. It probably should be, and maybe it was in 2000. Since then, web developers have gotten into the habit of using AJAX indiscriminately, plus they feel pressured to add features such as customized fonts, advertising, third party tracking, etc. I am not sure if CDN is really a net positive when a web site might need to do 30 DNS lookups because it uses 30 CDNs. It just takes one of those lookups to be slow to obliterate the savings from the CDN. CDNs might help with the median, even the average load time, but I am not sure they help the 95% load time which is what causes customer pain. Add up all those round trips and the overhead of access control (maybe those patents on slotted ALOHA for satellite applications have expired by now) and you are talking upwards of 0.5 sec and it doesn't take many round trips for that 0.5 sec to turn into 5 to 10 seconds. Worst thing is that people who are developing locally or from places with fast connections to the data center will think these apps are really fast.
- gedmark 9y ago
  Agree on the AJAX calls. Modern web apps are a mess. Thankfully mobile apps are built quite differently, so that's an improvement. And the biggest potential market out there to serve is new users that will be coming online for the first time mobile-first.
- shaklee3 9y ago
  SSH is not a typical use case. He said 90-95% of traffic isn't latency sensitive, and yours isn't in the majority.
deleted 9y ago
[deleted]
mirimir 9y ago
That was my first question too, once I saw that you'll be using geostationary satellites. But yes, I get your argument about acceptable latency vs cost. Sometimes I use Tor via nested VPN chains, for decent anonymity, and total rtt can be hundreds of ms or more. Web browsing, chats, downloads, and even ssh sessions, are all workable. And if push to talk is acceptable, even VoIP is workable at 1-2 sec rtt.