8 ms·

1,300 SpaceX satellite units went offline over funding issues

46 points by mzs 4y ago

greatgib 4y ago
Can someone explain to me why Ukrainians have to pay 2500 to 4500$ in monthly fee while normal consumers of the service are normally only paying around 100$?
- killingtime74 4y ago
  Private companies will charge whatever they can. I never understood why people seek justification for a certain price. The bargaining power imbalance is plain to see. When starlink charges you $100 it's because where it does so it has competitors that charge at that level. (Pendants will challenge this). In Australia where starlink is the superior satellite service there is an inferior NBN service and various mobile services if you can get signal. In Ukraine what do the alternatives charge that have consistent service?
  - ROTMetro 4y ago
    So war profiteering then? Since Ukraine HAD alternatives pre-war, then your model only applies because they can get away with it in a war zone. If I own the only gas station with gas after a hurricane or earthquake, I should be able to raise my rates to whatever I want?
    - killingtime74 4y ago
      Is not my model it's an observation. If you don't raise prices after hurricane or earthquake you run out, which is what usually happens. This is what happened at the start of COVID with masks and hand sanitizer. It is profiteering. Read your country's local news to see if anything was done about it at the time, in mine nothing was.
    - celestialcheese 4y ago
      What network alternatives do they have in the combat zones? My understanding was that communication infra was taken offline in the first few weeks of the war.
      - ROTMetro 4y ago
        Pre-war there were alternatives. The only reason for scarcity is the war, hence it being war profiteering.
    - smileysteve 4y ago
      Many governments (especially states in the U.S) have anti price gouging laws.
      - killingtime74 4y ago
        So does mine (Australian). However during covid we didn't see much actual enforcement
    - mensetmanusman 4y ago
      Not necessarily, having security resources to counter Russian attempts at civilian bombings is expensive from an opportunity cost standpoint.
    - Animatronio 4y ago
      It's only $20 mil. If I remember correctly Raytheon was awarded a $182 mil contract to develop 2x NASAMS for Ukraine. Just ask them for a slight price cut and voila, money for SpaceX.
  - warbler73 4y ago
    Satellites cost a lot to operate and UA is using significant bandwidth for these satellites. SpaceX is not a charity.
    - thot_experiment 4y ago
      What is the operational cost of a satellite? It's all upfront cost, there is no operational cost other than running uplink stations/bandwidth on the ground. Is there really significant opportunity cost for Spacex here? I fucking doubt it, the starlink satellites serve relatively small geographic areas. I very much doubt that satellites over the Donbass are in a position where they are not making money because all the traffic that would otherwise be paid for by a bunch of people is being clobbered by priority UA mil packets. I also can't imagine that the entire UA military is particularly significant on the backbone end of things when you have large metros etc mixed in, it just doesn't line up for me.
      - celestialcheese 4y ago
        LEO sats have a lifespan (5 years IIRC), so you have to constantly refresh your constellation with new launches and sats. These launch and replacement costs will drop over time, but they aren't trivial. Don't get me wrong, it's an incredible business, but there are real operating costs to keeping the network alive.
        thot_experiment 4y ago
        As mentioned in the other comment and the parent, Ukrainian wartime demand is not relevant to that side of the business, there aren't satellites that are deployed specifically to serve Ukraine and I am informed guessing that there is no aspect of the network that is actually degraded in a customer-losing sense, and while there may be greater bandwidth costs I am unconvinced that the total cost to Spacex is particularly high. (fwiw it would be chill of the US government to pitch them some money for the actual cost imo, but i think the actual cost to spacex (real and opportunity) is likely much smaller than the numbers being bandied about here)
      - 4y ago
- MichaelZuo 4y ago
  I've heard it's a combination of 1. Usage patterns being way different from normal customers, i.e. at max capacity nearly 24/7, whereas regular customers can be oversubscribed by as much as 50 to 1. 2. Cyberattacks requiring enormous effort to mitigate or block 3. More stringent and demanding requirements for customer support, etc., because of the military application. Though on the other hand there should be economies of scale somewhere here, so 50x greater usage likely isn't a linear 50x increase in cost.
  - viraptor 4y ago
    In not sure why cyberattacks would affect the price. For ISPs this is expected to be a dumb pipe. They don't care about the contents, just the volume. (ok, they kinda do, but not at a security level, they're happy for you to be pwned) But DoS protection shouldn't cost that much - you need the capacity to mitigate whether it happens or not. The cost of actually receiving that traffic shouldn't be 100x the normal price.
  - CommieBobDole 4y ago
    >max capacity nearly 24/7 At some point during this whole thing, Musk tweeted a bandwidth usage graph that showed that all of the Starlink units combined (many thousands of them) were using a total of 7TB a day. Which is a minuscule amount of data and wouldn't be much of an issue even if it was for a single unit. I got the impression that they were turning on the super expensive max unrestricted service (which has no incremental cost over the regular service) then pretending that the price of the service times the number of units was money that they were losing every month.
    - MichaelZuo 4y ago
      That 7TB figure seems pretty low for several thousand terminals over 24 hours, do you have a link to the numbers?
      - CommieBobDole 4y ago
        https://twitter.com/elonmusk/status/1580572380535001088 https://twitter.com/elonmusk/status/1580572380535001088 Looking at the graph, it looks like it's from May, so I don't know what it looks like today. But I think most of the terminals were in place by then.
- iam-TJ 4y ago
  It is based on the higher performance of "Starlink Business" [0] that has more performant hardware, increased throughput, priority, and support and which, in the U.K., is billed at "... £400/mo for service and £2,410 for hardware." It presumably also has to be a hybrid with the roaming services such as RV and Marine which may imply having to dynamically adjust satellite and ground-link paths. With the number of terminals in operation inside UA it is also possible there is a dedicated ground station in, or close to, Ukraine which would also assure very low latencies. As others have said, the demands of military use probably require 24/7 dedicated support and monitoring with real-time threat detection and evasion from skilled engineers at SpaceX/Starlink and actively monitoring the satellites passing over Ukraine, Russia, and Belarus. [0] https://www.starlink.com/business https://www.starlink.com/business
- aaron695 4y ago
- anelson 4y ago
  I have a residential Starlink terminal in Ukraine, service costs $60/mo. I assume this higher fee is for a higher tier of service suitable for military use where reliability is mission critical.
warbler73 4y ago
The article has errors. UA pays $500 a month for the $5000/mo high bandwidth high priority mobile service tier which they need for their high bandwidth drone video streaming. SpX eats the rest of the cost. This does not mean either that "it costs $4500/mo" or that "they pay $4500/mo".
- keneda7 4y ago
  Do you happen to have a source for that? I've seen numbers similar a couple times but never had a source on it.
wilg 4y ago
Someone on Reddit suggested that these are specific units that were acquired through a third party or something weird: > Reading the article: what seems to have happened is that Ukrainian government bought a batch of 1300 Starlink units from some private company in UK, had them shipped to Ukraine and distributed among the military, then had a disagreement over actually paying for the service. As a result, those units were cut off the network. > Literally no side of the dispute is commenting on the whole ordeal, so this is about the extent of information we have. > Starlink clearly sold those units expecting to get paid. Then the terminnals got resold and shipped to Ukraine and it seems like no one is paying for them now. https://www.reddit.com/r/worldnews/comments/ym4w1q/ukraine_suffered_a_comms_outage_when_1300_spacex/iv2k9pb/?context=1 https://www.reddit.com/r/worldnews/comments/ym4w1q/ukraine_s...
- mzs 4y ago
  one side is attributable to a senior official at the DoD: >Negotiations between SpaceX and the Defense Department continue despite Musk’s claim that SpaceX withdrepw its request, according to a senior defense official. >“Negotiations are very much underway. Everyone in our building knows we’re going to pay them,” the senior Pentagon official told CNN, adding that the department is eager to have commitments in writing “because we worry he’ll change his mind.”
  - candiodari 4y ago
    Funny how he's creating a reputation of being a total loose cannon, then uses this directly to get more money "or else who knows what he'll do" ... Interesting tactic.
- deleted 4y ago
  [deleted]
abeppu 4y ago
The article ends with an implication that Musk's behavior is partly about impacting the actual conflict: > As a result, Musk’s control of the signal gives him significant sway over the battlefield at a time when he has come under heavy criticism for arguing that Ukraine should sue for peace and give up some of its territory. I wonder if really the play is less ambitious: press covering disruptions in the service, and quotes from military personnel about how critical or valuable the service is are bolstering the starlink brand more than providing uninterrupted service would.
partiallypro 4y ago
I want SpaceX to continue to help Ukraine (and I loathed Elon's public crying about it), but I think it's a bit much to ask a private company to just eat the costs and not complain, especially when I don't think SpaceX has a lot of cash on hand. Hopefully there is a remedy, and this can quickly be resolved. $400M just today was added to aid to Ukraine, you'd think SpaceX and NATO/DoD could strike a deal, especially if these have been valuable battlefield tools.
- mikeyouse 4y ago
  I have no opposition whatsoever to paying for Starlink service - but how much can service actually cost? Transit for a few terabytes of data must be on the order of dollars per day. I guess if the terminal sales price was greatly marked down and they intend to make the rest up on service charges -- even then, this is more of an "opportunity cost" than an actual cost, no?
  - realgeniushere 4y ago
    Opportunity cost is no less real than other kinds of costs. If not for the cost, their bank account would have a higher number in it.
    - mikeyouse 4y ago
      It's explicitly a less real type of cost. If you have a bank account with $100 in it, a $10 daily expense will deplete your account in 10 days. A $10 daily opportunity cost would never deplete your account.
      - indecisive_user 4y ago
        >A $10 daily opportunity cost would never deplete your account Well that's assuming you have no expenses. In your example the difference between realizing the opportunity cost or not is the difference between 10 days of runway and paying the bills in perpetuity. Bills pile up no matter what, so there's only a limited amount of opportunity cost you can afford to eat.
eagerpace 4y ago
They have something like 25k terminals in use. They’ve been acquired from various sources. It’s highly likely a temporary billing or technical issue yet the article makes it sound like it’s being done deliberately. Maybe it is? But Occam’s Razor. Also, this has proven to likely be the most important piece of tech on the battle field. The US government could march into SpaceX HQ in a moment and turn it back on if needed.
whamlastxmas 4y ago
Why is it "went offline over funding issues" instead of "stopped receiving service after failing to pay bills"? The way the headline is worded makes it sound like there was some sort of hardware malfunction and that SpaceX is having financial difficulties.
7e 4y ago
From past stories, it appears that Ukraine asked for the $500 a month service, but Starling instead gave them $4500 a month service so that they can milk the Pentagon. Musk knows a government teat when he sucks on one.

46 points by mzs 4y ago

Can someone explain to me why Ukrainians have to pay 2500 to 4500$ in monthly fee while normal consumers of the service are normally only paying around 100$?

killingtime74 4y ago
Private companies will charge whatever they can. I never understood why people seek justification for a certain price. The bargaining power imbalance is plain to see. When starlink charges you $100 it's because where it does so it has competitors that charge at that level. (Pendants will challenge this). In Australia where starlink is the superior satellite service there is an inferior NBN service and various mobile services if you can get signal. In Ukraine what do the alternatives charge that have consistent service?
- ROTMetro 4y ago
  So war profiteering then? Since Ukraine HAD alternatives pre-war, then your model only applies because they can get away with it in a war zone. If I own the only gas station with gas after a hurricane or earthquake, I should be able to raise my rates to whatever I want?
  - killingtime74 4y ago
    Is not my model it's an observation. If you don't raise prices after hurricane or earthquake you run out, which is what usually happens. This is what happened at the start of COVID with masks and hand sanitizer. It is profiteering. Read your country's local news to see if anything was done about it at the time, in mine nothing was.
  - celestialcheese 4y ago
    What network alternatives do they have in the combat zones? My understanding was that communication infra was taken offline in the first few weeks of the war.
    - ROTMetro 4y ago
      Pre-war there were alternatives. The only reason for scarcity is the war, hence it being war profiteering.
  - smileysteve 4y ago
    Many governments (especially states in the U.S) have anti price gouging laws.
    - killingtime74 4y ago
      So does mine (Australian). However during covid we didn't see much actual enforcement
  - mensetmanusman 4y ago
    Not necessarily, having security resources to counter Russian attempts at civilian bombings is expensive from an opportunity cost standpoint.
  - Animatronio 4y ago
    It's only $20 mil. If I remember correctly Raytheon was awarded a $182 mil contract to develop 2x NASAMS for Ukraine. Just ask them for a slight price cut and voila, money for SpaceX.
- warbler73 4y ago
  Satellites cost a lot to operate and UA is using significant bandwidth for these satellites. SpaceX is not a charity.
  - thot_experiment 4y ago
    What is the operational cost of a satellite? It's all upfront cost, there is no operational cost other than running uplink stations/bandwidth on the ground. Is there really significant opportunity cost for Spacex here? I fucking doubt it, the starlink satellites serve relatively small geographic areas. I very much doubt that satellites over the Donbass are in a position where they are not making money because all the traffic that would otherwise be paid for by a bunch of people is being clobbered by priority UA mil packets. I also can't imagine that the entire UA military is particularly significant on the backbone end of things when you have large metros etc mixed in, it just doesn't line up for me.
    - celestialcheese 4y ago
      LEO sats have a lifespan (5 years IIRC), so you have to constantly refresh your constellation with new launches and sats. These launch and replacement costs will drop over time, but they aren't trivial. Don't get me wrong, it's an incredible business, but there are real operating costs to keeping the network alive.
      - thot_experiment 4y ago
        As mentioned in the other comment and the parent, Ukrainian wartime demand is not relevant to that side of the business, there aren't satellites that are deployed specifically to serve Ukraine and I am informed guessing that there is no aspect of the network that is actually degraded in a customer-losing sense, and while there may be greater bandwidth costs I am unconvinced that the total cost to Spacex is particularly high. (fwiw it would be chill of the US government to pitch them some money for the actual cost imo, but i think the actual cost to spacex (real and opportunity) is likely much smaller than the numbers being bandied about here)
    - 4y ago
MichaelZuo 4y ago
I've heard it's a combination of 1. Usage patterns being way different from normal customers, i.e. at max capacity nearly 24/7, whereas regular customers can be oversubscribed by as much as 50 to 1. 2. Cyberattacks requiring enormous effort to mitigate or block 3. More stringent and demanding requirements for customer support, etc., because of the military application. Though on the other hand there should be economies of scale somewhere here, so 50x greater usage likely isn't a linear 50x increase in cost.
- viraptor 4y ago
  In not sure why cyberattacks would affect the price. For ISPs this is expected to be a dumb pipe. They don't care about the contents, just the volume. (ok, they kinda do, but not at a security level, they're happy for you to be pwned) But DoS protection shouldn't cost that much - you need the capacity to mitigate whether it happens or not. The cost of actually receiving that traffic shouldn't be 100x the normal price.
- CommieBobDole 4y ago
  >max capacity nearly 24/7 At some point during this whole thing, Musk tweeted a bandwidth usage graph that showed that all of the Starlink units combined (many thousands of them) were using a total of 7TB a day. Which is a minuscule amount of data and wouldn't be much of an issue even if it was for a single unit. I got the impression that they were turning on the super expensive max unrestricted service (which has no incremental cost over the regular service) then pretending that the price of the service times the number of units was money that they were losing every month.
  - MichaelZuo 4y ago
    That 7TB figure seems pretty low for several thousand terminals over 24 hours, do you have a link to the numbers?
    - CommieBobDole 4y ago
      https://twitter.com/elonmusk/status/1580572380535001088 https://twitter.com/elonmusk/status/1580572380535001088 Looking at the graph, it looks like it's from May, so I don't know what it looks like today. But I think most of the terminals were in place by then.
iam-TJ 4y ago
It is based on the higher performance of "Starlink Business" [0] that has more performant hardware, increased throughput, priority, and support and which, in the U.K., is billed at "... £400/mo for service and £2,410 for hardware." It presumably also has to be a hybrid with the roaming services such as RV and Marine which may imply having to dynamically adjust satellite and ground-link paths. With the number of terminals in operation inside UA it is also possible there is a dedicated ground station in, or close to, Ukraine which would also assure very low latencies. As others have said, the demands of military use probably require 24/7 dedicated support and monitoring with real-time threat detection and evasion from skilled engineers at SpaceX/Starlink and actively monitoring the satellites passing over Ukraine, Russia, and Belarus. [0] https://www.starlink.com/business https://www.starlink.com/business
aaron695 4y ago
anelson 4y ago
I have a residential Starlink terminal in Ukraine, service costs $60/mo. I assume this higher fee is for a higher tier of service suitable for military use where reliability is mission critical.

warbler73 4y ago

The article has errors. UA pays $500 a month for the $5000/mo high bandwidth high priority mobile service tier which they need for their high bandwidth drone video streaming. SpX eats the rest of the cost. This does not mean either that "it costs $4500/mo" or that "they pay $4500/mo".

keneda7 4y ago
Do you happen to have a source for that? I've seen numbers similar a couple times but never had a source on it.

wilg 4y ago

Someone on Reddit suggested that these are specific units that were acquired through a third party or something weird: > Reading the article: what seems to have happened is that Ukrainian government bought a batch of 1300 Starlink units from some private company in UK, had them shipped to Ukraine and distributed among the military, then had a disagreement over actually paying for the service. As a result, those units were cut off the network. > Literally no side of the dispute is commenting on the whole ordeal, so this is about the extent of information we have. > Starlink clearly sold those units expecting to get paid. Then the terminnals got resold and shipped to Ukraine and it seems like no one is paying for them now. https://www.reddit.com/r/worldnews/comments/ym4w1q/ukraine_suffered_a_comms_outage_when_1300_spacex/iv2k9pb/?context=1 https://www.reddit.com/r/worldnews/comments/ym4w1q/ukraine_s...

mzs 4y ago
one side is attributable to a senior official at the DoD: >Negotiations between SpaceX and the Defense Department continue despite Musk’s claim that SpaceX withdrepw its request, according to a senior defense official. >“Negotiations are very much underway. Everyone in our building knows we’re going to pay them,” the senior Pentagon official told CNN, adding that the department is eager to have commitments in writing “because we worry he’ll change his mind.”
- candiodari 4y ago
  Funny how he's creating a reputation of being a total loose cannon, then uses this directly to get more money "or else who knows what he'll do" ... Interesting tactic.
deleted 4y ago
[deleted]

abeppu 4y ago

The article ends with an implication that Musk's behavior is partly about impacting the actual conflict: > As a result, Musk’s control of the signal gives him significant sway over the battlefield at a time when he has come under heavy criticism for arguing that Ukraine should sue for peace and give up some of its territory. I wonder if really the play is less ambitious: press covering disruptions in the service, and quotes from military personnel about how critical or valuable the service is are bolstering the starlink brand more than providing uninterrupted service would.

partiallypro 4y ago

I want SpaceX to continue to help Ukraine (and I loathed Elon's public crying about it), but I think it's a bit much to ask a private company to just eat the costs and not complain, especially when I don't think SpaceX has a lot of cash on hand. Hopefully there is a remedy, and this can quickly be resolved. $400M just today was added to aid to Ukraine, you'd think SpaceX and NATO/DoD could strike a deal, especially if these have been valuable battlefield tools.

mikeyouse 4y ago
I have no opposition whatsoever to paying for Starlink service - but how much can service actually cost? Transit for a few terabytes of data must be on the order of dollars per day. I guess if the terminal sales price was greatly marked down and they intend to make the rest up on service charges -- even then, this is more of an "opportunity cost" than an actual cost, no?
- realgeniushere 4y ago
  Opportunity cost is no less real than other kinds of costs. If not for the cost, their bank account would have a higher number in it.
  - mikeyouse 4y ago
    It's explicitly a less real type of cost. If you have a bank account with $100 in it, a $10 daily expense will deplete your account in 10 days. A $10 daily opportunity cost would never deplete your account.
    - indecisive_user 4y ago
      >A $10 daily opportunity cost would never deplete your account Well that's assuming you have no expenses. In your example the difference between realizing the opportunity cost or not is the difference between 10 days of runway and paying the bills in perpetuity. Bills pile up no matter what, so there's only a limited amount of opportunity cost you can afford to eat.

eagerpace 4y ago

They have something like 25k terminals in use. They’ve been acquired from various sources. It’s highly likely a temporary billing or technical issue yet the article makes it sound like it’s being done deliberately. Maybe it is? But Occam’s Razor. Also, this has proven to likely be the most important piece of tech on the battle field. The US government could march into SpaceX HQ in a moment and turn it back on if needed.

whamlastxmas 4y ago

Why is it "went offline over funding issues" instead of "stopped receiving service after failing to pay bills"? The way the headline is worded makes it sound like there was some sort of hardware malfunction and that SpaceX is having financial difficulties.

7e 4y ago

From past stories, it appears that Ukraine asked for the $500 a month service, but Starling instead gave them $4500 a month service so that they can milk the Pentagon. Musk knows a government teat when he sucks on one.