9 ms·
The July Galileo Outage: What happened and why
- ahubert 7y agoauthor here - if anyone has any questions or remarks, let me know!
- bobviolier 7y agoNo question, but thanks for the research and article!
- ahubert 7y agothanks!
- efm 7y agoThanks for writing this. The title doesn't mention that Galileo is a Global Navigation System (GPS).
- deleted 7y ago[deleted]
- 9nGQluzmnq3M 7y agoGPS refers specifically to the US system.
- pacaro 7y agoIndeed, the second paragraph spells out that this is a Global Navigation Satellite System (GNSS) and places it in context with GPS, GLONASS, and BeiDou
- michaelt 7y agoUnfortunately, anyone who knows what the acronym GNSS stands for probably also already knows Galileo is such a system. So changing the title to 'Galileo GNSS outage' probably wouldn't clarify matters :(
- marcosdumay 7y ago"Galileo" can stand for a lot of things.
- myself248 7y agoWhich is funny, because I've heard it both ways. For a while, as new systems were coming online, GPS was being used as a generic term, and the US system was being referred to by its original name, NAVSTAR. But only purists would do that, and everyone else kept saying GPS to refer to the US system, so the new term GNSS was invented as the generic. I try to say NAVSTAR and GNSS to avoid the ambiguous GPS, just like I say "gridiron" and "soccer" to avoid "football".
- reaperducer 7y agoBut only purists would do that Perhaps the same sort of space nerds who still call Dish Network "Echostar." /Waves hand.
- myself248 7y agoOr track the ISS as Zarya. /waves a trusty Omnipoint phone
- heydabop 7y agoThat's because Galileo is a Global Navigation Satellite System (GNSS). Saying Galileo is a GPS is like saying Target is a Walmart.
- michaelt 7y agoI gather the US GPS system has "Autonav mode" where each satellite has long-term ephemerides, allowing 60 days of reduced-precision operation if the ground segment fails. Does Galileo have no such thing, or had the errors propagated into such data by the time someone sounded the alarm?
- makomk 7y agoThe 60 day long-term ephemerides were the older sytem. Autonav on the current GPS satellites is considerably fancier - basically, the satellites carry out ranging measurements between each other and compute their own updated clock and ephemeris solutions on the satellites themselves in a distributed fashion. Specced to provide essentially full accuracy for something like six months, minus ionosphere estimates which can only be measured from the ground. As far as I can tell, Galileo has nothing like this. They have a small buffer of future ephemeris data, but the ground segment failure here was so long that it completely ran out. Their approach seems to be full accuracy or why even bother.
- toomuchtodo 7y agoAny chance we'll see WAAS reference stations with more autonomy for uploading ionsphere correction data directly to the constellation, instead of having to aggregate at master stations, which then forwards to ground uplinks?
- londons_explore 7y agoWhy would you want that? For slightly more accuracy in a severe ground station outage? Seems only marginally worth it...
- toomuchtodo 7y agoMore redundancy, less single points of failure.
- DuskStar 7y ago
- cyanoacry 7y agoNice article! I used to work in this space (was formerly in charge of GNSS hardware at my company), and it's neat to see Galileo being put through its paces. I see now that the US gets to take GPS (and the fact that the Air Force pretty much runs it single-handedly) for granted. We are seeing _some_ problems (like the GPS III block upgrade and OCX, ahah), but the org chart seems to be considerably simpler than what Galileo's got to deal with.
- Aloha 7y agoThe operational and management aspects strike me as what happens when you have too many cooks in the kitchen - if everyone is responsible for everything, no one is responsible for anything
- jtwaleson 7y agoWhen was the last time GPS had issues like this? I can imagine that during the early years this happened there as well? Or maybe not, as military quality engineering requirements were stricter.
- toyg 7y agoOr rather silence requirements were stricter. The military is not in the business of publicising failure and weaknesses.
- vel0city 7y agoThis is a bit of the interesting aspect to this failure. Official channels have overall been pretty weak on details on what all happened in this error. Most public knowledge of what happened has been gleaned from the signals coming off the satellites themselves which anyone can see. This live data combined with understandings of how the system should work is most of the public knowledge of what has been going on. I don't have a ton of knowledge into the deep technical aspects to GPS, but I imagine we would probably have some similar clues of an outage of this scale in GPS. Maybe a little less technical details of what is happening behind the scenes, but knowing a high percentage of satellites entering a no guarantee precision mode should be possible.
- myself248 7y agoExactly. And the galmon network already does that. (It should really be renamed gnssmon at this point, it's far outgrown its Galileo-watching roots.) You can run a station yourself if you have a Ublox 8- or 9-series receiver.
- vel0city 7y agoAs an amateur radio operator interested in space signals, it definitely intrigues me. Does this galmon project collect data collectively as a wider analysis of the satellite network, or does operating a node only really benefit the group operating a node? I'd love to operate a station to analyze GNSS traffic.
- ken 7y agoThe "instantaneous violations" link is dead -- I was hoping to learn what that means!
- ahubert 7y agoback up, thanks! It is on our experimental node. What is means is: the Galileo service definition regards a 'dilution of position of >6' to be problematic, but only if this happens for more than 77% of the time. The instantaneous bit is "well it is >6 NOW".
- dzhiurgis 7y agoIs 1cm accuracy part of Galileo system already operational? If so, why our phones and other devices cannot utilise it yet?
- Rebelgecko 7y agoGalileo has like a dozen different signals and I believe that most of them aren't broadcasting yet. I'm not sure which of those signals is supposed to give 1cm accuracy, but it could be one of the encrypted signals. Another consideration is that the 1cm accuracy would only be possible with fancy DGPS or RTK setups. 1cm accuracy is doable with GPS today, but unless you're a farmer or land surveyor you probably don't own any hardware capable of realizing that accuracy.
- mehrdadn 7y agoI suspect this is an extremely naive question, but why would one signal give a 1cm accuracy and another one not? Shouldn't the satellite just tell you its trajectory + identification info so you can calculate where you are based on that? Or perhaps by signal do you mean the physical electromagnetic properties (frequency etc.) are unsuitable for high accuracy, rather than the actual information content it's supposed to carry?
- dzhiurgis 7y agoAFAIK each satellite runs two atomic clocks one being much more accurate. Completely different from GPS and way more advanced. Unsure how it works singal-wise. I assume not only you need different receiver, but also its hardware has to have much higher clock rate.
- mehrdadn 7y agoInteresting, thanks!
- petschge 7y agoTo get good precision you need to remove the effect of electrons in the ionosphere. These basically change the index of refraction and therefore the propagation speed of the radio waves. The delayed signal makes your receiver think it is further away from the satellite than it actually is. One way to estimate this delay (instead of relying e.g. on the correction signal of a differential-GPS base station close to you) is to receive signals from the satellites at two different frequencies. The time delay is proportional to f^-2. Knowing the frequencies and the delay between the two signals allows you to estimate the electron content and and correct both signal to the undelayed "infinite frequency" arrival time.
- deleted 7y ago[deleted]
- madengr 7y agoMaybe instead of 24*7 work, some manager thought it was 24/7 work, and sent everyone home.
- outworlder 7y agoThe most interesting part of the whole write up is that it sheds light on how dependent of the base stations these constellations are. I used to think that, other than sending orbital corrections and the occasional fixes, the constellation would run itself. Apparently this is not the case. So a localized outage can in fact affect the entire planet. This has also implications in disaster scenarios. Not sure how applicable this is for GPS.
- walrus01 7y agoThis is one of the reasons for the ridiculously high perimeter security around Schriever AFB in Colorado, which runs the GPS network, and a number of other critical DoD satellite programs. It's guarded as thoroughly as the Pantex plant in Texas.
- nwallin 7y agoThe GPS constellation is considerably more robust to a ground station outage. If the ground station goes out, the satellites themselves are a distributed network and they use each others' data to correct themselves. It can operate in this mode for something like 60 days before it's unable to continue correcting itself. Of course, if the ground station failure was a soft failure, and instead of ceasing to upload, it began uploading incorrect data, the GPS location results would be arbitrarily bad to unusable. It's not clear to me whether the Galileo outage was because of a hard ground station failure or a soft one. But given that one of the contributing factors was that the backup system was not online, it would indicate to me that this was a hard failure that GPS would have been able to correct.
- bradknowles 7y agoThe other factor for the US GPS system is that there are actually over 24 ground stations, one for each time zone plus at least a couple of spares. A local failure in one time zone doesn’t take out the whole fleet — they can fall over to a backup ground station, and backups for the backups, etc....
- guykk22 7y agoJust wanted to give you a little bit more info. I work for the Spanish company that has developed the OSPF, GMV, and although I'm not currently involved at all in this project, from what i know, the following statement: "around 5% of the Galileo capacity is lost to software problems likely in the Orbit Synchronization Processing Facility (OSPF), run by GMV." is not fully true. The OSPF receives data from sources (From my limited knowledge, time stations in order to generate the ephemeris), and it looks like the problem was not in the OSPF code but in one of these time stations sending the data.
- ahubert 7y agoHi - thanks for this (author here). What I mean is, currently, around 5% of the satellites on average broadcast NAPA status. I did not refer to the big outage with this statement, I mean to ongoing developments. Or perhaps you are referring to the same thing & in that case it is entirely possible that the NAPA flag gets raised because the ephemeris is being generated from stale data. Do you know? Thanks!
- zmk_ 7y agoSo that's 1 satellite on average. Even though 5% means the same it makes you think the number is much larger if you do not constantly think of the total available.
- FeepingCreature 7y agoMy current impression from reading this article is that SpaceX could create a better GNSS than Galileo in a year as a side project - not because of their cheap launches, but because of their diy development culture. I am stunned by the level of organizational inadequacy of allowing such critical parts of the system to be developed by not just a contractor but a chain of subsidiary contractors, with feedback cycles that move at the speed of corporate communication. My expectation now is that Galileo will never catch up to GPS and will eventually be forgotten.
- LeoPanthera 7y agoPerhaps a good lesson that the opposite of “move fast and break things” is “move slowly - but still break things”.
- abra0 7y agoMove slowly and be unable to fix things fast.
- blub 7y agoI think you're being downvoted because you made a pretty amateurish mistake: on HN you're supposed to say that you yourself can create a better GNSS over the week-end.
- abdulhaq 7y agoGalileo is a strategic asset in case of dispute with the US. It is not in a competition with GPS, it simply has to function as designed and continue to operate independently of US desires. The US objected to the construction of the Galileo system.
- creato 7y agoThe US did not object to the concept of Galileo, the US objected to the EU's choice of frequency, which made it impossible to jam Galileo without jamming GPS. The EU changed the design to avoid this, and I am not aware of any follow-up objections.