9 ms·

Tell HN: The 10-bit timers are about to overflow on September 17th

400 points by modinfo 4y ago

Due to the overflow of the 10-bit counter, some devices will "go back in time" by 1024 weeks (almost 20 years). This will occur on the night of September 17-18. The problem will affect - and this is now a sure thing - Microsemi's (AKA Symmetricom) SyncServer (100, 200 and 300 series), TimeProvider 1000, parts of TimeProvider 5000 and Timesource TS3x00 (and a few others), which are popular in industrial networks. Loss of historical data and event logs, logging and security problems, loss of process visualization - these are some of the surprises that can happen when, having received the wrong date, other devices also decide to "time travel." What to do? If your network is running one of the aforementioned devices, it's best to disconnect it in advance. Unfortunately, most of them are no longer supported by the manufacturer and no patches are expected. So it looks like they will become quite useless after September 17. This leaves very little time, therefore, to replace them with new solutions. If you are not sure whether the problem also affects your device, you can unplug it on September seventeenth, and if it shows the correct date the next day, plug it in again. Of course, such a maneuver is possible only in those networks that can operate without time synchronization according to GPS for several/some hours.

fanf2 4y ago
This is not the GPS week counter rollover https://en.m.wikipedia.org/wiki/GPS_week_number_rollover https://en.m.wikipedia.org/wiki/GPS_week_number_rollover But instead it is a device-specific rollover. It’s relatively common for GPS receivers to handle the limited week counter by having a baked-in epoch that is more recent than the latest rollover, which gives them a nearly 20 year lifetime after that epoch instead of failing at the GPS rollover.
- 8organicbits 4y ago
  If I'm following, the thought is: It's currently Sept 17th 2002, our counter will roll over in 2019 (17 years). But since we know 1999-2001 has already past, we can hard code logic to add 20 years to any counter that's reporting that range. That updated logic rolls over on Sept 17th 2022, which buys us three extra years. This works for devices that will EOL within 20 years. I think devices that need a longer life should keep track of the last seen timestamp, then they can detect a rollover. The device would keep a local rollover counter and just add 20 years per count. This should stay correct until that counter rolls over, or if the device is offline for >20 years.
  - deleted 4y ago
    [deleted]
  - bathtub365 4y ago
    The epoch is the date & time that the minimum value of that field refers to. For Unix derived or inspired machines it’s typically the Unix epoch of 00:00:00 on January 1, 1970. This means that a timestamp of “0” refers to that date and time, and increases at whatever granularity a timestamp value maps to. If you bake the epoch into the device at the factory as being the date of manufacture and your timestamp value has 20 years worth of granularity then you get a 20 year life from manufacture time before the timestamp rolls over to 0 again. If you’re going to use more space keep a rollover counter you may as well just add more bits to your timestamp field instead, since every bit doubles the amount of time that field can represent.
    - mlyle 4y ago
      > If you’re going to use more space keep a rollover counter you may as well just add more bits to your timestamp field instead, since every bit doubles the amount of time that field can represent. The GPS week counter is part of the signal GPS satellites send and is not easily changed by the manufacturer of receivers. Of course, devices that accept CNAV now have a few extra bits to work with.
    - nsteel 4y ago
      Rollover fields are helpful, see https://news.ycombinator.com/item?id=32700482 https://news.ycombinator.com/item?id=32700482 for why
  - mlyle 4y ago
    > I think devices that need a longer life should keep track of the last seen timestamp, then they can detect a rollover. The device would keep a local rollover counter and just add 20 years per count. This should stay correct until that counter rolls over, or if the device is offline for >20 years. Better in some ways, worse in others. Can still result in unexpected behavior if configuration memory is lost. And if it ever accepts an invalid future time for whatever reason, it gets "stuck" in that failure state.
leshenka 4y ago
This is maybe a very dumb question but was it so expensive to add several bits (and extend its lifespan exponentially) to what is basically a counter?
- kixpanganiban 4y ago
  It's probably less about adding a few bits to the counter, and more about replacing the existing counters that are already deployed everywhere in the world. Industrial applications usually go by "if it ain't broke, don't fix it" and stick with decades-old machines as long as they still get the job done.
  - awestroke 4y ago
    The question is if it would have been that expensive to make it more than 10 bits to begin with
    - kixpanganiban 4y ago
      We as a human race have been terrible at anticipating how fast numbers grow. A similar story is that with databases, we used to think that 32bit primary keys were plenty big enough to store all the numbers we'll ever need. In all likelihood, people who manufactured these timers in the past thought that their equipment would never outlive the need for 10 bits.
      - swores 4y ago
        There's a difference between not anticipating use of products changing in scale to the point that database PKs might need to be longer (which I would assume, albeit as an uneducated guess, has seen closer to exponential than linear growth in terms of maximum length required for unique IDs in databases), vs. anticipating "this timer will definitely hit a limit in 2022, unless the physics of time as we know it somehow changes before then". Sure it's still a relatively easy mistake to either wrongly assume that all of whatever you're creating will be in landfill by 2022 if you were making it long enough ago that all devices would be dead by now for other reasons (or to unethically do that intentionally, to cause customers to repurchase products this year - seems pretty unlikely but perhaps...), or to just forget to think about it, or think about it and assume it's fine for now and can be checked later before then forgetting. But "We as a human race have been terrible at anticipating how fast numbers grow" - we're talking about a number that's growing exactly in sync with time, and with an entirely simple way of checking when the limit will be hit, so I think it's just sloppy/lazy or expecting less longevity of use for their product, more than a flaw of humanity.
snickerbockers 4y ago
Im confused...were 10-bit timers being used in recent history? Also they're using weeks as their atomic unit? This seems like something that never should have existed.
- andyjohnson0 4y ago
  The GPS system uses 10 bits for the week number part of the broadcast timestamp. This causes a rollover [1] every 19.6 years, and devices that aren't designed to anticipate it will report the current date/time as being two decades (or multiples of that) in the past. I guess the ultimate cause is data framing in the transmission protocol. The timestamp contains the number of weeks since 00:00:00 1980-01-06, together with the number of seconds since the start of the current week. The number of seconds in a week won't fit into 16 bits, so my supposition is that the designers had to also use some of the bits that could otherwise have been used for a wider week counter. I'd like to think that nowadays we'd use self-describing, upgradeable protocols. But GPS was designed in the 70s for the constrained technology of that era. And I'm pretty sure nobody anticipated how widely deployed it would be 45+ years later. [1] https://en.m.wikipedia.org/wiki/GPS_week_number_rollover https://en.m.wikipedia.org/wiki/GPS_week_number_rollover
  - GekkePrutser 4y ago
    > I'd like to think that nowadays we'd use self-describing, upgradeable protocols. That would open a whole new can of security worms though. Being able to modify a protocol in-band is something we're starting to move away from. Things are becoming more static as a precaution, like stored procedures on SQL so an attacker can't inject a change.
    - ezconnect 4y ago
      Stored procedure was the norm when I was a young programmer. Dynamic SQLs are recent and was the cause of SQL injection vulnerabilities.
  - toyg 4y ago
    > And I'm pretty sure nobody anticipated how widely deployed it would be 45+ years later. Or maybe they thought by now we would have replaced it with something better. Spacefaring was still riding high in the '70s, the brakes were only applied after '89 (sadly one of the many indirect effects of glasnost, RIP Gorbachev).
noisy_boy 4y ago
I'm flying on 17th September. Looking forward to it now.
- messe 4y ago
  Same. I hope the Atlantic is warm this time of year.
  - tommiegannert 4y ago
    It will go swimmingly?
  - dudeinjapan 4y ago
    Too bad they didn't use a float.
tanner808 4y ago
[flagged]
defrost 4y ago
Counters roll over, this is not new. What matters is how rollover is dealt with. For interest, we (mostly) all use GPS .. and the 'onboard' broadcast 10 bit GPS week counter has already rolled over twice: * midnight (UTC) August 21 to 22, 1999 * midnight (UTC) April 6 to 7, 2019 The GPS "seconds since midnight last Sunday" timer resets to zero every week. [1] https://en.wikipedia.org/wiki/GPS_week_number_rollover https://en.wikipedia.org/wiki/GPS_week_number_rollover [2] Satellite Geodesy Günter Seeber https://www.geokniga.org/bookfiles/geokniga-seeber-g-satellite-geodesy-2003.pdf https://www.geokniga.org/bookfiles/geokniga-seeber-g-satelli...
- lifthrasiir 4y ago
  Note that GPS week number is now 13 bits long since the introduction of CNAV navigation message format; devices only accepting the original NAV format are now pretty rare.
  - defrost 4y ago
    Sure, the seconds are still lapsed seconds since (weekly) epoch . . . (Not to mention the post corrections broadcast upwards from the ground to adjust for relativistic time drag between Earth surface nominal 1G and sat orbit height gravity) Point being, many timing applications use lapsed time since X counters and are part of systems designed to handle rollover. The mindset of just adding more bits to a counter or recording more decimal points isn't always appropriate or the 'best' fix.
    - dreamcompiler 4y ago
      > to adjust for relativistic time drag between Earth surface nominal 1G and sat orbit height gravity That's just the General Relativity correction. There's also a Special Relativity correction to account for time dilation caused by the satellite's speed, which works in the opposite direction to the GR correction yet the two don't exactly cancel each other. Plus there are some others (Doppler shift, Sagnac effect, etc.) https://www.aapt.org/doorway/TGRU/articles/Ashbyarticle.pdf https://www.aapt.org/doorway/TGRU/articles/Ashbyarticle.pdf
mkl 4y ago
People designed new devices with just a 10-bit time counter after Y2K?!
- Gordonjcp 4y ago
  It's something that's inherent in the GPS standard, which was developed considerably before Y2K, and it's something that's very very easy to correct for. Are you surprised that people are still designing hardware where the time rolls over to 00:00 every 24 hours?
  - mkl 4y ago
    The signal is 10-bit, but it seems silly to make the receiving device 10-bit and suffer this problem when correcting for it is easy. OP says it wasn't always corrected for, even by people with the lesson of Y2K behind them, which is what surprises me.
    - nullc 4y ago
      Every one of the devices failing at this particular date have been 'corrected' for it: They are shifting the wrap point based on their manufacturing date. The natural wrap point isn't now-- the second natural wrap was in 2018. Correcting for it more elaborately than that is challenging, in particular if you depend on remembering the number of wraps a replacement part pulled off the shelf will return wrong dates when the original was fine, also if you remember then a false (or spoofed) signal can put the device in a bricked state. Most GPS receivers (including most of the ones listed here) can be temporarily fixed by setting the date manually. But their interfaces may be inaccessible in the devices they're installed in and the fix will be lost after restart. The L1C/L2C/L5 gps signals use a 13 bit week number. Hardware for the new signals was first deployed with the IIR-M satellites in 2005 with L2C support, which were delayed from something like 1997... but no broadcasting of L2C started until 2014. Right now very few receivers support the new signals.
  - relaxing 4y ago
    Luckily the 24 hour clock comes with a rollover counter, which may be confusingly named something like “day.” Consult accompanying documentation for details. If your hardware vendor did not include a “day” counter, you may need to implement in wetware.
vailbhima 4y ago
vailbhima 4y ago
vailbhima 4y ago
calebpeterson 4y ago
What kind of industrial applications? Are we talking about manufacturing or grid/infrastructure?
- ramigb 4y ago
  Anything that uses and depends on time synchronization basically. Which is a huge list. from railways to game servers. But again, do they all use these devices or in particular 10-bit counters? I have no idea.
tyingq 4y ago
I'm wondering how comprehensive the research is that says it's just those 3 vendors and ~7 devices. Given that it's more than one vendor, it feels like a pattern that's a common mistake or design compromise. I wouldn't be surprised if the impact is broader than expected.
- tyingq 4y ago
  Ah, re-reading, it sounds like these are all from one vendor that's known by two names..."Microsemi" and "Symmetricom" (acquisition, etc). So perhaps that risk is low.
- nullc 4y ago
  All GPS receivers(1) have this issue but the date they experience it on differs from device to device because to fix the rollover issue they adjust the wrap point in their firmware based on when the firmware was made. (1) Technically the modernized L2C/L1C/L5 signals use a 13-bit week number, but because the deployment of these signals was massively delayed receivers using them are basically non-existent today. (I also have no idea if the current receivers supporting them successfully use the 13-bit week number, given that they're not guaranteed to be able to receive the modernized signals as they're still less available than L1)
  - tyingq 4y ago
    Wouldn't some of them have rollover counters or adjustable epochs, etc?
    - nullc 4y ago
      The epoch is adjustable by updating the firmware... if there is an updated firmware. That's the reason for the current batch of devices dying now: it's based on a date set in their firmware. Unfortunately these are embedded receivers so making anything adjustable isn't necessarily easy. I'm not aware of any devices that implement a rollover counter but there may be some, though a rollover counter needs to figure out how to deal with corrupted or spoofed signals and will have issues like spares off the shelf not working like the device they're replacing.
bostick 4y ago
Are there any references for this? What is the "epoch" here? 1024 weeks before Sept 17 is February 1, 2003. What is significant about that date? References welcome.
- tyingq 4y ago
  Probably what this comment is suggesting: https://news.ycombinator.com/item?id=32700347 https://news.ycombinator.com/item?id=32700347 That given the 10-bit limitation, some devices chose their own epoch date to get as much use of the 10-bits as possible. Meaning they chose some arbitrary date earlier than the device manufacture date.
m3047 4y ago
It's called "Global Positioning System" not "Global Time System". Granted, it's a pretty accurate time system in the sense that it needs to know the time to an accuracy determined by the speed of light crossing the globe: that's how it knows where you are, that's its job. Day, weeks, months, years... feh! In 7 minutes the Earth transits its own [edit] diameter; in 4 hours, the distance to the moon; in a year it's (roughly) back where it started. Great hack, but maybe you ought to know what year it is if it matters to you. No way to figure that out? No clock available which is accurate enough?
- maxbond 4y ago
  GPS is widely used as a cheap way to get a timestamp from an atomic clock. Most datacenters will have a GPS receiver to seed their NTP services, for instance. At the massive scale of GPS, all observable behavior of the system are features that someone is relying on.
MegaDeKay 4y ago
Here are a few links. This is the key one. The units with this problem are obsolete and Microsemi has done very little to publicize this. This means you have a ticking time bomb if you thought everything was good because you got through the 2019 rollover. There are a few select models where a return-to-factory firmware update can help you dodge the problem, but it is probably too late to get something turned around now if this is the first you have heard of it. If all you need is a 10 MHz or 1PPS reference, those bits will continue to function. If you need it for NTP, you've got a problem. https://sync.empowerednetworks.com/keep-your-ntp-server-current/ https://sync.empowerednetworks.com/keep-your-ntp-server-curr... Another link stating "But other GPS receiver manufacturers set a delayed date for the rollover date to occur on – September 18th, 2022". "Other" in this case = Symmetricom / Microsemi. https://www.orolia.com/will-your-network-time-servers-be-affected-by-the-september-2022-delayed-gps-rollover/ https://www.orolia.com/will-your-network-time-servers-be-aff...

400 points by modinfo 4y ago

fanf2 4y ago

This is not the GPS week counter rollover https://en.m.wikipedia.org/wiki/GPS_week_number_rollover https://en.m.wikipedia.org/wiki/GPS_week_number_rollover But instead it is a device-specific rollover. It’s relatively common for GPS receivers to handle the limited week counter by having a baked-in epoch that is more recent than the latest rollover, which gives them a nearly 20 year lifetime after that epoch instead of failing at the GPS rollover.

8organicbits 4y ago
If I'm following, the thought is: It's currently Sept 17th 2002, our counter will roll over in 2019 (17 years). But since we know 1999-2001 has already past, we can hard code logic to add 20 years to any counter that's reporting that range. That updated logic rolls over on Sept 17th 2022, which buys us three extra years. This works for devices that will EOL within 20 years. I think devices that need a longer life should keep track of the last seen timestamp, then they can detect a rollover. The device would keep a local rollover counter and just add 20 years per count. This should stay correct until that counter rolls over, or if the device is offline for >20 years.
- deleted 4y ago
  [deleted]
- bathtub365 4y ago
  The epoch is the date & time that the minimum value of that field refers to. For Unix derived or inspired machines it’s typically the Unix epoch of 00:00:00 on January 1, 1970. This means that a timestamp of “0” refers to that date and time, and increases at whatever granularity a timestamp value maps to. If you bake the epoch into the device at the factory as being the date of manufacture and your timestamp value has 20 years worth of granularity then you get a 20 year life from manufacture time before the timestamp rolls over to 0 again. If you’re going to use more space keep a rollover counter you may as well just add more bits to your timestamp field instead, since every bit doubles the amount of time that field can represent.
  - mlyle 4y ago
    > If you’re going to use more space keep a rollover counter you may as well just add more bits to your timestamp field instead, since every bit doubles the amount of time that field can represent. The GPS week counter is part of the signal GPS satellites send and is not easily changed by the manufacturer of receivers. Of course, devices that accept CNAV now have a few extra bits to work with.
  - nsteel 4y ago
    Rollover fields are helpful, see https://news.ycombinator.com/item?id=32700482 https://news.ycombinator.com/item?id=32700482 for why
- mlyle 4y ago
  > I think devices that need a longer life should keep track of the last seen timestamp, then they can detect a rollover. The device would keep a local rollover counter and just add 20 years per count. This should stay correct until that counter rolls over, or if the device is offline for >20 years. Better in some ways, worse in others. Can still result in unexpected behavior if configuration memory is lost. And if it ever accepts an invalid future time for whatever reason, it gets "stuck" in that failure state.

leshenka 4y ago

This is maybe a very dumb question but was it so expensive to add several bits (and extend its lifespan exponentially) to what is basically a counter?

kixpanganiban 4y ago
It's probably less about adding a few bits to the counter, and more about replacing the existing counters that are already deployed everywhere in the world. Industrial applications usually go by "if it ain't broke, don't fix it" and stick with decades-old machines as long as they still get the job done.
- awestroke 4y ago
  The question is if it would have been that expensive to make it more than 10 bits to begin with
  - kixpanganiban 4y ago
    We as a human race have been terrible at anticipating how fast numbers grow. A similar story is that with databases, we used to think that 32bit primary keys were plenty big enough to store all the numbers we'll ever need. In all likelihood, people who manufactured these timers in the past thought that their equipment would never outlive the need for 10 bits.
    - swores 4y ago
      There's a difference between not anticipating use of products changing in scale to the point that database PKs might need to be longer (which I would assume, albeit as an uneducated guess, has seen closer to exponential than linear growth in terms of maximum length required for unique IDs in databases), vs. anticipating "this timer will definitely hit a limit in 2022, unless the physics of time as we know it somehow changes before then". Sure it's still a relatively easy mistake to either wrongly assume that all of whatever you're creating will be in landfill by 2022 if you were making it long enough ago that all devices would be dead by now for other reasons (or to unethically do that intentionally, to cause customers to repurchase products this year - seems pretty unlikely but perhaps...), or to just forget to think about it, or think about it and assume it's fine for now and can be checked later before then forgetting. But "We as a human race have been terrible at anticipating how fast numbers grow" - we're talking about a number that's growing exactly in sync with time, and with an entirely simple way of checking when the limit will be hit, so I think it's just sloppy/lazy or expecting less longevity of use for their product, more than a flaw of humanity.

snickerbockers 4y ago

Im confused...were 10-bit timers being used in recent history? Also they're using weeks as their atomic unit? This seems like something that never should have existed.

andyjohnson0 4y ago
The GPS system uses 10 bits for the week number part of the broadcast timestamp. This causes a rollover [1] every 19.6 years, and devices that aren't designed to anticipate it will report the current date/time as being two decades (or multiples of that) in the past. I guess the ultimate cause is data framing in the transmission protocol. The timestamp contains the number of weeks since 00:00:00 1980-01-06, together with the number of seconds since the start of the current week. The number of seconds in a week won't fit into 16 bits, so my supposition is that the designers had to also use some of the bits that could otherwise have been used for a wider week counter. I'd like to think that nowadays we'd use self-describing, upgradeable protocols. But GPS was designed in the 70s for the constrained technology of that era. And I'm pretty sure nobody anticipated how widely deployed it would be 45+ years later. [1] https://en.m.wikipedia.org/wiki/GPS_week_number_rollover https://en.m.wikipedia.org/wiki/GPS_week_number_rollover
- GekkePrutser 4y ago
  > I'd like to think that nowadays we'd use self-describing, upgradeable protocols. That would open a whole new can of security worms though. Being able to modify a protocol in-band is something we're starting to move away from. Things are becoming more static as a precaution, like stored procedures on SQL so an attacker can't inject a change.
  - ezconnect 4y ago
    Stored procedure was the norm when I was a young programmer. Dynamic SQLs are recent and was the cause of SQL injection vulnerabilities.
- toyg 4y ago
  > And I'm pretty sure nobody anticipated how widely deployed it would be 45+ years later. Or maybe they thought by now we would have replaced it with something better. Spacefaring was still riding high in the '70s, the brakes were only applied after '89 (sadly one of the many indirect effects of glasnost, RIP Gorbachev).

noisy_boy 4y ago

I'm flying on 17th September. Looking forward to it now.

messe 4y ago
Same. I hope the Atlantic is warm this time of year.
- tommiegannert 4y ago
  It will go swimmingly?
- dudeinjapan 4y ago
  Too bad they didn't use a float.

tanner808 4y ago

[flagged]

defrost 4y ago

Counters roll over, this is not new. What matters is how rollover is dealt with. For interest, we (mostly) all use GPS .. and the 'onboard' broadcast 10 bit GPS week counter has already rolled over twice: * midnight (UTC) August 21 to 22, 1999 * midnight (UTC) April 6 to 7, 2019 The GPS "seconds since midnight last Sunday" timer resets to zero every week. [1] https://en.wikipedia.org/wiki/GPS_week_number_rollover https://en.wikipedia.org/wiki/GPS_week_number_rollover [2] Satellite Geodesy Günter Seeber https://www.geokniga.org/bookfiles/geokniga-seeber-g-satellite-geodesy-2003.pdf https://www.geokniga.org/bookfiles/geokniga-seeber-g-satelli...

lifthrasiir 4y ago
Note that GPS week number is now 13 bits long since the introduction of CNAV navigation message format; devices only accepting the original NAV format are now pretty rare.
- defrost 4y ago
  Sure, the seconds are still lapsed seconds since (weekly) epoch . . . (Not to mention the post corrections broadcast upwards from the ground to adjust for relativistic time drag between Earth surface nominal 1G and sat orbit height gravity) Point being, many timing applications use lapsed time since X counters and are part of systems designed to handle rollover. The mindset of just adding more bits to a counter or recording more decimal points isn't always appropriate or the 'best' fix.
  - dreamcompiler 4y ago
    > to adjust for relativistic time drag between Earth surface nominal 1G and sat orbit height gravity That's just the General Relativity correction. There's also a Special Relativity correction to account for time dilation caused by the satellite's speed, which works in the opposite direction to the GR correction yet the two don't exactly cancel each other. Plus there are some others (Doppler shift, Sagnac effect, etc.) https://www.aapt.org/doorway/TGRU/articles/Ashbyarticle.pdf https://www.aapt.org/doorway/TGRU/articles/Ashbyarticle.pdf

mkl 4y ago

People designed new devices with just a 10-bit time counter after Y2K?!

Gordonjcp 4y ago
It's something that's inherent in the GPS standard, which was developed considerably before Y2K, and it's something that's very very easy to correct for. Are you surprised that people are still designing hardware where the time rolls over to 00:00 every 24 hours?
- mkl 4y ago
  The signal is 10-bit, but it seems silly to make the receiving device 10-bit and suffer this problem when correcting for it is easy. OP says it wasn't always corrected for, even by people with the lesson of Y2K behind them, which is what surprises me.
  - nullc 4y ago
    Every one of the devices failing at this particular date have been 'corrected' for it: They are shifting the wrap point based on their manufacturing date. The natural wrap point isn't now-- the second natural wrap was in 2018. Correcting for it more elaborately than that is challenging, in particular if you depend on remembering the number of wraps a replacement part pulled off the shelf will return wrong dates when the original was fine, also if you remember then a false (or spoofed) signal can put the device in a bricked state. Most GPS receivers (including most of the ones listed here) can be temporarily fixed by setting the date manually. But their interfaces may be inaccessible in the devices they're installed in and the fix will be lost after restart. The L1C/L2C/L5 gps signals use a 13 bit week number. Hardware for the new signals was first deployed with the IIR-M satellites in 2005 with L2C support, which were delayed from something like 1997... but no broadcasting of L2C started until 2014. Right now very few receivers support the new signals.
- relaxing 4y ago
  Luckily the 24 hour clock comes with a rollover counter, which may be confusingly named something like “day.” Consult accompanying documentation for details. If your hardware vendor did not include a “day” counter, you may need to implement in wetware.

vailbhima 4y ago

calebpeterson 4y ago

What kind of industrial applications? Are we talking about manufacturing or grid/infrastructure?

ramigb 4y ago
Anything that uses and depends on time synchronization basically. Which is a huge list. from railways to game servers. But again, do they all use these devices or in particular 10-bit counters? I have no idea.

tyingq 4y ago

I'm wondering how comprehensive the research is that says it's just those 3 vendors and ~7 devices. Given that it's more than one vendor, it feels like a pattern that's a common mistake or design compromise. I wouldn't be surprised if the impact is broader than expected.

tyingq 4y ago
Ah, re-reading, it sounds like these are all from one vendor that's known by two names..."Microsemi" and "Symmetricom" (acquisition, etc). So perhaps that risk is low.
nullc 4y ago
All GPS receivers(1) have this issue but the date they experience it on differs from device to device because to fix the rollover issue they adjust the wrap point in their firmware based on when the firmware was made. (1) Technically the modernized L2C/L1C/L5 signals use a 13-bit week number, but because the deployment of these signals was massively delayed receivers using them are basically non-existent today. (I also have no idea if the current receivers supporting them successfully use the 13-bit week number, given that they're not guaranteed to be able to receive the modernized signals as they're still less available than L1)
- tyingq 4y ago
  Wouldn't some of them have rollover counters or adjustable epochs, etc?
  - nullc 4y ago
    The epoch is adjustable by updating the firmware... if there is an updated firmware. That's the reason for the current batch of devices dying now: it's based on a date set in their firmware. Unfortunately these are embedded receivers so making anything adjustable isn't necessarily easy. I'm not aware of any devices that implement a rollover counter but there may be some, though a rollover counter needs to figure out how to deal with corrupted or spoofed signals and will have issues like spares off the shelf not working like the device they're replacing.

bostick 4y ago

Are there any references for this? What is the "epoch" here? 1024 weeks before Sept 17 is February 1, 2003. What is significant about that date? References welcome.

tyingq 4y ago
Probably what this comment is suggesting: https://news.ycombinator.com/item?id=32700347 https://news.ycombinator.com/item?id=32700347 That given the 10-bit limitation, some devices chose their own epoch date to get as much use of the 10-bits as possible. Meaning they chose some arbitrary date earlier than the device manufacture date.

m3047 4y ago

It's called "Global Positioning System" not "Global Time System". Granted, it's a pretty accurate time system in the sense that it needs to know the time to an accuracy determined by the speed of light crossing the globe: that's how it knows where you are, that's its job. Day, weeks, months, years... feh! In 7 minutes the Earth transits its own [edit] diameter; in 4 hours, the distance to the moon; in a year it's (roughly) back where it started. Great hack, but maybe you ought to know what year it is if it matters to you. No way to figure that out? No clock available which is accurate enough?

maxbond 4y ago
GPS is widely used as a cheap way to get a timestamp from an atomic clock. Most datacenters will have a GPS receiver to seed their NTP services, for instance. At the massive scale of GPS, all observable behavior of the system are features that someone is relying on.

MegaDeKay 4y ago

Here are a few links. This is the key one. The units with this problem are obsolete and Microsemi has done very little to publicize this. This means you have a ticking time bomb if you thought everything was good because you got through the 2019 rollover. There are a few select models where a return-to-factory firmware update can help you dodge the problem, but it is probably too late to get something turned around now if this is the first you have heard of it. If all you need is a 10 MHz or 1PPS reference, those bits will continue to function. If you need it for NTP, you've got a problem. https://sync.empowerednetworks.com/keep-your-ntp-server-current/ https://sync.empowerednetworks.com/keep-your-ntp-server-curr... Another link stating "But other GPS receiver manufacturers set a delayed date for the rollover date to occur on – September 18th, 2022". "Other" in this case = Symmetricom / Microsemi. https://www.orolia.com/will-your-network-time-servers-be-affected-by-the-september-2022-delayed-gps-rollover/ https://www.orolia.com/will-your-network-time-servers-be-aff...