6 ms·

NTPsec is not quite a full rewrite

72 points by wtbob 11y ago

dmit 11y ago
Interesting, this and phk's ntimed are both sponsored by the Linux Foundation. I guess it makes sense on some level to have concurrent projects that tackle the same problem from different directions.
- hga 11y ago
  Here's ESR's comment on ntimed (http://esr.ibiblio.org/?p=6863&cpage=1#comment-1635375 http://esr.ibiblio.org/?p=6863&cpage=1#comment-1635375): esr on 2015-10-09 at 09:22:05 said: >Eric, what do you think of ntimed and their goal of replacing ntpd? PHK is a bright guy with many clues. I respect him a lot, and I say this despite the fact that he’s hostile to me because he thinks my open-source advocacy has encouraged crappy programming. I have not looked closely at ntimed but I have little doubt it is high-quality work, probably as good as anything I am capable of producing, possibly better. That said, I think ntimed is going nowhere. A clean-sheet design was the right thing from a technical point of view, but it’s not what the time-service userbase wants. Large time-service users are intensely conservative and risk-averse; what they want, and what NTPsec will deliver, is not radical innovation but a cleaned-up and hardened version of what they know. The only active GitHub ntimed fork is for the Amiga/AROS version, and phk's GitHub visible work as of late seems to be mostly on Varnish Cache.
  - dmit 11y ago
    >> I have not looked closely at ntimed but I have little doubt it is high-quality work, probably as good as anything I am capable of producing, possibly better. Ok, that is just precious. =) One is a kernel hacker and time-nuts regular with one of the world's most precise clocks in his home, and the other recently wrote: > There’s a longstanding legend that only Dave Mills ever really understood the Byzantine time-synchronization algorithms at NTP’s heart, but I used to be a mathematician and I think I already get most of it outside of a few arcana about statistical filtering of noisy signals. From a purely technical standpoint, I know who I'd bet my money on. By the way, phk recently mentioned that he was getting back into ntimed, so fingers crossed that we get two safe and correct NTP alternatives in the observable future. https://twitter.com/allanjude/status/635635386832715777 https://twitter.com/allanjude/status/635635386832715777
    - hga 11y ago
      While he said as of August 23rd that he's agin working on ntimed, that's not reflected in GitHub updates: https://github.com/bsdphk/Ntimed https://github.com/bsdphk/Ntimed And it bears mentioning that ESR has been working on GPSD for more than a decade (a daemon to extract accurate time from GPS receivers), and he sure sounds math literate to my not entirely untutored ears in various discussions over the years. Given the well known curse of total rewrites, which I've experienced myself, I too know who I'd bet my money on.
      - dfc 11y ago
        Saying that GPSD is "a daemon to extract accurate time from GPS receivers" is a little misleading. GPSd primary function is to make "data on the location/course/velocity of the sensors available to...the host computer." It also has functionality to share data with ntpd but timekeeping is certainly not the primary purpose.
      - gonzo 11y ago
        > "he sure sounds math literate" This is the standard ESR parlor trick. Eric took several courses in math and philosophy at the University of Pennsylvania. http://poynder.blogspot.com/2006/03/interview-with-eric-raymond.html http://poynder.blogspot.com/2006/03/interview-with-eric-raym... "As a freshman at the University of Pennsylvania Raymond was immediately marked out as a potential math prodigy. Having found school insufficiently stretching for his above average talents, however, he lacked the necessary discipline or emotional maturity to cope with the demands of an undergraduate course, and after suffering a "math burnout" left without a degree." I'd put my money on phk. Every. Time.
        deleted 11y ago
        [deleted]
      - imglorp 11y ago
        > ESR has been working on GPSD for more than a decade Timekeeping and location calculation is very easy to do clean sheet. That is, until you start to hit corner cases and exceptions and the realities of physical real world inputs like jitter and offset and dispersion (and those are just the grit that ntpq tells you about) and all the crud humans made up like leap seconds and calendrical calculations. My money's on the guy who's been blowing the grit out of the machine.
  - gonzo 11y ago
    Let's look at what phk had to say about esr's advocacy. https://queue.acm.org/detail.cfm?id=2349257 https://queue.acm.org/detail.cfm?id=2349257 Pull quote: "That is the sorry reality of the bazaar Raymond praised in his book: a pile of old festering hacks, endlessly copied and pasted by a clueless generation of IT "professionals" who wouldn't recognize sound IT architecture if you hit them over the head with it. It is hard to believe today, but under this embarrassing mess lies the ruins of the beautiful cathedral of Unix, deservedly famous for its simplicity of design, its economy of features, and its elegance of execution. (Sic transit gloria mundi, etc.)"
    - nickpsecurity 11y ago
      I regularly drop that link when I see people talk about FOSS's many eyeballs like they inherently mean something. That said, I recently got to read Cathedral and Bazaar. In that, he made good points esp about parallel debugging. I think the truth is somewhere between their two claims. Plan on trying to figure out the specifics in the future. I lean more toward PHK given FOSS model has never produced a high assurance system while proprietary and core team models have done many: https://www.schneier.com/blog/archives/2014/04/reverse_heartbl.html#c5581737 https://www.schneier.com/blog/archives/2014/04/reverse_heart... Recently I've been redefining the situation to try to find a model that balances the benefits of open-source against the ability to pay a competent team to ensure it's a cathedral. Start of that is here: https://www.schneier.com/blog/archives/2014/05/friday_squid_bl_424.html#c6051639 https://www.schneier.com/blog/archives/2014/05/friday_squid_...
- RyJones 11y ago
  From what I could find ESR's work is funded by ICEI, which is funded by an NSF grant and a CACR grant. Am I missing something? http://icei.org/activities/ntp.html http://icei.org/activities/ntp.html
  - dmit 11y ago
    http://esr.ibiblio.org/?p=6820 http://esr.ibiblio.org/?p=6820 > A couple of stories by Charles Babcock and (my coincidentally old friend) Steven J. Vaughan-Nichols have mentioned the existence of an ‘NTPsec’ project being funded by the Core Infrastructure Initiative as an alternative and perhaps eventual replacement for the reference implementation of Network Time Protocol maintained by Harlan Stenn and the Network Time Foundation. I confirm that NTPsec does exist, and that I am deeply involved in it. https://www.coreinfrastructure.org/faq https://www.coreinfrastructure.org/faq > The Core Infrastructure Initiative is a multi-million dollar project to fund and support critical elements of the global information infrastructure. It is organized by The Linux Foundation and supported by Amazon Web Services, Adobe, Bloomberg, Cisco, Dell, Facebook, Fujitsu, Google, Hitachi, HP, Huawei, IBM, Intel, Microsoft, NetApp, NEC, Qualcomm, RackSpace, salesforce.com, and VMware.
    - thrownaway2424 11y ago
      It seems like a serious mistake to waste your money on ESR-code. The man's lifetime output of software could have been barfed out by any randomly-selected industry veteran in 2 weeks. Writing all of those Slashdot comments and participating in lunatic-fringe demonstrations of openly carried firearms, of course, must have taken many years of full-time effort.
  - gonzo 11y ago
    http://www.ntpsec.org http://www.ntpsec.org
hannob 11y ago
One thing that I find a bit worrying is that there are now two projects that try to create better implementations of the existing ntp protocol, but the protocol itself is inherently insecure. ntp right now is a clear-text protocol with no authentication and no protection at all against man-in-the-middle-attacks. There are two authentication modes from which one is insecure and the other relies on symmetric keys, which is impractical. There's some work done on ntp security [1], I hope we'll see this implemented soon. [1] https://tools.ietf.org/html/draft-ietf-ntp-using-nts-for-ntp https://tools.ietf.org/html/draft-ietf-ntp-using-nts-for-ntp
- dfc 11y ago
  I am assuming that when you refer to "the other project trying to create a better implementation" you are referring to ntimed? It seems that you are forgetting about Chrony and OpenNTPd.
tptacek 11y ago
* strcpy, strncpy, strcat: Use strlcpy and strlcat instead. * sprintf, vsprintf: use snprintf and vsnprintf instead. * In scanf and friends, the %s format without length limit is banned. * strtok: use strtok_r() or unroll this into the obvious loop. * gets: Use fgets instead. * gmtime(), localtime(), asctime(), ctime(): use the reentrant *_r variants. * tmpnam() - use mkstemp() or tmpfile() instead. * dirname() - the Linux version is re-entrant but this property is not portable. And so, instead of competent systems code of 1995 vintage, we get ESR code of 1998 vintage. Presumably, someone else, somewhere else, is solving the problem that needs to be solved, which is to get an NTP implementation that is simple but complete enough for the 98% use case deployed based either on thoroughly modern C code (which looks nothing like a heap of strlcats and snprintfs) or a safer language, like Rust or Go. Maybe that's ntimed, maybe it's something else. It's not this, whatever the hell this is going to turn out to be.
- NelsonMinar 11y ago
  I really like the description of ntimed, but I'm worried there's been no blog updates or code uploaded to GitHub in months. https://github.com/bsdphk/Ntimed https://github.com/bsdphk/Ntimed
  - alricb 11y ago
    To be fair, we know about the progress on NTPsec by ESR's posts, but there isn't a public repository.
- ploxiln 11y ago
  strlcpy and c99/POSIX-2001 didn't really become widely used until the early 2000s. I disagree that the difference in "vintage" between early-2000s and today's software makes the real difference - what really matters is the quality. And it's not like this project doesn't use DVCS or static analysis. It even uses a modern alternative build system! It's very hard to quantify or even make specific rules about quality. These rules are just the basics. The strlcpy rule in particular is simplified, there are times when you need to use strncpy: when the source string might not be terminated (but openbsd guys say they use memcpy in that case ... anyway, it takes some thought). 2004 vintage C code is the sweet spot for portable systems software right now IMHO. But the real difference in quality can't be identified by any simplistic rule or attribute.
nickpsecurity 11y ago
A real-time guard can handle most of what it needs: packet-inspection, rate limiting, spotting NTP layer abuse, transport security, authentication, small TCB, real-time, etc. Highly-assured guards have been used for about every other insecure app and protocol so I don't see why not this one. Throw in tech like Code Pointer Integrity or Softbound on client side for extra measure. Or reimplement it in an inherently safer language.
scaramanga 11y ago
Hilarious reading, of course, "I do have an an advantage because I’m very bright"
technion 11y ago
There's at least one thing ESR and phk agree on - the autoconf situation. I'm looking at the list of tests done in current ntpd. * Checking whether compiler accepts -g * Checking whether compiler supports C89 * Checking if stdlib.h exists * Check if compiler supports -o * Hunts for just about every command line tool, sed, awk, ln, grep * Is size_t a type? There are dozens of checks that could be replaced with "Checking for OS = Windows". There are specific code paths for Digital Unix 4.0, a fix specific to AIX 4.3 with the comment "hopefully adding ... fixing this without breaking any other platform", Ultrix, winnt3.5?, How many NTP developers routinely test new code against all these platforms? However it's done, more big name programs need to move on from this.

72 points by wtbob 11y ago

dmit 11y ago

Interesting, this and phk's ntimed are both sponsored by the Linux Foundation. I guess it makes sense on some level to have concurrent projects that tackle the same problem from different directions.

hga 11y ago
Here's ESR's comment on ntimed (http://esr.ibiblio.org/?p=6863&cpage=1#comment-1635375 http://esr.ibiblio.org/?p=6863&cpage=1#comment-1635375): esr on 2015-10-09 at 09:22:05 said: >Eric, what do you think of ntimed and their goal of replacing ntpd? PHK is a bright guy with many clues. I respect him a lot, and I say this despite the fact that he’s hostile to me because he thinks my open-source advocacy has encouraged crappy programming. I have not looked closely at ntimed but I have little doubt it is high-quality work, probably as good as anything I am capable of producing, possibly better. That said, I think ntimed is going nowhere. A clean-sheet design was the right thing from a technical point of view, but it’s not what the time-service userbase wants. Large time-service users are intensely conservative and risk-averse; what they want, and what NTPsec will deliver, is not radical innovation but a cleaned-up and hardened version of what they know. The only active GitHub ntimed fork is for the Amiga/AROS version, and phk's GitHub visible work as of late seems to be mostly on Varnish Cache.
- dmit 11y ago
  >> I have not looked closely at ntimed but I have little doubt it is high-quality work, probably as good as anything I am capable of producing, possibly better. Ok, that is just precious. =) One is a kernel hacker and time-nuts regular with one of the world's most precise clocks in his home, and the other recently wrote: > There’s a longstanding legend that only Dave Mills ever really understood the Byzantine time-synchronization algorithms at NTP’s heart, but I used to be a mathematician and I think I already get most of it outside of a few arcana about statistical filtering of noisy signals. From a purely technical standpoint, I know who I'd bet my money on. By the way, phk recently mentioned that he was getting back into ntimed, so fingers crossed that we get two safe and correct NTP alternatives in the observable future. https://twitter.com/allanjude/status/635635386832715777 https://twitter.com/allanjude/status/635635386832715777
  - hga 11y ago
    While he said as of August 23rd that he's agin working on ntimed, that's not reflected in GitHub updates: https://github.com/bsdphk/Ntimed https://github.com/bsdphk/Ntimed And it bears mentioning that ESR has been working on GPSD for more than a decade (a daemon to extract accurate time from GPS receivers), and he sure sounds math literate to my not entirely untutored ears in various discussions over the years. Given the well known curse of total rewrites, which I've experienced myself, I too know who I'd bet my money on.
    - dfc 11y ago
      Saying that GPSD is "a daemon to extract accurate time from GPS receivers" is a little misleading. GPSd primary function is to make "data on the location/course/velocity of the sensors available to...the host computer." It also has functionality to share data with ntpd but timekeeping is certainly not the primary purpose.
    - gonzo 11y ago
      > "he sure sounds math literate" This is the standard ESR parlor trick. Eric took several courses in math and philosophy at the University of Pennsylvania. http://poynder.blogspot.com/2006/03/interview-with-eric-raymond.html http://poynder.blogspot.com/2006/03/interview-with-eric-raym... "As a freshman at the University of Pennsylvania Raymond was immediately marked out as a potential math prodigy. Having found school insufficiently stretching for his above average talents, however, he lacked the necessary discipline or emotional maturity to cope with the demands of an undergraduate course, and after suffering a "math burnout" left without a degree." I'd put my money on phk. Every. Time.
      - deleted 11y ago
        [deleted]
    - imglorp 11y ago
      > ESR has been working on GPSD for more than a decade Timekeeping and location calculation is very easy to do clean sheet. That is, until you start to hit corner cases and exceptions and the realities of physical real world inputs like jitter and offset and dispersion (and those are just the grit that ntpq tells you about) and all the crud humans made up like leap seconds and calendrical calculations. My money's on the guy who's been blowing the grit out of the machine.
- gonzo 11y ago
  Let's look at what phk had to say about esr's advocacy. https://queue.acm.org/detail.cfm?id=2349257 https://queue.acm.org/detail.cfm?id=2349257 Pull quote: "That is the sorry reality of the bazaar Raymond praised in his book: a pile of old festering hacks, endlessly copied and pasted by a clueless generation of IT "professionals" who wouldn't recognize sound IT architecture if you hit them over the head with it. It is hard to believe today, but under this embarrassing mess lies the ruins of the beautiful cathedral of Unix, deservedly famous for its simplicity of design, its economy of features, and its elegance of execution. (Sic transit gloria mundi, etc.)"
  - nickpsecurity 11y ago
    I regularly drop that link when I see people talk about FOSS's many eyeballs like they inherently mean something. That said, I recently got to read Cathedral and Bazaar. In that, he made good points esp about parallel debugging. I think the truth is somewhere between their two claims. Plan on trying to figure out the specifics in the future. I lean more toward PHK given FOSS model has never produced a high assurance system while proprietary and core team models have done many: https://www.schneier.com/blog/archives/2014/04/reverse_heartbl.html#c5581737 https://www.schneier.com/blog/archives/2014/04/reverse_heart... Recently I've been redefining the situation to try to find a model that balances the benefits of open-source against the ability to pay a competent team to ensure it's a cathedral. Start of that is here: https://www.schneier.com/blog/archives/2014/05/friday_squid_bl_424.html#c6051639 https://www.schneier.com/blog/archives/2014/05/friday_squid_...
RyJones 11y ago
From what I could find ESR's work is funded by ICEI, which is funded by an NSF grant and a CACR grant. Am I missing something? http://icei.org/activities/ntp.html http://icei.org/activities/ntp.html
- dmit 11y ago
  http://esr.ibiblio.org/?p=6820 http://esr.ibiblio.org/?p=6820 > A couple of stories by Charles Babcock and (my coincidentally old friend) Steven J. Vaughan-Nichols have mentioned the existence of an ‘NTPsec’ project being funded by the Core Infrastructure Initiative as an alternative and perhaps eventual replacement for the reference implementation of Network Time Protocol maintained by Harlan Stenn and the Network Time Foundation. I confirm that NTPsec does exist, and that I am deeply involved in it. https://www.coreinfrastructure.org/faq https://www.coreinfrastructure.org/faq > The Core Infrastructure Initiative is a multi-million dollar project to fund and support critical elements of the global information infrastructure. It is organized by The Linux Foundation and supported by Amazon Web Services, Adobe, Bloomberg, Cisco, Dell, Facebook, Fujitsu, Google, Hitachi, HP, Huawei, IBM, Intel, Microsoft, NetApp, NEC, Qualcomm, RackSpace, salesforce.com, and VMware.
  - thrownaway2424 11y ago
    It seems like a serious mistake to waste your money on ESR-code. The man's lifetime output of software could have been barfed out by any randomly-selected industry veteran in 2 weeks. Writing all of those Slashdot comments and participating in lunatic-fringe demonstrations of openly carried firearms, of course, must have taken many years of full-time effort.
- gonzo 11y ago
  http://www.ntpsec.org http://www.ntpsec.org

hannob 11y ago

One thing that I find a bit worrying is that there are now two projects that try to create better implementations of the existing ntp protocol, but the protocol itself is inherently insecure. ntp right now is a clear-text protocol with no authentication and no protection at all against man-in-the-middle-attacks. There are two authentication modes from which one is insecure and the other relies on symmetric keys, which is impractical. There's some work done on ntp security [1], I hope we'll see this implemented soon. [1] https://tools.ietf.org/html/draft-ietf-ntp-using-nts-for-ntp https://tools.ietf.org/html/draft-ietf-ntp-using-nts-for-ntp

dfc 11y ago
I am assuming that when you refer to "the other project trying to create a better implementation" you are referring to ntimed? It seems that you are forgetting about Chrony and OpenNTPd.

tptacek 11y ago

* strcpy, strncpy, strcat: Use strlcpy and strlcat instead. * sprintf, vsprintf: use snprintf and vsnprintf instead. * In scanf and friends, the %s format without length limit is banned. * strtok: use strtok_r() or unroll this into the obvious loop. * gets: Use fgets instead. * gmtime(), localtime(), asctime(), ctime(): use the reentrant *_r variants. * tmpnam() - use mkstemp() or tmpfile() instead. * dirname() - the Linux version is re-entrant but this property is not portable. And so, instead of competent systems code of 1995 vintage, we get ESR code of 1998 vintage. Presumably, someone else, somewhere else, is solving the problem that needs to be solved, which is to get an NTP implementation that is simple but complete enough for the 98% use case deployed based either on thoroughly modern C code (which looks nothing like a heap of strlcats and snprintfs) or a safer language, like Rust or Go. Maybe that's ntimed, maybe it's something else. It's not this, whatever the hell this is going to turn out to be.

NelsonMinar 11y ago
I really like the description of ntimed, but I'm worried there's been no blog updates or code uploaded to GitHub in months. https://github.com/bsdphk/Ntimed https://github.com/bsdphk/Ntimed
- alricb 11y ago
  To be fair, we know about the progress on NTPsec by ESR's posts, but there isn't a public repository.
ploxiln 11y ago
strlcpy and c99/POSIX-2001 didn't really become widely used until the early 2000s. I disagree that the difference in "vintage" between early-2000s and today's software makes the real difference - what really matters is the quality. And it's not like this project doesn't use DVCS or static analysis. It even uses a modern alternative build system! It's very hard to quantify or even make specific rules about quality. These rules are just the basics. The strlcpy rule in particular is simplified, there are times when you need to use strncpy: when the source string might not be terminated (but openbsd guys say they use memcpy in that case ... anyway, it takes some thought). 2004 vintage C code is the sweet spot for portable systems software right now IMHO. But the real difference in quality can't be identified by any simplistic rule or attribute.

nickpsecurity 11y ago

A real-time guard can handle most of what it needs: packet-inspection, rate limiting, spotting NTP layer abuse, transport security, authentication, small TCB, real-time, etc. Highly-assured guards have been used for about every other insecure app and protocol so I don't see why not this one. Throw in tech like Code Pointer Integrity or Softbound on client side for extra measure. Or reimplement it in an inherently safer language.

scaramanga 11y ago

Hilarious reading, of course, "I do have an an advantage because I’m very bright"

technion 11y ago

There's at least one thing ESR and phk agree on - the autoconf situation. I'm looking at the list of tests done in current ntpd. * Checking whether compiler accepts -g * Checking whether compiler supports C89 * Checking if stdlib.h exists * Check if compiler supports -o * Hunts for just about every command line tool, sed, awk, ln, grep * Is size_t a type? There are dozens of checks that could be replaced with "Checking for OS = Windows". There are specific code paths for Digital Unix 4.0, a fix specific to AIX 4.3 with the comment "hopefully adding ... fixing this without breaking any other platform", Ultrix, winnt3.5?, How many NTP developers routinely test new code against all these platforms? However it's done, more big name programs need to move on from this.