18 ms·

Ntimed – NTPD replacement

89 points by 925dk 12y ago

ams6110 12y ago
According to the OpenNTPD web page, "The portable version is outdated and in need of a maintainer." Why another from-scratch rewrote, instead of helping with that project, especially when it was also motivated by an aversion to the "100,000 KLOC" in the ntp.org ntpd?
- lazyjones 12y ago
  > Why another from-scratch rewrote, And why oh why in C with so many safer options available today ..
  - nwmcsween 12y ago
    What safer options are there with equal size, overhead (read: none) and not full of layers upon layers of abstractions?
    - SrslyJosh 12y ago
      That sounds like you're implying that safety has no value, the only "overhead" worth considering is CPU cycles (what about the programmer's time/effort/sanity?), and that abstractions have negative value. I don't think that's how you're supposed to software.
      - zzzcpan 12y ago
        Yeah. Especially considering that CPU overhead has very little value for time syncing program.
        StillBored 12y ago
        but latency, and jitter, are important in this application. Two things that discount most of the "safer" languages. After ignoring inmature ones, the field is pretty barren.
        jnbiche 12y ago
        Yes, but if you're building an app for the future, then what excuse is there for not using Rust, which is both very low latency and very close to a 1.0 release? I get that sometimes you just want to hack in a language that's familiar to you, but then call your project a fun side project and not the "NTPD replacement" (and yes, I know who phk is).
        xorcist 12y ago
        Please start by showing us the PLL in Rust (details are available at phk's blog) and reason about how those changes affects the workings of the program. Is it more readable? Less? More exact? That I would be interested in.
        jnbiche 12y ago
        It's a little bit sad that at -3 votes, this is the lowest voted comment I've ever had in my ~4 years on HN, lower than a number of controversial political comments. To clarify, I'm not claiming that everyone should start using Rust. Now was I claiming that PHK is somehow unqualified for the task, far from it. I'm not even saying C is a bad language (I like C), or that people should stop using C altogether. But if you're building an security-critical application designed to be used in the future across millions of servers, and one that requires low-latency, I simply don't understand why you wouldn't strongly consider Rust, which offers strong guarantees about memory and type safety along with low latency and a very modern and complete standard library. Or Ada. Or OCaml.
        phkamp 12y ago
        It's always a good idea to do a quick back-of-the-envelope calculation before generalizing like that: About 2 million servers were sold world-wide in 2014Q2 Assume 25% runs UNIX and NTPD -> half a million servers Assume NTPD uses 0.1% of machine resources -> 500 fully loaded servers. Assume 100W/server -> 50 kW 50kW for half a year -> 220,000 kWh Assume 500g CO2/kWh -> 110 tons of CO2 QED: I think CPU overhead is a very relevant concern for time synching programs.
        lazyjones 12y ago
        > Assume NTPD uses 0.1% of machine resources -> 500 fully loaded servers. Bold assumption. It uses 0.0006% on my slow 6 years old box (over 13 days uptime). Let's be generous and say it's 0.001%, then you get 1.1 ton of CO2, which is the equivalent of the production of about 32.8 Kg of beef. I don't think such dramatic conclusions can be made from those facts.
        phkamp 12y ago
        First, I doubt your kernels statistics actually account correctly at that level of precision. (If it does, that alone would waste a lot of CPU cycles!) Second, have you checked the memory footprint of NTPD ? "machine resources" is more than CPU cycles. And don't forget: That number was one quarters purchase of servers, running for half a year. All the servers bought in 2010, 2011, 2012, 2013 and the other half of the year were not included. Things add up.
        lazyjones 12y ago
        > Second, have you checked the memory footprint of NTPD ? "machine resources" is more than CPU cycles. 284K, which is a whopping 0.04% of available memory. I don't see how this increases CO2 output though. > Things add up To a still very insignificant number, dwarfed by the beef consumption of the people reading this. I'd say there are better arguments in favour of using dated languages than this.
        phkamp 12y ago
      - threeseed 12y ago
        It's a core operating system component not some web app. Overhead is important since Linux is designed to run on a whole spectrum of devices. Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead. And frankly nobody could care less about programmer's time/effort/sanity in this case. It is a tightly focused, specific utility that shouldn't see too many changes after a certain point.
        moe 12y ago
        Overhead is important since Linux is designed to run on a whole spectrum of devices. Like the difference between a ntpd.c vs ntpd.go is going to matter for anyone... If you really find yourself stuck with an embedded device small enough that a few megabytes of RAM matter then you can still use the old C implementations, it's not like they are going away. Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead. In this case abstractions mean you can probably write the whole thing in under 2 kloc, get memory safety for free, and base it on a network stack that other people actually use and debug independently. It is a tightly focused, specific utility that shouldn't see too many changes after a certain point. Umm. Yea. Right. Tell that to the ntpd guys with their 100kloc codebase...
        mcgoo 12y ago
        The jitter introduced by garbage collection in a go implementation could well be significant, even with the new "never pause the mutator for more than 1ms" guarantee.
        moe 12y ago
        How much of the code is actually sensitive to interruptions? Why not write that part in C and leave the rest to the more suitable language? I would guess that's a rather small portion of the codebase, essentially the callouts to adjtime(). I don't buy into the low-latency FUD. C isn't a magical realtime language either; what happens when the system is under high load? What about delays in the kernel/network stack? As a layman I would think incoming frames will have to be stamped with a hardware high resolution timestamp at arrival anyway, regardless of the language. And yes, this internal timestamp will have to be compared immediately before setting the hwclock. I would argue this is the only part that really needs to be written in a GC free language. I might be wrong on this, but I'd like to see a stronger argument than "could well be significant" for writing yet another generation of a baseline system daemon in an unsafe language... And speaking of latency: If this really is such an issue, why hasn't NTP long been moved into the kernel?
      - nwmcsween 12y ago
        Safety has value it just doesn't imply overhead. Abstractions are a negative, they make reasoning burdensome.
        cbd1984 12y ago
        Abstractions are what make it possible to reason about complex systems. Being able to ignore parts of your codebase is the only way to make changes to it once it gets beyond a trivial size. The whole of the Unix philosophy is enforcing this policy of strong abstractions by making each part of a complex overall system its own binary, such that they can only communicate using the file I/O abstraction, possibly via pipes. C was therefore designed for a system which promotes abstractions.
        deleted 12y ago
        [deleted]
        cbd1984 12y ago
        Even if your comment both made sense and was factually accurate, it wouldn't have much relevance to the kind of C programming I'm talking about: For example, mmap(2) isn't defined in the C standard but in POSIX, but it's universal in the Unix and Unix-like OS world to the extent programs can safely rely on it. Standards are good and useful things, but they're only fully written down in retrospect.
        tedunangst 12y ago
        Section 7.19.3.
        function_seven 12y ago
        I'm getting 7.21.3, reading here[1], but that's the draft spec for C11. I couldn't find the official one anyhere. [1] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf
    - dezgeg 12y ago
      Rust. Some of it's design goals (as stated on their webpage) are among the exact things you mentioned: zero-cost abstractions, minimal runtime, guaranteed memory safety, threads without data races.
      - threeseed 12y ago
        Rust isn't even at version 1.0 yet and is probably a year away from being in a "suitable for production" state. System time is far, far too critical to take chances with.
        vertex-four 12y ago
        On the other hand, this program is going to take a year or so before anybody seriously considers using it, simply due to the fact that few people are going to do something other than their distro default for such a small part of their system.
        lttlrck 12y ago
        waiting for rust to hit production readiness will not avoid that.
        vertex-four 12y ago
        The point is simply that if one started a project now with the idea that it would probably only be a serious competitor in a year, Rust will most likely be ready by then - it doesn't really matter if Rust's ready right now if your project's not going to be used by anybody right now.
      - nwmcsween 12y ago
        Zero-cost to whom? The developer? no as now the implementation details have to be known, the machine? no as time will be spent in the compiler churning away removing junk or if the compiler isn't smart enough including it. Also the C++ / rust definition of zero-cost is only if you don't use the abstraction. Minimal runtime is relative.
    - rtpg 12y ago
      Can you explain why abstraction is bad? Abstraction does _not_ mean your program will be slower. In fact, the opposite can happen, because the compiler has more context to work with, and can get rid of stuff. Haskell + stream fusion gives better results than well tuned C in some very typical situations http://research.microsoft.com/en-us/um/people/simonpj/papers/ndp/haskell-beats-C.pdf http://research.microsoft.com/en-us/um/people/simonpj/papers... Even just comparing C++/C, C++'s templating mechanisms mean that generic containers can be optimised on a per-type basis. Abstraction and overhead are not as tightly coupled as they used to be, and not introducing Heartbleed-style bugs is worth a lot in my opinion
      - nwmcsween 12y ago
        Abstraction in my definition is hiding of complexity and it does just that.
      - saosebastiao 12y ago
        The problem with abstraction is that in order to deterministically control performance, you have to break down all the walls of the abstraction anyway. If I'm not 100% confident that my compiler will optimize a vector op with SIMD, and I absolutely require SIMD performance, then I end up having to know the entire layer cake to be able to verify that it is doing exactly that. In most cases it isn't a bad thing at all. But a use case like a timekeeping daemon, where a single unexpected garbage collection pause could cause a world of problems, it most definitely is more pain than gain.
      - phkamp 12y ago
        Abstractions are certainly not "bad", there are many abstractions in Ntimed, it's also pretty object-oriented if you care to take a look. This is more a matter of "the right language for the job" and for timing-critical systems programming running as root, that language is C. And there are plenty of common abstractions I would love see added to the C language: basic linked lists, byte-endianess and packing for struct members, validity intervals for integers and FP variables (like Ada!) Unfortunately my taste seems to be the direct opposite of ISO-C which have instead wasted time giving us another thread-API.
  - vezzy-fnord 12y ago
    I'd wager that the lower level nature would be useful for something like accurate timekeeping, which is actually an intricate task if you intend on doing it fully.
  - stonogo 12y ago
    The appropriate systems language for use on UNIX and Linux is C. The entire interface is based on C conventions. Sometimes, when all you have is nails, you really ought to reach for a hammer.
    - asveikau 12y ago
      I seriously think the C-hater crowd on HN is just disappointed that they can't reason well about pointers. They can't write good C so no one can! It needs, like, more javascript, man. (Preemptive response to downvoters: please have a sense of humor about yourselves. :P)
      - jnbiche 12y ago
        You realize that the most vociferous of those decrying this language choice come from the proponents of a language that has 3 different types of pointers, right? You should check out Rust and Go. Their users are most definitely among those who fail to understand C pointers.
        jnbiche 12y ago
        Oops, that was supposed to be "are not among those who fail". I'm a Rust supporter and think Go is also a better alternative to C in many instances. My point was, Rust supporters have no problems understanding pointers.
        easytiger 12y ago
        > Go is also a better alternative to C in many instances. Go isn't in the same ballpark nor do the authors (venerable as they are) seem to have considered that many people need consistent power over their run time execution when microseconds matter. stop the world is not a viable memory management trade off. Also managing memory in c++ these days is pretty "easy". I've leived for years with people spending hundreds of man hours a year trying to get around issues caused by STW pauses in high throughput java applications. Makes no sense.
        12y ago
  - rab_oof 12y ago
    The Rust and Go clones popping up as Show HN will predictably follow.
  - msiebuhr 12y ago
    AFAIK, PHK has written quite a bit of the FreeBSD kernel, most of Varnish (and a few other popular bits) in C. I think it's safe to say that he's forgotten more about C/kernel/low-level/... programming than HN'ers will ever know.
  - mrweasel 12y ago
    C programs are still much smaller than something written in many of the newer languages. This still matters for embedded software. The portability of C is also huge because you want this running on a large number of hardware platforms. Everything from tiny home routers to the biggest server imaginable. For something like an ntpd daemon you also want a language that many developers read and understand. C is used by more developers than pretty much any of the newer safer languages. It might be old and weird, but C is still really hard to beat.
  - phkamp 12y ago
    1. C is not unsafe, if you use it as it was intended. 2. Because this is exactly the kind of task C was intended for: systems programming. 3. Because I want this to be a light-weight and portable program. 4. Because I have 30+ years of C-experience and happen to like the language.
- gaadd33 12y ago
  Given PHK's interest in timekeeping I would guess it's going to be more accurate and precise than both ntp.org ntpd and OpenNTPD. However beyond that speculation, I don't know. It would be interesting to know.
  - msiebuhr 12y ago
    Check his series of posts on the subject: http://phk.freebsd.dk/time/index.html http://phk.freebsd.dk/time/index.html (TL;DR: Yes, he thought things through and are ntimed is implementing a few new tricks.)
- zzzcpan 12y ago
  Well, he answered why in the first ntpd thread: "The main reason I didn't start from OpenBSD's NTP implementation is that it is not aimed at being part of a larger family of time-keeping programs, like I intend to deliver, so it wouldn't save me any time in the end." https://news.ycombinator.com/item?id=8776155 https://news.ycombinator.com/item?id=8776155
atoponce 12y ago
If Ntimed separates out the client from the daemon, so I can install just the client package only, IMO that will be a big win for computer time synchronization. One thing that has always bothered me with the NTP project, is the lack of a separate daemon and client. If you want the NTP project's client, then you must also install the daemon, even if it is listening only on localhost, or if you opt to not start it up on boot. It still must be installed though. On my Debian system, I only have the following "clients" available: * libnet-ntp-perl * openntpd (which also installs a listening daemon) * python-ntplib In most cases, such as on my laptop, workstations, and embedded systems, I don't want the complexities of a full blown daemon running, nor do I want to manage firewalling and configuring the daemon to listen on localhost to prevent remote queries. The only other two options are programming libraries. As such, I'm looking forward to systemd-timesyncd. It implements a simple SNTP client to synchronize your clock. As such, the full NTP complexity is not a concern. This is very likely an attractive scenario for most GNU/Linux installations, as most users won't be interested in running an NTP daemon synchronizing other NTP client clocks.
- swinglock 12y ago
  Debian has a chrony package as well, another alternative ntpd.
  - easytiger 12y ago
    That simply won't do for OP as he seems to want solely client only mode implementation of NTP. chrony supports both. He seems to think the authors of these daemons don't have his best interests at heart by adding functionality he doesn't want personally yet many other people will use.
    - phkamp 12y ago
      I fully agree with him that client-only mode deserves its own program, given that less than 1% of all computers ever need to be NTP servers.
      - 12y ago
zanny 12y ago
Why the hell would you write this in C, when so many other programming languages exist now. I mean, you could write a time sync program in Python today. But if you still want to be maximally performant since you intend to run on millions of computers, Go and Rust are new and fresh and memory safe, and modern C++ is great too. A lot better than malloc and pointer arithmetic. I get that the original ntpd is almost 20 years old, and opentpd is about a decade old, and back then writing every in C was forgivable, because the alternates were not as far ahead in every metric possible as they are today. Now it just looks like someone trying to use Cobol or Fortran in new production products.
- kolev 12y ago
  I think Rust should be the default choice from projects like this going forward.
  - perlgeek 12y ago
    While I agree with the motivation behind your statement, I don't think Rust has proven itself yet to the degree that it should be a default choice for anything. Remember that compilers and runtime libraries can have bugs too, and new ones tend to have more bugs than those maintained over some time.
    - kolev 12y ago
      Yes, runtimes and libraries can have bugs, too, agreed, but at least you will have to just recompile with the patched version vs having to fix and test your code as fast as possible. It's more efficient as well as fixing the core will benefit many (well, of course, that would have compromised many as well... to start with).
  - ta0967 12y ago
    you should publish a Rust version of Ntimed, then its advantages would be obvious and nobody would touch C again. while i'm here, compare the tone of your comment with that of the following (lifted from somebody's comment in another HN thread): Personally I wish this was done with $otherlang.. But then again I'm not writing it myself, so I can't really complain.
iyyappangovid 12y ago
hi
jumanji 12y ago
I think there is too much negativity here. Thank you phk for developing free and open source software. Don't let the complainers wear you down.
nullc 12y ago
This is interesting work, NTP is a surprisingly large piece of software considering it's narrow task... and at this point rather crufty, ... e.g. in spite of having a ton of monitoring code that keeps turning up vulnerabilities, it's rather hard to monitor. It has complex filtering, but still can perform rather poorly (esp with broken servers available). etc. And accurate and (increasingly-) precise time keeping is mission (and, in some cases, security) critical to many applications. Time really should be a solved problem but sadly it's not. Though fairly accurate temperature compensated oscillators are not terribly expensive, they don't find their way into typical server clocks. Many public ntp servers give bogus data. etc. Meanwhile, kernel timekeeping has become more sophisticated. I suppose one of the principles in varnish was letting the kernel do its job, so there may be some opportunity to apply that principle here too. So I think there is plenty that PHK could do interesting here while keeping the codebase quite focused and compact. I do wonder how he would compare this effort to some of the other NTP alternatives (in particular chrony).
- phkamp 12y ago
  I'm not keen on saying too much about Chrony, I'd rather let people without a stake in the game provide comparisons. But obviously: if I had throught it was just the thing, I wouldn't have started writing Ntimed. I also don't expect Ntimed to out-compete Chrony, and if it did, I'd have to start a fourth alternative myself, to keep competition healty. A very large part of NTPDs problem was that there were no competition, which meant that everything got crammed into NTPD, come hell or 300KLOC. We saw this also with GCC, which had become a stagnated arrogant monopoly, and suddenly LLVM forced them to care about users again. Having competing projects is good thing, and I hope Chrony sees things that way too.
zimbatm 12y ago
The biggest annoyance with NTPD is that it will give up synching if the time delta is too large. It just shows that the software was primarily coded to be a server. If you just use NTPD to keep your machine times in sync `chrony` is a better solution for now.
- feld 12y ago
  There's a setting for that. tinker panic 0
- nullc 12y ago
  Giving up is also a security feature. Whats more likely? your system clock sped up by a factor of 2 and everything kept working, or some guy on your network is trying to dork around with your time so he can bypass some time limited authentication? Of course, the software needs to correctly handle cases like suspend where the time may honestly need to step. If it doesn't thats a short coming of the software, not a reason to eagerly make "presumed impossible" leaps just because some unauthenticated packets on the network tell you to.
cnst 12y ago
Well, so far -- this is only a client, so, it's not really much of a unique replacement of anything just yet. Apart from OpenBSD's ntpd (OpenNTPD) -- http://bxr.su/OpenBSD/usr.sbin/ntpd/ http://bxr.su/OpenBSD/usr.sbin/ntpd/ -- which is a compete ntpd client/server solution done the OpenBSD way since some 2004, there's also DragonFly's dntpd -- http://bxr.su/DragonFly/usr.sbin/dntpd/ http://bxr.su/DragonFly/usr.sbin/dntpd/ -- a client-only ntpd, originally written by Matt Dillon in 2005 -- http://lists.dragonflybsd.org/pipermail/commits/2005-April/242742.html http://lists.dragonflybsd.org/pipermail/commits/2005-April/2... -- e.g. nearly concurrently with OpenNTPD development and polish. Oh, yeah, and PHK hates OpenNTPD. I'm surprised he apparently has recently used such nice words about OpenNTPD! One of the main reasons he was against OpenBSD's sensors framework going into FreeBSD was the timedelta sensor support, a type of sensor which basically tells you the offset of your system clock compared to some other clock source (e.g. http://mdoc.su/o/nmea.4 http://mdoc.su/o/nmea.4 / http://bxr.su/OpenBSD/sys/kern/tty_nmea.c http://bxr.su/OpenBSD/sys/kern/tty_nmea.c), and these timedelta hardware sensors are used by OpenBSD's ntpd to set the correct time, which seems to do the job just fine.

89 points by 925dk 12y ago

According to the OpenNTPD web page, "The portable version is outdated and in need of a maintainer." Why another from-scratch rewrote, instead of helping with that project, especially when it was also motivated by an aversion to the "100,000 KLOC" in the ntp.org ntpd?

lazyjones 12y ago
> Why another from-scratch rewrote, And why oh why in C with so many safer options available today ..
- nwmcsween 12y ago
  What safer options are there with equal size, overhead (read: none) and not full of layers upon layers of abstractions?
  - SrslyJosh 12y ago
    That sounds like you're implying that safety has no value, the only "overhead" worth considering is CPU cycles (what about the programmer's time/effort/sanity?), and that abstractions have negative value. I don't think that's how you're supposed to software.
    - zzzcpan 12y ago
      Yeah. Especially considering that CPU overhead has very little value for time syncing program.
      - StillBored 12y ago
        but latency, and jitter, are important in this application. Two things that discount most of the "safer" languages. After ignoring inmature ones, the field is pretty barren.
        jnbiche 12y ago
        Yes, but if you're building an app for the future, then what excuse is there for not using Rust, which is both very low latency and very close to a 1.0 release? I get that sometimes you just want to hack in a language that's familiar to you, but then call your project a fun side project and not the "NTPD replacement" (and yes, I know who phk is).
        xorcist 12y ago
        Please start by showing us the PLL in Rust (details are available at phk's blog) and reason about how those changes affects the workings of the program. Is it more readable? Less? More exact? That I would be interested in.
        jnbiche 12y ago
        It's a little bit sad that at -3 votes, this is the lowest voted comment I've ever had in my ~4 years on HN, lower than a number of controversial political comments. To clarify, I'm not claiming that everyone should start using Rust. Now was I claiming that PHK is somehow unqualified for the task, far from it. I'm not even saying C is a bad language (I like C), or that people should stop using C altogether. But if you're building an security-critical application designed to be used in the future across millions of servers, and one that requires low-latency, I simply don't understand why you wouldn't strongly consider Rust, which offers strong guarantees about memory and type safety along with low latency and a very modern and complete standard library. Or Ada. Or OCaml.
      - phkamp 12y ago
        It's always a good idea to do a quick back-of-the-envelope calculation before generalizing like that: About 2 million servers were sold world-wide in 2014Q2 Assume 25% runs UNIX and NTPD -> half a million servers Assume NTPD uses 0.1% of machine resources -> 500 fully loaded servers. Assume 100W/server -> 50 kW 50kW for half a year -> 220,000 kWh Assume 500g CO2/kWh -> 110 tons of CO2 QED: I think CPU overhead is a very relevant concern for time synching programs.
        lazyjones 12y ago
        > Assume NTPD uses 0.1% of machine resources -> 500 fully loaded servers. Bold assumption. It uses 0.0006% on my slow 6 years old box (over 13 days uptime). Let's be generous and say it's 0.001%, then you get 1.1 ton of CO2, which is the equivalent of the production of about 32.8 Kg of beef. I don't think such dramatic conclusions can be made from those facts.
        phkamp 12y ago
        First, I doubt your kernels statistics actually account correctly at that level of precision. (If it does, that alone would waste a lot of CPU cycles!) Second, have you checked the memory footprint of NTPD ? "machine resources" is more than CPU cycles. And don't forget: That number was one quarters purchase of servers, running for half a year. All the servers bought in 2010, 2011, 2012, 2013 and the other half of the year were not included. Things add up.
        lazyjones 12y ago
        > Second, have you checked the memory footprint of NTPD ? "machine resources" is more than CPU cycles. 284K, which is a whopping 0.04% of available memory. I don't see how this increases CO2 output though. > Things add up To a still very insignificant number, dwarfed by the beef consumption of the people reading this. I'd say there are better arguments in favour of using dated languages than this.
        phkamp 12y ago
    - threeseed 12y ago
      It's a core operating system component not some web app. Overhead is important since Linux is designed to run on a whole spectrum of devices. Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead. And frankly nobody could care less about programmer's time/effort/sanity in this case. It is a tightly focused, specific utility that shouldn't see too many changes after a certain point.
      - moe 12y ago
        Overhead is important since Linux is designed to run on a whole spectrum of devices. Like the difference between a ntpd.c vs ntpd.go is going to matter for anyone... If you really find yourself stuck with an embedded device small enough that a few megabytes of RAM matter then you can still use the old C implementations, it's not like they are going away. Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead. In this case abstractions mean you can probably write the whole thing in under 2 kloc, get memory safety for free, and base it on a network stack that other people actually use and debug independently. It is a tightly focused, specific utility that shouldn't see too many changes after a certain point. Umm. Yea. Right. Tell that to the ntpd guys with their 100kloc codebase...
        mcgoo 12y ago
        The jitter introduced by garbage collection in a go implementation could well be significant, even with the new "never pause the mutator for more than 1ms" guarantee.
        moe 12y ago
        How much of the code is actually sensitive to interruptions? Why not write that part in C and leave the rest to the more suitable language? I would guess that's a rather small portion of the codebase, essentially the callouts to adjtime(). I don't buy into the low-latency FUD. C isn't a magical realtime language either; what happens when the system is under high load? What about delays in the kernel/network stack? As a layman I would think incoming frames will have to be stamped with a hardware high resolution timestamp at arrival anyway, regardless of the language. And yes, this internal timestamp will have to be compared immediately before setting the hwclock. I would argue this is the only part that really needs to be written in a GC free language. I might be wrong on this, but I'd like to see a stronger argument than "could well be significant" for writing yet another generation of a baseline system daemon in an unsafe language... And speaking of latency: If this really is such an issue, why hasn't NTP long been moved into the kernel?
    - nwmcsween 12y ago
      Safety has value it just doesn't imply overhead. Abstractions are a negative, they make reasoning burdensome.
      - cbd1984 12y ago
        Abstractions are what make it possible to reason about complex systems. Being able to ignore parts of your codebase is the only way to make changes to it once it gets beyond a trivial size. The whole of the Unix philosophy is enforcing this policy of strong abstractions by making each part of a complex overall system its own binary, such that they can only communicate using the file I/O abstraction, possibly via pipes. C was therefore designed for a system which promotes abstractions.
        deleted 12y ago
        [deleted]
        cbd1984 12y ago
        Even if your comment both made sense and was factually accurate, it wouldn't have much relevance to the kind of C programming I'm talking about: For example, mmap(2) isn't defined in the C standard but in POSIX, but it's universal in the Unix and Unix-like OS world to the extent programs can safely rely on it. Standards are good and useful things, but they're only fully written down in retrospect.
        tedunangst 12y ago
        Section 7.19.3.
        function_seven 12y ago
        I'm getting 7.21.3, reading here[1], but that's the draft spec for C11. I couldn't find the official one anyhere. [1] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf
  - dezgeg 12y ago
    Rust. Some of it's design goals (as stated on their webpage) are among the exact things you mentioned: zero-cost abstractions, minimal runtime, guaranteed memory safety, threads without data races.
    - threeseed 12y ago
      Rust isn't even at version 1.0 yet and is probably a year away from being in a "suitable for production" state. System time is far, far too critical to take chances with.
      - vertex-four 12y ago
        On the other hand, this program is going to take a year or so before anybody seriously considers using it, simply due to the fact that few people are going to do something other than their distro default for such a small part of their system.
        lttlrck 12y ago
        waiting for rust to hit production readiness will not avoid that.
        vertex-four 12y ago
        The point is simply that if one started a project now with the idea that it would probably only be a serious competitor in a year, Rust will most likely be ready by then - it doesn't really matter if Rust's ready right now if your project's not going to be used by anybody right now.
    - nwmcsween 12y ago
      Zero-cost to whom? The developer? no as now the implementation details have to be known, the machine? no as time will be spent in the compiler churning away removing junk or if the compiler isn't smart enough including it. Also the C++ / rust definition of zero-cost is only if you don't use the abstraction. Minimal runtime is relative.
  - rtpg 12y ago
    Can you explain why abstraction is bad? Abstraction does _not_ mean your program will be slower. In fact, the opposite can happen, because the compiler has more context to work with, and can get rid of stuff. Haskell + stream fusion gives better results than well tuned C in some very typical situations http://research.microsoft.com/en-us/um/people/simonpj/papers/ndp/haskell-beats-C.pdf http://research.microsoft.com/en-us/um/people/simonpj/papers... Even just comparing C++/C, C++'s templating mechanisms mean that generic containers can be optimised on a per-type basis. Abstraction and overhead are not as tightly coupled as they used to be, and not introducing Heartbleed-style bugs is worth a lot in my opinion
    - nwmcsween 12y ago
      Abstraction in my definition is hiding of complexity and it does just that.
    - saosebastiao 12y ago
      The problem with abstraction is that in order to deterministically control performance, you have to break down all the walls of the abstraction anyway. If I'm not 100% confident that my compiler will optimize a vector op with SIMD, and I absolutely require SIMD performance, then I end up having to know the entire layer cake to be able to verify that it is doing exactly that. In most cases it isn't a bad thing at all. But a use case like a timekeeping daemon, where a single unexpected garbage collection pause could cause a world of problems, it most definitely is more pain than gain.
    - phkamp 12y ago
      Abstractions are certainly not "bad", there are many abstractions in Ntimed, it's also pretty object-oriented if you care to take a look. This is more a matter of "the right language for the job" and for timing-critical systems programming running as root, that language is C. And there are plenty of common abstractions I would love see added to the C language: basic linked lists, byte-endianess and packing for struct members, validity intervals for integers and FP variables (like Ada!) Unfortunately my taste seems to be the direct opposite of ISO-C which have instead wasted time giving us another thread-API.
- vezzy-fnord 12y ago
  I'd wager that the lower level nature would be useful for something like accurate timekeeping, which is actually an intricate task if you intend on doing it fully.
- stonogo 12y ago
  The appropriate systems language for use on UNIX and Linux is C. The entire interface is based on C conventions. Sometimes, when all you have is nails, you really ought to reach for a hammer.
  - asveikau 12y ago
    I seriously think the C-hater crowd on HN is just disappointed that they can't reason well about pointers. They can't write good C so no one can! It needs, like, more javascript, man. (Preemptive response to downvoters: please have a sense of humor about yourselves. :P)
    - jnbiche 12y ago
      You realize that the most vociferous of those decrying this language choice come from the proponents of a language that has 3 different types of pointers, right? You should check out Rust and Go. Their users are most definitely among those who fail to understand C pointers.
      - jnbiche 12y ago
        Oops, that was supposed to be "are not among those who fail". I'm a Rust supporter and think Go is also a better alternative to C in many instances. My point was, Rust supporters have no problems understanding pointers.
        easytiger 12y ago
        > Go is also a better alternative to C in many instances. Go isn't in the same ballpark nor do the authors (venerable as they are) seem to have considered that many people need consistent power over their run time execution when microseconds matter. stop the world is not a viable memory management trade off. Also managing memory in c++ these days is pretty "easy". I've leived for years with people spending hundreds of man hours a year trying to get around issues caused by STW pauses in high throughput java applications. Makes no sense.
      - 12y ago
- rab_oof 12y ago
  The Rust and Go clones popping up as Show HN will predictably follow.
- msiebuhr 12y ago
  AFAIK, PHK has written quite a bit of the FreeBSD kernel, most of Varnish (and a few other popular bits) in C. I think it's safe to say that he's forgotten more about C/kernel/low-level/... programming than HN'ers will ever know.
- mrweasel 12y ago
  C programs are still much smaller than something written in many of the newer languages. This still matters for embedded software. The portability of C is also huge because you want this running on a large number of hardware platforms. Everything from tiny home routers to the biggest server imaginable. For something like an ntpd daemon you also want a language that many developers read and understand. C is used by more developers than pretty much any of the newer safer languages. It might be old and weird, but C is still really hard to beat.
- phkamp 12y ago
  1. C is not unsafe, if you use it as it was intended. 2. Because this is exactly the kind of task C was intended for: systems programming. 3. Because I want this to be a light-weight and portable program. 4. Because I have 30+ years of C-experience and happen to like the language.
gaadd33 12y ago
Given PHK's interest in timekeeping I would guess it's going to be more accurate and precise than both ntp.org ntpd and OpenNTPD. However beyond that speculation, I don't know. It would be interesting to know.
- msiebuhr 12y ago
  Check his series of posts on the subject: http://phk.freebsd.dk/time/index.html http://phk.freebsd.dk/time/index.html (TL;DR: Yes, he thought things through and are ntimed is implementing a few new tricks.)
zzzcpan 12y ago
Well, he answered why in the first ntpd thread: "The main reason I didn't start from OpenBSD's NTP implementation is that it is not aimed at being part of a larger family of time-keeping programs, like I intend to deliver, so it wouldn't save me any time in the end." https://news.ycombinator.com/item?id=8776155 https://news.ycombinator.com/item?id=8776155

atoponce 12y ago

If Ntimed separates out the client from the daemon, so I can install just the client package only, IMO that will be a big win for computer time synchronization. One thing that has always bothered me with the NTP project, is the lack of a separate daemon and client. If you want the NTP project's client, then you must also install the daemon, even if it is listening only on localhost, or if you opt to not start it up on boot. It still must be installed though. On my Debian system, I only have the following "clients" available: * libnet-ntp-perl * openntpd (which also installs a listening daemon) * python-ntplib In most cases, such as on my laptop, workstations, and embedded systems, I don't want the complexities of a full blown daemon running, nor do I want to manage firewalling and configuring the daemon to listen on localhost to prevent remote queries. The only other two options are programming libraries. As such, I'm looking forward to systemd-timesyncd. It implements a simple SNTP client to synchronize your clock. As such, the full NTP complexity is not a concern. This is very likely an attractive scenario for most GNU/Linux installations, as most users won't be interested in running an NTP daemon synchronizing other NTP client clocks.

swinglock 12y ago
Debian has a chrony package as well, another alternative ntpd.
- easytiger 12y ago
  That simply won't do for OP as he seems to want solely client only mode implementation of NTP. chrony supports both. He seems to think the authors of these daemons don't have his best interests at heart by adding functionality he doesn't want personally yet many other people will use.
  - phkamp 12y ago
    I fully agree with him that client-only mode deserves its own program, given that less than 1% of all computers ever need to be NTP servers.
    - 12y ago

zanny 12y ago

Why the hell would you write this in C, when so many other programming languages exist now. I mean, you could write a time sync program in Python today. But if you still want to be maximally performant since you intend to run on millions of computers, Go and Rust are new and fresh and memory safe, and modern C++ is great too. A lot better than malloc and pointer arithmetic. I get that the original ntpd is almost 20 years old, and opentpd is about a decade old, and back then writing every in C was forgivable, because the alternates were not as far ahead in every metric possible as they are today. Now it just looks like someone trying to use Cobol or Fortran in new production products.

kolev 12y ago
I think Rust should be the default choice from projects like this going forward.
- perlgeek 12y ago
  While I agree with the motivation behind your statement, I don't think Rust has proven itself yet to the degree that it should be a default choice for anything. Remember that compilers and runtime libraries can have bugs too, and new ones tend to have more bugs than those maintained over some time.
  - kolev 12y ago
    Yes, runtimes and libraries can have bugs, too, agreed, but at least you will have to just recompile with the patched version vs having to fix and test your code as fast as possible. It's more efficient as well as fixing the core will benefit many (well, of course, that would have compromised many as well... to start with).
- ta0967 12y ago
  you should publish a Rust version of Ntimed, then its advantages would be obvious and nobody would touch C again. while i'm here, compare the tone of your comment with that of the following (lifted from somebody's comment in another HN thread): Personally I wish this was done with $otherlang.. But then again I'm not writing it myself, so I can't really complain.

iyyappangovid 12y ago

jumanji 12y ago

I think there is too much negativity here. Thank you phk for developing free and open source software. Don't let the complainers wear you down.

nullc 12y ago

This is interesting work, NTP is a surprisingly large piece of software considering it's narrow task... and at this point rather crufty, ... e.g. in spite of having a ton of monitoring code that keeps turning up vulnerabilities, it's rather hard to monitor. It has complex filtering, but still can perform rather poorly (esp with broken servers available). etc. And accurate and (increasingly-) precise time keeping is mission (and, in some cases, security) critical to many applications. Time really should be a solved problem but sadly it's not. Though fairly accurate temperature compensated oscillators are not terribly expensive, they don't find their way into typical server clocks. Many public ntp servers give bogus data. etc. Meanwhile, kernel timekeeping has become more sophisticated. I suppose one of the principles in varnish was letting the kernel do its job, so there may be some opportunity to apply that principle here too. So I think there is plenty that PHK could do interesting here while keeping the codebase quite focused and compact. I do wonder how he would compare this effort to some of the other NTP alternatives (in particular chrony).

phkamp 12y ago
I'm not keen on saying too much about Chrony, I'd rather let people without a stake in the game provide comparisons. But obviously: if I had throught it was just the thing, I wouldn't have started writing Ntimed. I also don't expect Ntimed to out-compete Chrony, and if it did, I'd have to start a fourth alternative myself, to keep competition healty. A very large part of NTPDs problem was that there were no competition, which meant that everything got crammed into NTPD, come hell or 300KLOC. We saw this also with GCC, which had become a stagnated arrogant monopoly, and suddenly LLVM forced them to care about users again. Having competing projects is good thing, and I hope Chrony sees things that way too.

zimbatm 12y ago

The biggest annoyance with NTPD is that it will give up synching if the time delta is too large. It just shows that the software was primarily coded to be a server. If you just use NTPD to keep your machine times in sync `chrony` is a better solution for now.

feld 12y ago
There's a setting for that. tinker panic 0
nullc 12y ago
Giving up is also a security feature. Whats more likely? your system clock sped up by a factor of 2 and everything kept working, or some guy on your network is trying to dork around with your time so he can bypass some time limited authentication? Of course, the software needs to correctly handle cases like suspend where the time may honestly need to step. If it doesn't thats a short coming of the software, not a reason to eagerly make "presumed impossible" leaps just because some unauthenticated packets on the network tell you to.

cnst 12y ago

Well, so far -- this is only a client, so, it's not really much of a unique replacement of anything just yet. Apart from OpenBSD's ntpd (OpenNTPD) -- http://bxr.su/OpenBSD/usr.sbin/ntpd/ http://bxr.su/OpenBSD/usr.sbin/ntpd/ -- which is a compete ntpd client/server solution done the OpenBSD way since some 2004, there's also DragonFly's dntpd -- http://bxr.su/DragonFly/usr.sbin/dntpd/ http://bxr.su/DragonFly/usr.sbin/dntpd/ -- a client-only ntpd, originally written by Matt Dillon in 2005 -- http://lists.dragonflybsd.org/pipermail/commits/2005-April/242742.html http://lists.dragonflybsd.org/pipermail/commits/2005-April/2... -- e.g. nearly concurrently with OpenNTPD development and polish. Oh, yeah, and PHK hates OpenNTPD. I'm surprised he apparently has recently used such nice words about OpenNTPD! One of the main reasons he was against OpenBSD's sensors framework going into FreeBSD was the timedelta sensor support, a type of sensor which basically tells you the offset of your system clock compared to some other clock source (e.g. http://mdoc.su/o/nmea.4 http://mdoc.su/o/nmea.4 / http://bxr.su/OpenBSD/sys/kern/tty_nmea.c http://bxr.su/OpenBSD/sys/kern/tty_nmea.c), and these timedelta hardware sensors are used by OpenBSD's ntpd to set the correct time, which seems to do the job just fine.