5 ms·
> Is there a way that these could mediated by a capability without having to incur syscall overhead? Is that even warranted? What applications can you imagine
by naasking 9y ago
> Is there a way that these could mediated by a capability without having to incur syscall overhead?
Is that even warranted? What applications can you imagine would make so many clock calls so as to incur noticeable overhead?
Your example of logging costs might work, but I'm very skeptical that user/kernel transition costs for a clock call would drown out the costs of writing the log entry to disk.
But to answer your question directly, the ability to access the clock in a shared memory segment can itself be reified as a handle that's granted to a process. The process would then issue a map operation and provide an address at which to map the clock (or you could just always map it at the same address too if that's preferable for some reason).
- kllrnohj 9y agoclock_gettime is extremely heavily used by nearly everything. Any form of work queue that supports delayed work, for example, is sitting on clock_gettime. Any form of media uses it heavily, as does self-monitoring to look for performance regressions in the wild. You might be shielded from this depending on what level you're working at, but there's a reason that vDSO exists basically solely for clock_gettime, too. Yes it allows for timing attacks, but you can't really avoid that, either, not without utterly crippling your platform.
- naasking 9y ago> Any form of work queue that supports delayed work, for example, is sitting on clock_gettime. That seems strange to me. Why would you use the clock/gettime call for that instead of a work counter of some sort?
- kllrnohj 9y agoI meant time-delayed. Eg, run this in 10ms type of thing. How else would you do things like exponential backoff?
- naasking 9y agoRight, but your original statement was about clock_gettime, hence my confusion. Something like sleep() is an operation on your own schedule, not an operation on a global system clock. That's not nearly as problematic. Consider if you wanted to virtualize a process, say to deterministically replay it to trigger a fault or something. sleep(100 ticks) doesn't require additional kernel support for virtualization, but an ambient clock requires a lot of extra kernel support. If the clock were only accessible via a handle, then you could proxy invocations on the handle without any extra support in the kernel. See my other comment for more details: https://news.ycombinator.com/item?id=16817462 https://news.ycombinator.com/item?id=16817462
- kllrnohj 9y agoYou could use sleep, sure, but then you require a seperate thread for every delayed message, which is a whole different ball of not-fun. Otherwise what actually happens when you do a postDelayed(func, delay) is it immediately is translated into a postAt(func, clock_gettime() + delay). Then as messages are consumed the work queue consistently knows how long it should wait to wake back up, no matter how many delayed tasks are in flight.
- naasking 9y agoI feel like I'm missing some context. I thought you were talking about work stealing in threaded systems, in which case sleeping seems reasonable. Are you instead talking about some kind of event loop? If so, then clock_gettime seems like it's merely convenient, not essential. You could just as easily keep a ticket incremented on successful operations or successive loops (or some other metric), and exponential backoff is a wait operation on a ticket number. Unless you're suggesting the delay must be based on real time for some reason?
- kllrnohj 9y ago> You could just as easily keep a ticket incremented [..], and exponential backoff is a wait operation on a ticket number. Congrats, you've re-invented clock_gettime(CLOCK_MONOTONIC) :) You're right that it's not essential to have, but if you can re-implement it then you've also just re-introduced the timing attack that the removal of clock_gettime was trying to prevent. The trivial-ness with which a clock suitable for timing attacks is able to be created is literally why SharedArrayBuffer was panic-removed from all major browsers. Because that's all you need, shared memory & a worker. Congrats, you have a high-precision timer with zero OS support. So as soon as you allow any form of threading or shared memory to occur you've given apps a high precision timer, so you might as well just give them an actual timer, too.
- paulmlewis 9y agoTed Unangst had a few posts where time getting featured heavily, e.g. https://www.tedunangst.com/flak/post/firefox-vs-rthreads https://www.tedunangst.com/flak/post/firefox-vs-rthreads and the linked mailing list thread https://marc.info/?l=openbsd-ports&m=144649333120401&w=2 https://marc.info/?l=openbsd-ports&m=144649333120401&w=2 There's a few more related articles on his blog if you search, and I've read of it being implicated elsewhere, that's not to say the software calling that much was doing the right thing, just that it had impact.
- lambda 9y agoEach log entry isn't necessarily written to disk one at a time. They will generally be buffered until enough have happened to need to flush. And they can be written compressed, and frequently have a lot of redundancy, so it can take a while before you accumulate enough data to need to flush to disk. And besides logging, there are things like nanosleep or spinlocks which may briefly spin while querying the time before yielding. I know of at least one CDN which was measurably impacted by the gettimeofday issue. And yes, you could have the access to the address itself be something you are given access to via a capability/handle, but then the system call itself (which is actually a vDSO call) wouldn't have to actually take a handle as a parameter.
- naasking 9y ago> And besides logging, there are things like nanosleep or spinlocks which may briefly spin while querying the time before yielding. Another poster said something similar, but using the clock for this seems strange to me. Yielding a time slice for a certain number of ticks doesn't need access to the current time. I have less objection to an ambient yield since that's really an operation on your own schedule capability. If you're after some kind of exponential backoff for a spinlock, that again doesn't seem to need the current time so much as a growing counter of the number of ticks to sleep. > And yes, you could have the access to the address itself be something you are given access to via a capability/handle, but then the system call itself (which is actually a vDSO call) wouldn't have to actually take a handle as a parameter. Correct, you'd use the handle to install an ambient clock in your environment. This leaves open the possibility that you can easily virtualize the clock by proxying the clock handle, ie. instead of the kernel updating your shared memory segment, it's another process. The point being that reifying everything as a handle makes arbitrary virtualization patterns possible, but having ambient authorities all the way down to root makes it much more difficult.
- hueving 9y ago>Another poster said something similar, but using the clock for this seems strange to me. Does it matter if it seems strange to you? The GP gave real world examples of where this is an impact. I'm not sure why you are arguing against the design.
- jeff_marshall 9y agoI've got a customer that wants to get access to the PTP[1] registers of a NIC from user space. The customer suggests that context switches introduce too much latency and indeterminism (pre systems integration phase - lots of software from different teams ends up running on the final platform) to introduce a reference monitor (capability-based microkernel or otherwise) given known techniques on modern hardware. I'm afraid I can't share the specifics because I don't have them, but I trust the source. I will grant that this case is an outlier, but this customer use-case is real enough to drive development dollars. It may not be the norm, but precision timing access appears to be very useful in some hard real-time contexts given current hw/sw realities. [1] https://en.wikipedia.org/wiki/Precision_Time_Protocol https://en.wikipedia.org/wiki/Precision_Time_Protocol
- naasking 9y agoWouldn't this involve modifying the driver then?
- jeff_marshall 9y agoYes, as well as some lower-level software. (sharing portions of a PCI BAR region isn't trivial in a hypervisor, when multiple guests are involved). Mixing safety criticality levels is hard.
- rdtsc 9y ago> What applications can you imagine would make so many clock calls so as to incur noticeable overhead? Low latency processing or realtime-ish applications. If you have a budget of a few milliseconds only, lots of gettimeofday can start to add up. Now true that nowadays they are mostly vDSO so it doesn't matter as much. I remember before they were we saw a decent speedup when we upgraded to vDSO kernel version. > But to answer your question directly, the ability to access the clock in a shared memory segment can itself be reified as a handle that's granted to a process. rdtsc is not shared memory but more like a register read. Though it is a virtualized instruction so on proper VMs (not a container) it is possible to control what the guest sees as the value.
- naasking 9y agoIndeed, but then access to a realtime clock factory should be reified as a handle. You would have to be given this handle and explicitly invoke it to install access to the clock if that's needed. The point being, a handle should be involved at some point in order to make the access control explicit and not implicit and ambient.
- vardump 9y agoNUMA systems might take 1 microsecond just to retrieve time internally from a shared hardware resource. The operation is not concurrent, so there's a chance for it to block for much longer. This can happen because RDTSC is not synchronized between physical CPU sockets (NUMA regions).