4 ms·

Really? We are good at nanoseconds and milliseconds but not at microseconds? Last I checked a microsecond was 1000 nanoseconds so you can't really be good at

by YZF 10y ago

Really? We are good at nanoseconds and milliseconds but not at microseconds? Last I checked a microsecond was 1000 nanoseconds so you can't really be good at nanoseconds but somehow bad at microseconds. This ties in a little to the recent HN discussions about the cost of a context switch. I think what they're trying to say, and not very well, is that there is somewhat of a discontinuity when you move between different levels of abstraction. There's examples of this phenomena in operating systems where the overhead of making a system call can be so high you can't get the latency down or in programming languages, e.g. running over a virtual machine or an interpreter. But this is far from new and there's a continuum of solutions from hardware like DSPs through real time operating systems, lightweight threads, lower lever languages, kernel bypass. Abstractions have cost, you want protected/virtual memory there's a cost and you pay that cost in your context switches. Not sure you can have your cake and eat it here but there's plenty of different choices on the menu for different situations.

dom0 10y ago
As I've said before, a context switch is really cheap on modern hardware. What kills it if you need to drag all sorts of dirt through all the caches. I.e. cold caches kill performance. Hardly news.
- jacquesm 10y ago
  A context switch is only cheap if the task you switch to doesn't do anything. Even 2 decades ago you could to upwards of 100K task switches per second on relatively anemic hardware. That's never been the problem.
  - dom0 10y ago
    In terms of cycles the switch has become more expensive since a lot of registers were added that need saving/restoring. (But since frequencies have increased a lot the absolute time shrank when we reached the "3 GHz era"; I think it's been pretty much constant for the last ~10 years or so). But you are right that the switch itself on modern-era (=32+ bit few-chip systems with MMU) hardware was never really that expensive (on the order of 10-20 us).
- Dylan16807 10y ago
  Thousands of instructions is not "really cheap". You can cause a lot of slowdown by accidentally calling into the kernel in a tight loop, even if L1 is big enough for both. A different CPU design could make context switches essentially free.
  - deleted 10y ago
    [deleted]
caf 10y ago
This is probably the best summary paragraph from the article that answers your question: Techniques optimized for nanosecond or millisecond time scales do not scale well for this microsecond regime. Superscalar out-of-order execution, branch prediction, prefetching, simultaneous multithreading, and other techniques for nanosecond time scales do not scale well to the microsecond regime; system designers do not have enough instruction-level parallelism or hardware-managed thread contexts to hide the longer latencies. Likewise, software techniques to tolerate millisecond-scale latencies (such as software-directed context switching) scale poorly down to microseconds; the overheads in these techniques often equal or exceed the latency of the I/O device itself. To expand a bit: the reason you can be good at nanoseconds but bad at microseconds is that you need to throw a thousand times more resources at the problem to solve it using the same techniques. You can get good instruction-level parallelism with current memory latencies using a reorder buffer with a hundred or so entries, but can you really just scale that up to a ROB with 100,000 entries?
- YZF 10y ago
  What's the problem that you're trying to solve? OOE, branch prediction, prefetching, are to allow a CPU to execute multiple instructions per cycle on a GHz clock. I.e. within a nanosecond. Memory latency all the way out to DRAM is more than enough to do things in microseconds and the caches are big enough to get you a lot of concurrency within that. It's true that when you're coding over a modern OS in a high level language you can't get e.g. a GPIO pin to update in a microsecond. But that's life. Use a real-time OS you can do better. Get rid of the OS and you'll do even better. Use a different CPU or an FGPA and you may do better. The various techniques used to get a CPU to execute 20 instructions in a nanosecond aren't the limiting factor in that stack. EDIT: Once upon a time there was a UART called the 8250. You could either poll it or if you wanted to do something else concurrently you'd use interrupts. Only interrupts have to do a context switch (in this case pushing a few things onto the stack) so they cost a little more. Once serial rates became high enough interrupts started consuming more and more cycles. So they added a buffer, and we got the 16550 (or one of its cousins). Now the thing would only interrupt above a certain watermark and we can cut the number of interrupts by 10 so we can get higher throughput at the expense of higher latency. Same thing happens with network interfaces. You can drive a 10Gbps network because a lot of stuff is buffered and/or hardware offloaded. It doesn't make sense to bitbang 10G Ethernet out of a general purpose CPU. These sorts of tradeoffs in different parts of the system are unavoidable but yet the tools to solve problems down to GHz clocks are out there both for software and hardware.

by YZF 10y ago

dom0 10y ago

As I've said before, a context switch is really cheap on modern hardware. What kills it if you need to drag all sorts of dirt through all the caches. I.e. cold caches kill performance. Hardly news.

jacquesm 10y ago
A context switch is only cheap if the task you switch to doesn't do anything. Even 2 decades ago you could to upwards of 100K task switches per second on relatively anemic hardware. That's never been the problem.
- dom0 10y ago
  In terms of cycles the switch has become more expensive since a lot of registers were added that need saving/restoring. (But since frequencies have increased a lot the absolute time shrank when we reached the "3 GHz era"; I think it's been pretty much constant for the last ~10 years or so). But you are right that the switch itself on modern-era (=32+ bit few-chip systems with MMU) hardware was never really that expensive (on the order of 10-20 us).
Dylan16807 10y ago
Thousands of instructions is not "really cheap". You can cause a lot of slowdown by accidentally calling into the kernel in a tight loop, even if L1 is big enough for both. A different CPU design could make context switches essentially free.
- deleted 10y ago
  [deleted]

caf 10y ago

This is probably the best summary paragraph from the article that answers your question: Techniques optimized for nanosecond or millisecond time scales do not scale well for this microsecond regime. Superscalar out-of-order execution, branch prediction, prefetching, simultaneous multithreading, and other techniques for nanosecond time scales do not scale well to the microsecond regime; system designers do not have enough instruction-level parallelism or hardware-managed thread contexts to hide the longer latencies. Likewise, software techniques to tolerate millisecond-scale latencies (such as software-directed context switching) scale poorly down to microseconds; the overheads in these techniques often equal or exceed the latency of the I/O device itself. To expand a bit: the reason you can be good at nanoseconds but bad at microseconds is that you need to throw a thousand times more resources at the problem to solve it using the same techniques. You can get good instruction-level parallelism with current memory latencies using a reorder buffer with a hundred or so entries, but can you really just scale that up to a ROB with 100,000 entries?

YZF 10y ago
What's the problem that you're trying to solve? OOE, branch prediction, prefetching, are to allow a CPU to execute multiple instructions per cycle on a GHz clock. I.e. within a nanosecond. Memory latency all the way out to DRAM is more than enough to do things in microseconds and the caches are big enough to get you a lot of concurrency within that. It's true that when you're coding over a modern OS in a high level language you can't get e.g. a GPIO pin to update in a microsecond. But that's life. Use a real-time OS you can do better. Get rid of the OS and you'll do even better. Use a different CPU or an FGPA and you may do better. The various techniques used to get a CPU to execute 20 instructions in a nanosecond aren't the limiting factor in that stack. EDIT: Once upon a time there was a UART called the 8250. You could either poll it or if you wanted to do something else concurrently you'd use interrupts. Only interrupts have to do a context switch (in this case pushing a few things onto the stack) so they cost a little more. Once serial rates became high enough interrupts started consuming more and more cycles. So they added a buffer, and we got the 16550 (or one of its cousins). Now the thing would only interrupt above a certain watermark and we can cut the number of interrupts by 10 so we can get higher throughput at the expense of higher latency. Same thing happens with network interfaces. You can drive a 10Gbps network because a lot of stuff is buffered and/or hardware offloaded. It doesn't make sense to bitbang 10G Ethernet out of a general purpose CPU. These sorts of tradeoffs in different parts of the system are unavoidable but yet the tools to solve problems down to GHz clocks are out there both for software and hardware.