5 ms·
In practice it sometimes is the case, though. SMT/hyperthreading is complicated. If you have a workload dominated by non-local DRAM fetches, it's a huge win b
by ajross 8y ago
In practice it sometimes is the case, though.
SMT/hyperthreading is complicated. If you have a workload dominated by non-local DRAM fetches, it's a huge win because when the CPU pipeline is stalled on one thread it can still issue instructions from the other.
If you have a workload dominated by L1 cache bandwidth, the opposite is true because the threads compete for the same resource.
On balance, on typical workloads, it's a win. But there are real-world problems for which turning it off is a legitimate performance choice.
- greglindahl 8y agoTypical workloads? What's that? People run hugely diverse workloads on cpus, and they change over time.
- ajross 8y agoBuilding software, serving web pages, executing database queries, running a DOM layout, managing game logic... I mean, come on. You knew what I meant. Those are all tasks with "medium" cache residency and "occasional" stalls on DRAM. Anything that does a bunch of different things with a big-ish world of data. Conversely: finding a task that is L1-cache-bound but does not frequently have to stall for memory is much harder. The only ones off the top of my head are streaming tasks like software video decode.
- greglindahl 8y agoOh, you meant typical for you. One task that is L1 cache bound and does not frequently stall for memory (if you code it up well) is matrix multiply.
- kbenson 8y ago> Oh, you meant typical for you. I'm pretty sure those are meant to be, and I think are, "typical" for the general purpose CPU in use, and thus the general case. Both mobile and desktop CPUs will be doing DOM layout, DB queries (whether to SQLite or the registry or just the filesystem), and possibly computing game logic on a regular basis.
- greglindahl 8y agoIt's becoming popular to want to push machine learning tasks onto edge devices like mobile and desktop CPUs, for example apps that include some machine learning. Some of these machine learning algorithms do a lot of matrix multiplies. "Typical" is highly varied, and it changes. Edit: here's an example: Google brings on-device machine learning to mobile with TensorFlow Lite https://thenextweb.com/artificial-intelligence/2017/11/15/google-brings-on-device-machine-learning-to-mobile-with-tensorflow-lite/ https://thenextweb.com/artificial-intelligence/2017/11/15/go...
- kbenson 8y agoWould they be using mostly CPU for that, or would they offload it to the GPU or a dedicated chip? I would assume you would use your general purpose CPU only if all else wasn't available (and generally there's a GPU available on most end user devices these days).
- greglindahl 8y agoIf possible the GPU, but not all GPUs have either a library or enough documentation to write one. I’ve seen complaints about this issue on mobile GPUs for years, no idea how widespread it is now. BTW, this is just one example algorithm that I picked because it does (on the cpu) what the person I replied to said was rare.
- rbanffy 8y agoRunning the model is much easier than training it. On power-constrained environments, DSPs can do it.
- adrianN 8y agoI don't know whether it's still true, but a couple of years ago a majority of the world's CPU cycles were spent sorting things.
- smaddox 8y agoThat's an interesting claim. Do you remember the source?
- adrianratnapala 8y ago> workload dominated by non-local DRAM fetches, How often is that a polite way of saying "software that is inefficient"?
- yk 8y agoWith most software, well most software is pretty inefficent and profits from HT. However there are a lot of reasons for that, writing in something interpreted because it is faster to develop and the software does not need to be very efficient in the first place would be one application. (Not to say that all Python/JS/etc is inefficient, just that software that needs to be efficient is precisely the kind were one would consider an unmanaged language.) Additionally, things like webservers or dbs often just don't know which piece of data they need next, simply because they don't know the next query, have a tendency to profit from HT, even though the software is hardly known for being inefficient.
- paulirwin 8y agoFWIW, you mention databases, but even some database workloads can have better performance with HT turned off. I first learned this from a DBA at a former job when I was curious as to why they turned HT off. A member of the SQL Server team back in 2005 ran some experiments and found that you can get a 10% performance improvement in some workloads with HT off [1]. I don't know how much of that is still true today, however, as nearly all of my recent experience is PaaS in the cloud. [1]: https://blogs.msdn.microsoft.com/slavao/2005/11/12/be-aware-to-hyper-or-not-to-hyper/ https://blogs.msdn.microsoft.com/slavao/2005/11/12/be-aware-...
- imtringued 8y agoI'm not sure there is a correlation. I can think of many situations in which non-local DRAM fetches are more efficient and I can think of many other situations where the opposite is true. Trees or hashmaps which use non-local DRAM fetches can be more efficient than a brute force linear search through a continguous array given a sufficiently high enough number of elements. At the same time continguous arrays can be significantly more efficient than linked lists which use non-local DRAM fetches.