16 ms·
What are the forces in chip design that are at play here? Over the last 10-15 years, fabs have continued to fit more and more logic gates per unit area, but hav
by abainbridge 6y ago
What are the forces in chip design that are at play here? Over the last 10-15 years, fabs have continued to fit more and more logic gates per unit area, but haven't reduced the power consumption per gate as much. As a result, if you fill your modern chip with compute gates, you cannot use them all at once because the chip will melt. Or at least you can't have them all running at max clock rates. One solution is to increase the proportion of the chip used for SRAM (it uses less power per unit area than compute gates), this is what Graphcore have done. Another is to put down multiple different compute blocks, each designed for a different purpose, and only use them a-few-at-a-time. The big-little Arm designs in smartphones are an example of that. But I feel like AVX512 might be an example too. When they add ML accelerator blocks next, they also will not be able to be used flat out at the same time as the rest of the cores' resources.
I'm sure Intel should fix the problems Linus is complaining about, but I feel like chip vendors are being forced into this "add special purpose blocks" approach, as the only way to make their new chips better than their old ones.
- pjmlp 6y agoThe main problem is software, with GPGPUs you need to explicitly program for them, while with stuff like AVX there is this implicit hope that you just code as always and the compiler will take care of the rest via auto-vectorization and PhD level optimization algorithms. Because outside artificial intelligence, graphics and audio, there is little else that common applications would use the GPGPU for, so the large majority of software developers keeps ignoring heterogeneous programming models.
- zozbot234 6y ago> Because outside artificial intelligence, graphics and audio, there is little else that common applications would use the GPGPU for, so the large majority of software developers keeps ignoring heterogeneous programming models. I think you got this backwards - the lack of developers' interest is what leads to the mistaken impression that GPU compute is only good for multimedia and FP-crunching workloads. Even looking at the success of GPU compute in mining cryptocoins (only ASIC's do better) ought to be enough to tell you that we could do a lot more with them if we cared to.
- pjmlp 6y agoFrom my point of view cryptomining is a useless fad, and typical line of business applications don't need anything more than what I listed.
- teruakohatu 6y ago> the compiler will take care of the rest via auto-vectorization and PhD level optimization algorithms. With how AVX512 is implemented, there isn't much point in a compiler auto optimizing general purpose code to use it, because even if there is a theoretical speedup, it may well be slower in practice.
- pjmlp 6y agoThere might not be one, but all major C, C++, Hotspot, Graal and RyuJIT compilers do it to some extent.
- viraptor 6y agoIn my ideal world you'd be able to mark a function "this should compile to / run on gpgpu" and the compiler would potentially tell you why it can't do that. I'm not even sure if anything is stopping us apart from implementing that apart from the effort required. Sure, many ways to write that code will result in terrible performance, but it would still be closer to the auto-vectorisation experience. Actually we already have openmp to cuda (http://www2.engr.arizona.edu/~ece569a/Readings/GPU_Papers/3.OpenMPtoGPU.pdf http://www2.engr.arizona.edu/~ece569a/Readings/GPU_Papers/3....) so just making it more production-ready would be perfect.
- gnufx 6y agoThe current OpenMP spec has GPU offload features specifically for what was expected of the Sierra supercomputer. I'm not sure how relevant a paper that old (relatively, I hasten to add) is.
- throwaway_pdp09 6y ago> while with stuff like AVX there is this implicit hope that you just code as always and the compiler will take care of the rest via auto-vectorization and PhD level optimization algorithms No. I recently could really, really have used the packed saturated integer arithmetic and horizontal addition in AVX2 (but my old machine doesn't support it) and even better, the same but 512 bits wide on AVX512. It would only have been 6 or 7 instructions, if that, but it was inner loop, and mattered. Using compiler intrinsics would have been fine. I think you're looking at things too narrowly.
- pjmlp 6y agoI am looking at it of the point of view of joe/jane developer that cannot tell head from tail regarding vector programming and doesn't even know what compiler intrinsics are for, and use languages that don't expose them anyway.
- voldacar 6y agoWell those people will never be getting the most out of their CPUs to begin with.
- pjmlp 6y agoWhich is the whole point of "this implicit hope that you just code as always and the compiler will take care of the rest via auto-vectorization and PhD level optimization algorithms.", because not only do those people not get it, there is a general decline in using languages that expose vector intrisics like C and C++ for regular LOB applications.
- gridlockd 6y agoAVX is not a "special purpose block", it's Intel's answer to not adding special purpose blocks on customer demand, like you can do with ARM. Crypto or video decoding comes to mind, those would be much faster with dedicated silicon, but more general AVX instructions can get you halfway there. Well, maybe a quarter. People point out that AVX uses a lot of power, but they ignore that the same algorithm running instead on more but simpler cores would use even more power.
- throwaway_pdp09 6y ago> but more general AVX instructions can get you halfway there Maybe misunderstand you but there are some fairly non-general ops for encoding/decoding crypto https://en.wikipedia.org/wiki/AVX-512#VAES https://en.wikipedia.org/wiki/AVX-512#VAES
- ArnoVW 6y agoThey exist today, but they were added after AVX. Every year we figure out how to cram more transistors on a cubic cm, and once the low hanging fruit was added and we knew how to add more transistors, we decided to start putting more and more specific functions. That is the point of Linus. He would have preferred to use that increase in transistor count for other things, like more cache.
- zozbot234 6y agoMore cache has diminishing returns, because cache wants to be as close as possible to the core logic. And modern CPU's are mostly cache anyway. Special-purpose blocks for common compute tasks are quite cheap.
- xxs 6y ago>And modern CPU's are mostly cache anyway. Skylake is less than 30% cache. However internally it's 512bus, thanks to avx-512 - which could be considered suboptimal.
- 6y ago
- tails4e 6y agoJim Keller had an interesting talk recently [1] about ways of doing parallel processing to better us the billions of transistors we have - assuming the task is parallelizable. There's the scalar core (i.e the basic CPU) which is easy to program realtively. Then a scalar core with vector instructions - difficult to program efficiently. Then there are arrays of scalar cores, i.e. GPUs, so relatively easy to program again, and now a lot of startups with arrays of scalar cores each with vector engines, so expected to be most difficult to program. He didn't go into why vector instructions are hard to use efficiently, and hard for compiler writers, but I'd be interested if anyone here could explain that. 1. https://youtu.be/8eT1jaHmlx8 https://youtu.be/8eT1jaHmlx8
- zozbot234 6y agoAre GPU's really easier to program than scalar w/ SIMD (or vector insns)? The programming models you have to work with for GPGPU seem quite obscure, whereas with CPU and SIMD flipping a compiler switch gets you most of the way there, and self-contained intrinsics do the rest.
- TinkersW 6y agoGPU programming is easy enough, the complexity comes from the seperate memory system and the tedious(and not portable) API you need to use to access the GPU. I prefer intrinsics as they give more control than shader languages and they can be written in C++ instead of fiddling with some garbage GPU API that runs async.
- pjmlp 6y agoMSL, CUDA and SYSCL are C++ with extra topping. Also one of the reasons CUDA won developer love is that it fully embraced polyglot programming on the GPU.
- TinkersW 6y agoNone of those are both portable and widely available on end user machines, which is needed for games CUDA seems nice, but being Nvidia only makes it a total dead end.
- michaelt 6y ago> Over the last 10-15 years, fabs have continued to fit more and more logic gates per unit area, but haven't reduced the power consumption per gate as much. Kids these days get 8 cores for a 100W TDP. When I was a boy, 100W got you a single core. And you didn't get dynamic frequency scaling, so it'd be putting out that heat all the time. (We also had to walk to school barefoot in the snow, uphill both ways)
- noisem4ker 6y ago>putting out that heat all the time Even if the frequency was fixed, dissipated heat did definitely vary together with the computing load.
- msh 6y agoCould you not use TDP to melt the snow ;)
- em500 6y agoYou must be young. Home PC CPUs from my youth drew only single digit watts. They didn't require any fan until the Pentium.
- acqq 6y agoIndeed: 386, introduced 1985: http://www.cpu-world.com/CPUs/80386/Intel-A80386-16.html http://www.cpu-world.com/CPUs/80386/Intel-A80386-16.html Typical/Maximum power dissipation: 1.85 Watt / 2.3 Watt And even no Pentium III 1999-2003 needed more than around 30 W: https://en.wikipedia.org/wiki/List_of_Intel_Pentium_III_microprocessors https://en.wikipedia.org/wiki/List_of_Intel_Pentium_III_micr...
- karmakaze 6y agoThe Pentium II was not as efficient as the III. I remember setting up a dual socket machine where the PS started to matter. The best thing was that the web browser would only suck 100% from one processor.
- 6y ago
- ksec 6y ago>but haven't reduced the power consumption per gate as much. That is simply not true. You can run the 64 Core on EPYC 2 all at once at 3Ghz all with Air Cooling. At every node they have reduced power consumption that is also one reason you see continuous performance improvement.
- abainbridge 6y ago> That is simply not true. I'm not claiming anything controversial. Power not having scaled as well as area recently is often referred to as the end of Dennard scaling: https://en.wikipedia.org/wiki/Dennard_scaling#Breakdown_of_Dennard_scaling_around_2006 https://en.wikipedia.org/wiki/Dennard_scaling#Breakdown_of_D... > You can run the 64 Core on EPYC 2 all at once at 3Ghz all with Air Cooling. That can be true despite the fact that power hasn't scaled as well as area. > At every node they have reduced power consumption Yep, just not as much as they improved area.
- bsder 6y ago> What are the forces in chip design that are at play here? The "weak form" of Moore's Law--"Performance doubles every 12-18 months"--is dead and buried. The "strong form" of Moore's Law is still active--"Transistor cost halves every 12-18 months". This means that you can't make the primary paths any faster. So, all you can do is add functionality and pray that someone magically can make that functionality relevant to the primary use cases.