5 ms·

For a nice dose of doom and gloom, I quite like the dark silicon paper[1], which explores the limited use we'll get out of Moore's Law, even if it manages to co

For a nice dose of doom and gloom, I quite like the dark silicon paper[1], which explores the limited use we'll get out of Moore's Law, even if it manages to continue (and as far as I understand it, transistors/$ has already flat lined so it's at least temporarily over). Since we're no longer getting good power scaling out of shrinking, if Moore's Law keeps up, we're just essentially getting price discounts. A nice thought exercise for me is what would computing look like if you could fab wafers for free, today. Sort of the ad absurdum take on Moore's Law continuing. We'd have more memory, and a ridiculous amount of flash storage, and high end graphics cards would become cheaper (but not faster). Desktop CPUs might look more like server CPUs, but single core performance wouldn't change one bit. We'd probably see heterogeneous computing a whole lot more. Xeon Phi like processors on package next to Haswell cores at the very least. Probably computers would start to have FPGAs in them, as well as large amounts of niche circuitry to compute hashes, encode videos. We might see computation embedded inside memory (I think Hynix did this recently but I couldn't find a link). Maybe memory that accelerates garbage collection in order to accelerate modern workloads. So there are some interesting places to go, especially from a Hacker News perspective, but even if transistors became free, we still wouldn't see the rate of progress we did in the 80s or 90s (in terms of speed up to the tasks we're doing). Of course, this is all near to medium term stuff. I personally believe we'll see that rate of progress again when we move off of silicon, to a different computing paradigm, or (most likely) both - I can't believe we'll inch our way to science fiction computing, or that science fiction computing won't be possible at all. [1] Dark Silicon and the End of Multicore Scaling (ftp://ftp.cs.utexas.edu/pub/dburger/papers/ISCA11.pdf)

gizmo686 13y ago
I wouldn't underestimate our ability to through raw power at a problem. Imagine if we can give each process its own cpu/ram (with some mechanism of accessing a segment of another process's ram if given permision). Not only would this be good for security, responsiveness, general parallelism, but it would also reduce a lot of the complexity we have in emulating concurrency in software. Of course this also assumes that power is free.
- TazeTSchnitzel 13y ago
  Did you mean to throw raw power at a problem?
sliverstorm 13y ago
Oh, if transistors became free we could realize a huge boost in performance. We could basically take the cream of the crop and use only 9-sigma parts. (The range of part quality is pretty wide, cost/rarity is the challenge) We could also abandon yield-oriented design constraints, and a few other things. Now this probably isn't something that would keep doubling every year, but there would be an initial large bump.
- reitzensteinm 13y ago
  Not so fast! I must object to the loophole you claim to have found. I said the wafers would be free; testing ~100 trillion of them to find the best costs extra :) I never even thought of that angle - do you know how much better such a part would realistically be? I thought binning was mostly a matter of defect reduction (where some defects are slight). But at that extreme level, it could be a different game altogether.
  - marcosdumay 13y ago
    People keep demonstrating 1THz transistors on the lab. So, I'd say that such part would get up to a around 100GHz clock (a few switchings per pipeline stage), with our usual architectures. And it would use an insane amount of power.
ggreer 13y ago
Thanks for linking to that paper. It was interesting reading, especially since it makes predictions for 32nm and 22nm nodes. I wanted to judge the accuracy of those predictions, but I think I found an issue with their methodology. Figure 2(b) shows SPEC CPU2006 scores for then-current 45nm CPUs. For example, the Xeon X3440 is a Nehalem microarchitecture quad-core CPU with a 95W TDP.[1] It seems to benchmark around 30 points.[2] That means it's the lower-left of the two gray triangles in figure 2(b). The upper-right triangle is an Extreme Edition i7. Figures 2(e) and 2(f) show ITRS and conservative predictions for then-future feature sizes. The SPEC CPU2006 benchmark is single-threaded, making it hard to calculate performance-per-watt for multi-core CPUs. The researchers knew this. In the paper, they mention budgeting power for caches, then they drop this bombshell: > In the case of a multicore CPU, the remainder of the chip power budget is divided by the number of cores, resulting in the power budget allocated to a single core. That's a mistake. In a single-core benchmark, one core will run faster than base frequency, burning much more than its typical fraction of TDP. Some CPUs will run a single core at twice the base frequency. To pick a ridiculous example: The Xeon E5-2695 v2 has 12 cores and a 115W TDP.[3] It gets 55 points in the SPEC benchmark.[4] After doing the math for subtracting caches and dividing remaining TDP by the number of cores, the performance per core is even better than the ITRS estimates for 22nm. Comparing their predictions with modern quad-core CPUs, it does look like the numbers are closer to their conservative estimates than the ITRS's. Still, that oversight is pretty bad, and it makes me wonder what other mistakes are in the paper. 1. http://ark.intel.com/products/42928/Intel-Xeon-Processor-X3440-(8M-Cache-2_53-GHz) http://ark.intel.com/products/42928/Intel-Xeon-Processor-X34... 2. http://www.spec.org/cgi-bin/osgresults?conf=cpu2006&op=fetch&field=CPU&pattern=X3440 http://www.spec.org/cgi-bin/osgresults?conf=cpu2006&op=fetch... 3. http://ark.intel.com/products/75281/Intel-Xeon-Processor-E5-2695-v2-30M-Cache-2_40-GHz http://ark.intel.com/products/75281/Intel-Xeon-Processor-E5-... 4. http://spec.org/cgi-bin/osgresults?conf=cpu2006&op=fetch&field=CPU&pattern=E5-2695%20v2 http://spec.org/cgi-bin/osgresults?conf=cpu2006&op=fetch&fie...
kev009 13y ago
Dark silicon or even CPUs in general are orthogonal to Moore's law. There are other domains that can eat up as many and as small of transistors are you can throw at them. FPGAs are a great example. New fabrication technology will allow for new paradigms that we perhaps cannot sufficiently even predict right now.
jodrellblank 13y ago
How much of "the tasks we're doing" are repetetive? Someone in the 4TB SSD discussion said MicroSD cards at the moment could fit 50TB of storage in a standard laptop hard drive form factor - if anyone would pay for it. If transistors became free, how much could they be used for not-computation/storage? How much would your cheaper "(but not faster)" video card be able to get a speed boost by having terabytes of pre-computed things in cache? e.g. the difference between brute force password cracking and rainbow tables, or basic arithmetic and log tables.

gizmo686 13y ago

I wouldn't underestimate our ability to through raw power at a problem. Imagine if we can give each process its own cpu/ram (with some mechanism of accessing a segment of another process's ram if given permision). Not only would this be good for security, responsiveness, general parallelism, but it would also reduce a lot of the complexity we have in emulating concurrency in software. Of course this also assumes that power is free.

TazeTSchnitzel 13y ago
Did you mean to throw raw power at a problem?

sliverstorm 13y ago

Oh, if transistors became free we could realize a huge boost in performance. We could basically take the cream of the crop and use only 9-sigma parts. (The range of part quality is pretty wide, cost/rarity is the challenge) We could also abandon yield-oriented design constraints, and a few other things. Now this probably isn't something that would keep doubling every year, but there would be an initial large bump.

reitzensteinm 13y ago
Not so fast! I must object to the loophole you claim to have found. I said the wafers would be free; testing ~100 trillion of them to find the best costs extra :) I never even thought of that angle - do you know how much better such a part would realistically be? I thought binning was mostly a matter of defect reduction (where some defects are slight). But at that extreme level, it could be a different game altogether.
- marcosdumay 13y ago
  People keep demonstrating 1THz transistors on the lab. So, I'd say that such part would get up to a around 100GHz clock (a few switchings per pipeline stage), with our usual architectures. And it would use an insane amount of power.

ggreer 13y ago

Thanks for linking to that paper. It was interesting reading, especially since it makes predictions for 32nm and 22nm nodes. I wanted to judge the accuracy of those predictions, but I think I found an issue with their methodology. Figure 2(b) shows SPEC CPU2006 scores for then-current 45nm CPUs. For example, the Xeon X3440 is a Nehalem microarchitecture quad-core CPU with a 95W TDP.[1] It seems to benchmark around 30 points.[2] That means it's the lower-left of the two gray triangles in figure 2(b). The upper-right triangle is an Extreme Edition i7. Figures 2(e) and 2(f) show ITRS and conservative predictions for then-future feature sizes. The SPEC CPU2006 benchmark is single-threaded, making it hard to calculate performance-per-watt for multi-core CPUs. The researchers knew this. In the paper, they mention budgeting power for caches, then they drop this bombshell: > In the case of a multicore CPU, the remainder of the chip power budget is divided by the number of cores, resulting in the power budget allocated to a single core. That's a mistake. In a single-core benchmark, one core will run faster than base frequency, burning much more than its typical fraction of TDP. Some CPUs will run a single core at twice the base frequency. To pick a ridiculous example: The Xeon E5-2695 v2 has 12 cores and a 115W TDP.[3] It gets 55 points in the SPEC benchmark.[4] After doing the math for subtracting caches and dividing remaining TDP by the number of cores, the performance per core is even better than the ITRS estimates for 22nm. Comparing their predictions with modern quad-core CPUs, it does look like the numbers are closer to their conservative estimates than the ITRS's. Still, that oversight is pretty bad, and it makes me wonder what other mistakes are in the paper. 1. http://ark.intel.com/products/42928/Intel-Xeon-Processor-X3440-(8M-Cache-2_53-GHz) http://ark.intel.com/products/42928/Intel-Xeon-Processor-X34... 2. http://www.spec.org/cgi-bin/osgresults?conf=cpu2006&op=fetch&field=CPU&pattern=X3440 http://www.spec.org/cgi-bin/osgresults?conf=cpu2006&op=fetch... 3. http://ark.intel.com/products/75281/Intel-Xeon-Processor-E5-2695-v2-30M-Cache-2_40-GHz http://ark.intel.com/products/75281/Intel-Xeon-Processor-E5-... 4. http://spec.org/cgi-bin/osgresults?conf=cpu2006&op=fetch&field=CPU&pattern=E5-2695%20v2 http://spec.org/cgi-bin/osgresults?conf=cpu2006&op=fetch&fie...

kev009 13y ago

Dark silicon or even CPUs in general are orthogonal to Moore's law. There are other domains that can eat up as many and as small of transistors are you can throw at them. FPGAs are a great example. New fabrication technology will allow for new paradigms that we perhaps cannot sufficiently even predict right now.

jodrellblank 13y ago

How much of "the tasks we're doing" are repetetive? Someone in the 4TB SSD discussion said MicroSD cards at the moment could fit 50TB of storage in a standard laptop hard drive form factor - if anyone would pay for it. If transistors became free, how much could they be used for not-computation/storage? How much would your cheaper "(but not faster)" video card be able to get a speed boost by having terabytes of pre-computed things in cache? e.g. the difference between brute force password cracking and rainbow tables, or basic arithmetic and log tables.