3 ms·
There's a limit to what you can do without "simple" scaling, meaning, without making cheaper, faster, smaller individual gates. That is, an architecture optimiz
by _yosefk 11y ago
There's a limit to what you can do without "simple" scaling, meaning, without making cheaper, faster, smaller individual gates. That is, an architecture optimized for a given problem domain will give you maybe 1000x efficiency improvement over a more general architecture, but that's it. (Typically it will be <1000x, but let's say you're lucky; the point is, that's your last 1000x, and then you'll see no improvements, ever.)
So if Moore's law is indeed over, progress will be slow, domain-specific, and decelerating due to diminishing returns (specializing for a problem domain is worth it as long as that domain is not too narrow but at some point it's too narrow to justify the investment into building specialized hardware for it; you'll be better off using something less specialized. GPGPU, which does not provide amazing performance in an absolute sense, but does beat CPUs on a large range of problems, is an example to this - any accelerator more efficient than GPGPU needs to be justifiable in the sense that GPGPU is already there wherever there's a need for GPU doing graphics, which is where GPUs do provide amazing performance in an absolute sense [you can totally beat GPGPU with more specialized hardware on most benchmarks, while you can't beat GPUs in graphics.])
I'm a chip & accelerator architect, so it's not like I'm particularly happy about this, I think I'm realistic though. A higher-caliber architect saying the same pessimistic thing is Bob Colwell.
The one nice thing about really stopping at some manufacturing technology and not being able to improve any further is that the cost of using this technology will likely continue dropping for some years. Only when it reaches the bottom will progress have truly stopped.
- Jyaif 11y ago"it's too narrow to justify the investment into building specialized hardware for it" FYI, if the only way to improve perf is to build custom hw, the cost and complexity of building custom hw will drop.
- _yosefk 11y agoErm... Not necessarily. The only way to travel quickly between 2 cities is a direct flight, but there isn't a direct flight between any 2 cities, because the cost of having such a flight didn't yet drop to the point where it's economical. So it will be with many types of custom hw.
- adrianN 11y agoThe human brain needs about twenty watts, fits in a compact package and outperforms computers on a variety of tasks. Squishy wetware is pretty far from computronium. I think we still have a long way to go before we hit fundamental limits.
- ewzimm 11y agoInteresting analysis, thanks. What do you think about the development of more radical redesigns like quanum computers, memristors, or anything else that might show promise to displace the traditional CPU? It seems that there's a lot going on in the more experimental side of computing right now that is reaching enough maturity where experiments might become practical products within the next decade. Even if we don't have pocket-sized quantum computers, the existence of cloud computing might make something like quantum annealing practical in the near future. Is there anything that has you excited?
- _yosefk 11y agoDisplacing CPUs is not the issue (the CPU occupies less than half the area of a modern application processor chip for instance), the problem is displacing CMOS. I don't know much about the alternatives, but I trust Colwell's skepticism.