3 ms·

But once transistors can't get any smaller the only way to get more cores will be to have physically larger CPUs. That might work for a while. But, pretty quick

by smanek 17y ago

But once transistors can't get any smaller the only way to get more cores will be to have physically larger CPUs. That might work for a while. But, pretty quickly, those larger dies will use too much power and get too hot (not to mention be too physically large to fit in the form factors we want). At best that will buy us another few years of Moore-level growth. But you can't really have CPUs be more than 5 to 10 times larger than they currently are.

fauigerzigerk 17y ago
Moore's law isn't actually what's terribly important I think. What we want isn't transistor density but processing power. The human brain has a lot of processing power, uses very little energy and works with massive parallelism. But you may be right to question whether we can smoothly transition to some entirely new hardware paradigm quickly enough to double processing power every 18 months. In the long run I believe we can, but we may hit a temporary roadblock in the next few years.
- smanek 17y ago
  The human brain isn't all that powerful (hardware wise) ... The numbers I've seen suggest that it's total 'processing power' is only on the order of a few hundred teraflops - and we already have plenty of clusters bigger than that. I know that floating point operations per second is a crappy metric but consider that a neuron can only spike about 100 times per second (i.e., the brain runs at 100 Hz) - even billions of neurons put together just aren't that powerful in terms of hardware. Software, not hardware, is already the limiting reagent to 'smart' computers. Everyone says 'massive parallelism' is the answer. But once transistor density plateaus the only way to add more cores (i.e., add more parallelism) will be to make CPUs physically bigger (or make each core 'weaker'). Making each core 'weaker' adds very little net benefit (2 * 100 = 20 * 10) and having physically larger CPUs isn't really practical beyond a point ...
  - 8plot 17y ago
    I mostly agree with you, but I don't think an increase in cpu size will slow the pace down much; it will just shift the cpu load to the server side.
  - fauigerzigerk 17y ago
    If it is as you say, all the better. It means that increased processing power could be achieved by making better use of the parallel hardware we already have. Still, I contend that using parallelism is key in some shape or form. I cannot say much about transistor density issues as I'm not competent in that area. I read that IBM is experimenting with nano technology in order to create much more dense hardware that uses very little energy. I have no idea how far out that sort of approach is.
  - jonsen 17y ago
    The neuron runs at just 100 Hz, yes, but the brain is not a synchronously clocked machine. How much information processing capacity is due to the timing of spikes? How fine grained is its time sensitivity? 1ms? 1ns? 1µs? 1ps? ...
- jonsen 17y ago
  I agree. There seems to be no apparent technology mature enough to sustain hardware complexity growth. I think a pause would benefit software technology, in some sense allowing it to catch up on its potential.
jonsen 17y ago
.. you can't really have CPUs be more than 5 to 10 times larger than they currently are Why not? And if you can, it doesn't have to be located in your device considering improvements in network technology.
- swolchok 17y ago
  Well, basic physics -- it already takes nontrivial time for signals to cross the die. If you make the die bigger, you're probably just going to be using that to add more cores, and the time for the furthest-apart cores to communicate is going to grow.
  - joeyo 17y ago
    Transmission delays suck, but this is something that the brain has evolved to cope with (consider the speed of an action potential vs the speed of electrons in a CPU and the distances between neurons in a typical mammalian brain vs the distances between transistors in a CPU). So I think there there are strategies we could use to deal with larger silicon, but I do agree that there will always be an incentive to keep things as small as possible.

by smanek 17y ago

fauigerzigerk 17y ago

Moore's law isn't actually what's terribly important I think. What we want isn't transistor density but processing power. The human brain has a lot of processing power, uses very little energy and works with massive parallelism. But you may be right to question whether we can smoothly transition to some entirely new hardware paradigm quickly enough to double processing power every 18 months. In the long run I believe we can, but we may hit a temporary roadblock in the next few years.

smanek 17y ago
The human brain isn't all that powerful (hardware wise) ... The numbers I've seen suggest that it's total 'processing power' is only on the order of a few hundred teraflops - and we already have plenty of clusters bigger than that. I know that floating point operations per second is a crappy metric but consider that a neuron can only spike about 100 times per second (i.e., the brain runs at 100 Hz) - even billions of neurons put together just aren't that powerful in terms of hardware. Software, not hardware, is already the limiting reagent to 'smart' computers. Everyone says 'massive parallelism' is the answer. But once transistor density plateaus the only way to add more cores (i.e., add more parallelism) will be to make CPUs physically bigger (or make each core 'weaker'). Making each core 'weaker' adds very little net benefit (2 * 100 = 20 * 10) and having physically larger CPUs isn't really practical beyond a point ...
- 8plot 17y ago
  I mostly agree with you, but I don't think an increase in cpu size will slow the pace down much; it will just shift the cpu load to the server side.
- fauigerzigerk 17y ago
  If it is as you say, all the better. It means that increased processing power could be achieved by making better use of the parallel hardware we already have. Still, I contend that using parallelism is key in some shape or form. I cannot say much about transistor density issues as I'm not competent in that area. I read that IBM is experimenting with nano technology in order to create much more dense hardware that uses very little energy. I have no idea how far out that sort of approach is.
- jonsen 17y ago
  The neuron runs at just 100 Hz, yes, but the brain is not a synchronously clocked machine. How much information processing capacity is due to the timing of spikes? How fine grained is its time sensitivity? 1ms? 1ns? 1µs? 1ps? ...
jonsen 17y ago
I agree. There seems to be no apparent technology mature enough to sustain hardware complexity growth. I think a pause would benefit software technology, in some sense allowing it to catch up on its potential.

jonsen 17y ago

.. you can't really have CPUs be more than 5 to 10 times larger than they currently are Why not? And if you can, it doesn't have to be located in your device considering improvements in network technology.

swolchok 17y ago
Well, basic physics -- it already takes nontrivial time for signals to cross the die. If you make the die bigger, you're probably just going to be using that to add more cores, and the time for the furthest-apart cores to communicate is going to grow.
- joeyo 17y ago
  Transmission delays suck, but this is something that the brain has evolved to cope with (consider the speed of an action potential vs the speed of electrons in a CPU and the distances between neurons in a typical mammalian brain vs the distances between transistors in a CPU). So I think there there are strategies we could use to deal with larger silicon, but I do agree that there will always be an incentive to keep things as small as possible.