3 ms·

Additionally, his premise that CPUs will continue to get more and more cores is probably also flawed. The diminishing returns of the extra cores due to Amdahl's

by wwwtyro 14y ago

Additionally, his premise that CPUs will continue to get more and more cores is probably also flawed. The diminishing returns of the extra cores due to Amdahl's law will make them less and less economically viable.

bunderbunder 14y ago
And with programs for which Amdahl's Law isn't a huge concern, multiprocessing isn't necessarily even the most appropriate form of parallelism. Multiple cores is great for situations where the sequential fraction of a program is relatively high, because it helps minimize the cost of synchronization. But it can also bring some problems that might limit performance, such as bottlenecks on shared resources such as memory and I/O. If the problem can divided into fairly large and independent chunks of work, on the other hand, then distributed computing becomes a lot more attractive.
marshray 14y ago
Intel and AMD want to continue making processors that cost more than $150. If feature size continues to shrink, what else are they going to do with the extra room on the chip? (other than slash their own prices using ever-more-expensive fab equipment) In single threaded code, the unused cores can be powered off.
- rpledge 14y ago
  Larger caches is an option for using the extra die area. RAM access times aren't increasing at anywhere near the rate that CPU power has.
  - Joeri 14y ago
    Why have caches at all? Couldn't you move the entire RAM on-die.
    - sliverstorm 14y ago
      First, SRAM caches are still faster than DRAM. Second, DRAM is made differently from CPUs, and it is difficult to use two different processes on the same silicon. Third, squeezing 4GB of DRAM onto a single CPU die? How?
    - JohnBooty 14y ago
      It would be prohibitively expensive. Several megabytes of cache already take up about half of the CPU die size. http://techreport.com/review/21987/intel-core-i7-3960x-processor http://techreport.com/review/21987/intel-core-i7-3960x-proce... Think about how much larger a CPU would have to be to include an entire gigabyte of cache, much less multiple gigabytes.
  - mzl 14y ago
    Recently I heard a talk by a guy from Oracle designing Sparc CPUs. IIRC, he said that while they could potentially put bigger caches on their CPUs, for their test-cases it wasn't that much of a benefit above their chosen size. Another interesting note from that talk was that they had spent a large amount of time making an 8 socket architecture with quick-connect paths between all pairs of sockets for memory transport. Unfortunately that was not as important as believed either, since the programs that typically benefit from 8 CPUs normally have good memory properties anyway. Not sure if either data-points are true, but interesting nonetheless.
chubot 14y ago
That would only be true if systems are only capable of running a single program. Luckily I think there will be utility in running 10,000 processes on a box at the same time.

by wwwtyro 14y ago

bunderbunder 14y ago

And with programs for which Amdahl's Law isn't a huge concern, multiprocessing isn't necessarily even the most appropriate form of parallelism. Multiple cores is great for situations where the sequential fraction of a program is relatively high, because it helps minimize the cost of synchronization. But it can also bring some problems that might limit performance, such as bottlenecks on shared resources such as memory and I/O. If the problem can divided into fairly large and independent chunks of work, on the other hand, then distributed computing becomes a lot more attractive.

marshray 14y ago

Intel and AMD want to continue making processors that cost more than $150. If feature size continues to shrink, what else are they going to do with the extra room on the chip? (other than slash their own prices using ever-more-expensive fab equipment) In single threaded code, the unused cores can be powered off.

rpledge 14y ago
Larger caches is an option for using the extra die area. RAM access times aren't increasing at anywhere near the rate that CPU power has.
- Joeri 14y ago
  Why have caches at all? Couldn't you move the entire RAM on-die.
  - sliverstorm 14y ago
    First, SRAM caches are still faster than DRAM. Second, DRAM is made differently from CPUs, and it is difficult to use two different processes on the same silicon. Third, squeezing 4GB of DRAM onto a single CPU die? How?
  - JohnBooty 14y ago
    It would be prohibitively expensive. Several megabytes of cache already take up about half of the CPU die size. http://techreport.com/review/21987/intel-core-i7-3960x-processor http://techreport.com/review/21987/intel-core-i7-3960x-proce... Think about how much larger a CPU would have to be to include an entire gigabyte of cache, much less multiple gigabytes.
- mzl 14y ago
  Recently I heard a talk by a guy from Oracle designing Sparc CPUs. IIRC, he said that while they could potentially put bigger caches on their CPUs, for their test-cases it wasn't that much of a benefit above their chosen size. Another interesting note from that talk was that they had spent a large amount of time making an 8 socket architecture with quick-connect paths between all pairs of sockets for memory transport. Unfortunately that was not as important as believed either, since the programs that typically benefit from 8 CPUs normally have good memory properties anyway. Not sure if either data-points are true, but interesting nonetheless.

chubot 14y ago

That would only be true if systems are only capable of running a single program. Luckily I think there will be utility in running 10,000 processes on a box at the same time.