5 ms·

The Von Neumann architecture is dead, IMHO future computing architectures involve many cores, all with local memory (not just a bit of cache), Tensor computing

The Von Neumann architecture is dead, IMHO future computing architectures involve many cores, all with local memory (not just a bit of cache), Tensor computing capabilities and very high bandwidth connecting these cores. Intel has Xeon Phi, but what else are they doing? There is so much opportunity in new computing architectures but it seems like Intel has been taken by surprise. I hope this giant wakes up.

LgWoodenBadger 9y ago
Well, the core with its own memory is still a Von Neumann machine, is it not?
- adrianratnapala 9y ago
  More likely a Harvard machine, i.e. one where the instruction memory is distinct from the data memory. It is essentially what modern cores have with a their split caches, except those things are called "caches" because the architecture then has a lot of machinery for pretending to be a von Neumann Machine with a huge memory. This pretending is done because that's the kind of machine that software expects. But the first unices ran on machines with memories only as big as a modern L1-cache. Moreover, networking and parallelism are everyday issues for software now. It will be interesting to see if software evolves to explicitly recognise that it is really running on a network of small machines + a hierarchy of big external storage devices, of which DRAM is just one.
- visionscaper 9y ago
  Yes, true, Von Neumann is dead, long live Von Neuman! ;) The difference is of course that we should not see the whole cluster of cores as a Von Neumann machine, the memory bandwidth is just too low to drive multiple cores with some external memory, memory must be as close as possible to the cores themselves. Quote from wtallis lower in this sub-thread : > You can't pretend that memory is all equidistant from the processor cores unless you want everything running equally slow.
ww520 9y ago
NUMA style architecture has been existed for a long while now. It doesn't seem to take over the world.
- adrianratnapala 9y ago
  But the practical case for it should grow as long transistor counts increase while the speed of light doesn't.
  - wtallis 9y ago
    And NUMA-like architectures on a single CPU die have been increasingly common between Intel's multiple ring busses on larger Xeons and AMD's 4-core clusters on Ryzen. Even per-core L2 caches violate the assumption that a given memory address is equally accessible from any processor core. You can't pretend that memory is all equidistant from the processor cores unless you want everything running equally slow.
  - visionscaper 9y ago
    We already have practical use cases: training and evaluation of large neural networks.
- dom0 9y ago
  NUMA is used, when it is actually needed, which is not that often. Most other stuff gets by without requiring NUMA enlightenment (if you think competent parallel software developers are hard to come by... finding competent NUMA people will make you reconsider that assessment).

LgWoodenBadger 9y ago

Well, the core with its own memory is still a Von Neumann machine, is it not?

adrianratnapala 9y ago
More likely a Harvard machine, i.e. one where the instruction memory is distinct from the data memory. It is essentially what modern cores have with a their split caches, except those things are called "caches" because the architecture then has a lot of machinery for pretending to be a von Neumann Machine with a huge memory. This pretending is done because that's the kind of machine that software expects. But the first unices ran on machines with memories only as big as a modern L1-cache. Moreover, networking and parallelism are everyday issues for software now. It will be interesting to see if software evolves to explicitly recognise that it is really running on a network of small machines + a hierarchy of big external storage devices, of which DRAM is just one.
visionscaper 9y ago
Yes, true, Von Neumann is dead, long live Von Neuman! ;) The difference is of course that we should not see the whole cluster of cores as a Von Neumann machine, the memory bandwidth is just too low to drive multiple cores with some external memory, memory must be as close as possible to the cores themselves. Quote from wtallis lower in this sub-thread : > You can't pretend that memory is all equidistant from the processor cores unless you want everything running equally slow.

ww520 9y ago

NUMA style architecture has been existed for a long while now. It doesn't seem to take over the world.

adrianratnapala 9y ago
But the practical case for it should grow as long transistor counts increase while the speed of light doesn't.
- wtallis 9y ago
  And NUMA-like architectures on a single CPU die have been increasingly common between Intel's multiple ring busses on larger Xeons and AMD's 4-core clusters on Ryzen. Even per-core L2 caches violate the assumption that a given memory address is equally accessible from any processor core. You can't pretend that memory is all equidistant from the processor cores unless you want everything running equally slow.
- visionscaper 9y ago
  We already have practical use cases: training and evaluation of large neural networks.
dom0 9y ago
NUMA is used, when it is actually needed, which is not that often. Most other stuff gets by without requiring NUMA enlightenment (if you think competent parallel software developers are hard to come by... finding competent NUMA people will make you reconsider that assessment).