4 ms·

You've done a lot of interesting historical research there! I wanted to get into a discussion of "central processing unit", but decided my article was long enou

by kens 2y ago

You've done a lot of interesting historical research there! I wanted to get into a discussion of "central processing unit", but decided my article was long enough already :-) The term "central processing unit" is unusual since it is a seven-syllable term for a fundamental idea. Even "CPU" is a mouthful. I think that the "central" part is in opposition to systems such as ENIAC or the Harvard Mark I, where processing is spread out through the system through accumulators that each perform addition. Centralizing the processing was an important innovation by von Neumann.

zozbot234 2y ago
> Centralizing the processing was an important innovation by von Neumann a.k.a. the "von Neumann bottleneck" that we're now trying to get rid of.
- adrian_b 2y ago
  We will never get rid of the "von Neumann bottleneck", except for a relatively small number of niche applications. The bottleneck consists in the fact that instead of having a huge number of specialized automata that perform everything that must be done to execute a useful application you have just an universal automaton together with a big memory, where the universal automaton can perform anything when given an appropriate program. The use of a shared automaton prevents many actions to be done concurrently, but it also provides a huge economy of logical circuits. The "von Neumann bottleneck" is alleviated by implementing in a computer as many processor cores as possible at a given technological level, each with its own non-shared cache memory. However removing completely the concept of programmable processor with separate memory would multiply the amount of logic circuits too much for any imaginable technology. The idea of mixing computational circuits with the memory cells is feasible only for restricted well defined applications, e.g. perhaps for something like ML inference, but not for general-purpose applications.

by kens 2y ago

zozbot234 2y ago

> Centralizing the processing was an important innovation by von Neumann a.k.a. the "von Neumann bottleneck" that we're now trying to get rid of.

adrian_b 2y ago
We will never get rid of the "von Neumann bottleneck", except for a relatively small number of niche applications. The bottleneck consists in the fact that instead of having a huge number of specialized automata that perform everything that must be done to execute a useful application you have just an universal automaton together with a big memory, where the universal automaton can perform anything when given an appropriate program. The use of a shared automaton prevents many actions to be done concurrently, but it also provides a huge economy of logical circuits. The "von Neumann bottleneck" is alleviated by implementing in a computer as many processor cores as possible at a given technological level, each with its own non-shared cache memory. However removing completely the concept of programmable processor with separate memory would multiply the amount of logic circuits too much for any imaginable technology. The idea of mixing computational circuits with the memory cells is feasible only for restricted well defined applications, e.g. perhaps for something like ML inference, but not for general-purpose applications.