3 ms·

It's true, I can't help but imagine what we could do if we just down-clocked CPU in at the same maximum speed as the bus&RAM and went stupid-wide instead of stu

It's true, I can't help but imagine what we could do if we just down-clocked CPU in at the same maximum speed as the bus&RAM and went stupid-wide instead of stupid-complicated. Forget about smartly filling our L1 caches, super clever branch prediction, NUMA, very wide register sets, etc. and just go "embarassingly parallel" as you say, and with the right programming paradigm perhaps the tradeoffs would work out in either $$ or MIPS. Though of course I know it's been done and either failed technically, or failed in the market, several times. E.g. fairly recently the Parallela Epiphany stuff (https://parallella.org/2016/10/05/epiphany-v-a-1024-core-64-bit-risc-processor/ https://parallella.org/2016/10/05/epiphany-v-a-1024-core-64-...) which sounded super-gee-whiz-bang-neat but went nowhere. But maybe the application domains and compilers just weren't there for it yet. I like what e.g. Parallax has done with their Propeller 2, in the microcontroller domain. 8 cores round-robining on a shared memory, but with their own smaller localized workspace RAM, and a pile of I/O. No need for interrupts to manage concurrency, just assign a core to it. https://www.parallax.com/propeller-2/ https://www.parallax.com/propeller-2/

dontlaugh 4y ago
Embarrassingly parallel problems are relatively rare. Even GPU cores are quite big and they’re explicitly dedicated to mostly embarrassing parallelism. The complexity is sadly mostly inherent in the problems being solved.
convolvatron 4y ago
one place where more software/language/architecture work is really needed to make this work well is architectures which aren't cache coherent (either by simply not having any caches like the MTA or the CM, by having software controlled caches, by strongly delineating local and global memory, or some other approach).
- cmrdporcupine 4y ago
  I've been sitting here in my spare time writing / yak shaving an adaptive radix tree implementation I wrote... to try to keep various things in L1 cache and, well, y'know, it'd be nice in a way to just sometimes... have this all be explicit and not automatic? To be able to put the machine into a mode where I could have the various levels of memory declared by the hardware have separate manual methods for accessing each, e.g. to be able to load L1 myself, etc. The CPU & board could tell me the size and cost of each level, and I could write a software model to move things around myself. Fantasy.
  - convolvatron 4y ago
    There is a whole class of communication optimal algorithms. And cache oblivious ones. You really shouldn’t have to do what you’re doing, and progress is being made.

dontlaugh 4y ago

Embarrassingly parallel problems are relatively rare. Even GPU cores are quite big and they’re explicitly dedicated to mostly embarrassing parallelism. The complexity is sadly mostly inherent in the problems being solved.

convolvatron 4y ago

one place where more software/language/architecture work is really needed to make this work well is architectures which aren't cache coherent (either by simply not having any caches like the MTA or the CM, by having software controlled caches, by strongly delineating local and global memory, or some other approach).

cmrdporcupine 4y ago
I've been sitting here in my spare time writing / yak shaving an adaptive radix tree implementation I wrote... to try to keep various things in L1 cache and, well, y'know, it'd be nice in a way to just sometimes... have this all be explicit and not automatic? To be able to put the machine into a mode where I could have the various levels of memory declared by the hardware have separate manual methods for accessing each, e.g. to be able to load L1 myself, etc. The CPU & board could tell me the size and cost of each level, and I could write a software model to move things around myself. Fantasy.
- convolvatron 4y ago
  There is a whole class of communication optimal algorithms. And cache oblivious ones. You really shouldn’t have to do what you’re doing, and progress is being made.