3 ms·

I've worked on .. several? high core count 2d array machines, and they tend not to be to useful for general code because of limited memory bandwidth. some pipel

I've worked on .. several? high core count 2d array machines, and they tend not to be to useful for general code because of limited memory bandwidth. some pipelines and embarrassingly parallel things work really well. generally machines with higher dimensional interconnect tend to be more useful I agree we shouldn't have completely abandoned these machines in the 90s. the massive resurgence we see of vector and simd today proves that there was substantial value - and maybe the software story would be a lot better if we hadn't have taken a 20 year piss break.

cmrdporcupine 3y ago
It's true, I can't help but imagine what we could do if we just down-clocked CPU in at the same maximum speed as the bus&RAM and went stupid-wide instead of stupid-complicated. Forget about smartly filling our L1 caches, super clever branch prediction, NUMA, very wide register sets, etc. and just go "embarassingly parallel" as you say, and with the right programming paradigm perhaps the tradeoffs would work out in either $$ or MIPS. Though of course I know it's been done and either failed technically, or failed in the market, several times. E.g. fairly recently the Parallela Epiphany stuff (https://parallella.org/2016/10/05/epiphany-v-a-1024-core-64-bit-risc-processor/ https://parallella.org/2016/10/05/epiphany-v-a-1024-core-64-...) which sounded super-gee-whiz-bang-neat but went nowhere. But maybe the application domains and compilers just weren't there for it yet. I like what e.g. Parallax has done with their Propeller 2, in the microcontroller domain. 8 cores round-robining on a shared memory, but with their own smaller localized workspace RAM, and a pile of I/O. No need for interrupts to manage concurrency, just assign a core to it. https://www.parallax.com/propeller-2/ https://www.parallax.com/propeller-2/
- dontlaugh 3y ago
  Embarrassingly parallel problems are relatively rare. Even GPU cores are quite big and they’re explicitly dedicated to mostly embarrassing parallelism. The complexity is sadly mostly inherent in the problems being solved.
- convolvatron 3y ago
  one place where more software/language/architecture work is really needed to make this work well is architectures which aren't cache coherent (either by simply not having any caches like the MTA or the CM, by having software controlled caches, by strongly delineating local and global memory, or some other approach).
  - cmrdporcupine 3y ago
    I've been sitting here in my spare time writing / yak shaving an adaptive radix tree implementation I wrote... to try to keep various things in L1 cache and, well, y'know, it'd be nice in a way to just sometimes... have this all be explicit and not automatic? To be able to put the machine into a mode where I could have the various levels of memory declared by the hardware have separate manual methods for accessing each, e.g. to be able to load L1 myself, etc. The CPU & board could tell me the size and cost of each level, and I could write a software model to move things around myself. Fantasy.
    - convolvatron 3y ago
      There is a whole class of communication optimal algorithms. And cache oblivious ones. You really shouldn’t have to do what you’re doing, and progress is being made.

cmrdporcupine 3y ago

It's true, I can't help but imagine what we could do if we just down-clocked CPU in at the same maximum speed as the bus&RAM and went stupid-wide instead of stupid-complicated. Forget about smartly filling our L1 caches, super clever branch prediction, NUMA, very wide register sets, etc. and just go "embarassingly parallel" as you say, and with the right programming paradigm perhaps the tradeoffs would work out in either $$ or MIPS. Though of course I know it's been done and either failed technically, or failed in the market, several times. E.g. fairly recently the Parallela Epiphany stuff (https://parallella.org/2016/10/05/epiphany-v-a-1024-core-64-bit-risc-processor/ https://parallella.org/2016/10/05/epiphany-v-a-1024-core-64-...) which sounded super-gee-whiz-bang-neat but went nowhere. But maybe the application domains and compilers just weren't there for it yet. I like what e.g. Parallax has done with their Propeller 2, in the microcontroller domain. 8 cores round-robining on a shared memory, but with their own smaller localized workspace RAM, and a pile of I/O. No need for interrupts to manage concurrency, just assign a core to it. https://www.parallax.com/propeller-2/ https://www.parallax.com/propeller-2/

dontlaugh 3y ago
Embarrassingly parallel problems are relatively rare. Even GPU cores are quite big and they’re explicitly dedicated to mostly embarrassing parallelism. The complexity is sadly mostly inherent in the problems being solved.
convolvatron 3y ago
one place where more software/language/architecture work is really needed to make this work well is architectures which aren't cache coherent (either by simply not having any caches like the MTA or the CM, by having software controlled caches, by strongly delineating local and global memory, or some other approach).
- cmrdporcupine 3y ago
  I've been sitting here in my spare time writing / yak shaving an adaptive radix tree implementation I wrote... to try to keep various things in L1 cache and, well, y'know, it'd be nice in a way to just sometimes... have this all be explicit and not automatic? To be able to put the machine into a mode where I could have the various levels of memory declared by the hardware have separate manual methods for accessing each, e.g. to be able to load L1 myself, etc. The CPU & board could tell me the size and cost of each level, and I could write a software model to move things around myself. Fantasy.
  - convolvatron 3y ago
    There is a whole class of communication optimal algorithms. And cache oblivious ones. You really shouldn’t have to do what you’re doing, and progress is being made.