5 ms·
That was one of the strange highly-parallel machines of the 1980s-1990s. There were the NCube, the BBN Butterfly, the Transputer, and some other machines. The
by Animats 3y ago
That was one of the strange highly-parallel machines of the 1980s-1990s. There were the NCube, the BBN Butterfly, the Transputer, and some other machines.
The NCube was another power of 2 machine, and could have up to 1024 CPUS, 64 per meter-square card. Each CPU was conventional, about 1 MIPS, with, I think, 128KB RAM.
Stanford had one with one card of 64 CPUs. Well, 63; one was broken. It was a donation from an oil company that found it wasn't useful for seismic data processing. Nobody found a good use for it at Stanford, and it was donated to some other school.
Each CPU had a 0..1023 address. Message sending XORed the address of the source and destination, and the 0 bits told which paths would take the message closer to its destination. In theory there was supposed to be a path for each bit difference, like the Connection Machine, but actually there were some shared bus-type paths, I think. This allowed doing the whole thing with printed circuit cards and a printed circuit backplane, rather than a huge number of discrete wires. So the whole thing fit in a 1M cube.
With all these strange machines, the problem has to fit the machine, or you don't get the parallelism.
Trying to fit existing problems into those forms was mostly a flop. Which is why general purpose shared memory multiprocessors with caches won out.
Massive parallelism seems to be problem-first. Graphics has a mostly-forward pipeline, so GPUs have a mostly-forward pipeline. Bitcoin miners have barely any intercommunication needs at all, so they're standalone things with minimal intercommunication. Back-propagation has a very standard form, and now we're seeing hardware that's sort of like a GPU but with lots of short multiply/add hardware.
What's striking is how much compute people have been able to get out of GPUs. They are sometimes the wrong tool for the job, but they're mass produced and affordable. The multi-million dollar one-off parallel machines, though, were mostly dead ends.
- tvarghese7 3y agoYup, getting the the problem to fit the machine was hard. I thought NCUBE was better than the CM1 for doing actual work. Some people resorted to just running each node separately, after the initial conditions were set up there was no need for communications. That was one way around having to be clever and adapt your algorithm. They would run for days, sometimes because nobody else was on the machines :) At one point I got my 2D code running on a CM2 (IIRC), one of the nodes was bad and the Fortran log function always returned 0. So when I made the movie of my simulation there was one tile/rectangle that was just blank. Afterwards they asked me if they could use my program as a diagnostic tool to verify all the nodes were working correctly. :)) I moved one from parallel and supercomputers when I graduated. Don't miss them that much.
- mk_stjames 3y agoIt is interesting you mention the log function being what tripped up the broken node. On the CM1 and CM2, log was implemented via Feynman's algorithm for logarithms, which finds factors via a shift and subtract and then adds precomputed logarithms of them via a hit to a table that is shared by all the processors. If one of the nodes had a bad route to that lookup table (however that was actually done), it would result in just that node failing to return a result for a log instruction.
- pjmlp 3y agoAlthough there are already a couple of options, e.g. Futhark, compared with those machines, GPGPUs are still too much focused on low level programming practices. I think we need more expressive ways, like StarLisp has being researched for. Some of the current CUDA work in C++ libraries, or having a Python/Julia JITs to PTX, are already nice steps into that direction as well.