3 ms·

Thanks for the C-star link. At first glance shapes look similar to what have become launch dimensions. I think it'd be pretty interesting to hear a comparison f

by dahart 6y ago

Thanks for the C-star link. At first glance shapes look similar to what have become launch dimensions. I think it'd be pretty interesting to hear a comparison from someone who's used both C-star and CUDA, about the plusses and minuses of each... have you used both? > Very limited compared to modern GPUs Yeah, I'd expect the real true differences between CM5 SIMD and Pascal SIMT is just buried in the subtleties of those limitations compared to today's hardware, and that fundamentally they aren't super different ideas. In my mind, that doesn't mean that SIMT is meaningless or the same thing as SIMD. > NVidia was calling SIMT back in 2010, long before per-thread instruction counters were available in Pascal (GTX 10xx series of GPUs) Right; I'm saying the "SIMT" of 10 years ago has to do with masking and maybe other per-thread control (even though the Connection Machines might have had it as well), and that per-thread counters are now going beyond what we used to know as SIMT, perhaps deserving of some other acronym. Or maybe SIMT is becoming more appropriate and more differentiated from SIMD over time? Is it possible that CM5 should be called SIMT and we simply didn't have that acronym at the time?

dragontamer 6y ago
> Thanks for the C-star link. At first glance shapes look similar to what have become launch dimensions. I think it'd be pretty interesting to hear a comparison from someone who's used both C-star and CUDA, about the plusses and minuses of each... have you used both? Nope. I'm looking at C-Star through a historical lens, I was curious at the overall development of CUDA and was researching the "line of influence", so to speak. > Right; I'm saying the "SIMT" of 10 years ago has to do with masking and maybe other per-thread control (even though the Connection Machines might have had it as well), and that per-thread counters are now going beyond what we used to know as SIMT, perhaps deserving of some other acronym. Or maybe SIMT is becoming more appropriate and more differentiated from SIMD over time? Is it possible that CM5 should be called SIMT and we simply didn't have that acronym at the time? CM2 was just another SIMD machine, there were plenty of others before CM2 (its just that CM2 was one of the more popular ones). IIRC, Cray had similar SIMD machines that competed against them. The overall SIMD-methodology reaches back until the 70s at least, maybe earlier. Just noting that CM5 was MIMD at the lowest level, trying to correct a mistake I made a few posts back. ------------ I think what "happened" was that Intel adopted SWAR: "SIMD With A Register" when Intel created the MMX / SSE / AVX instruction sets. (compared to dedicated SIMD-computers like the CM-2). At some point, Intel's SWAR approach became colloquially known as SIMD (even though SWAR was much more limited compared to "proper" SIMD computers like the CM2). NVidia creates a new acronym called SIMT to differentiate Intel's SIMD (aka: SWAR) from a proper SIMD machine, even though the 80s-style SIMD was substantially similar to NVidia's new SIMT term.
- convolvatron 6y ago
  just a few minor comments: the cm-1 and cm-2 both supported up to 64k single bit processors, and as many 'virtual' processors as you wanted until their memories got too small the cm-5, while it did have a potentially mimd model, each mimd node was a sparc that was lashed to 4 simd vector units, with their own memory bandwidth. so you would be giving up a lot to just run on the sparcs. it was also pretty clumsy - I think there was just a message library one called from C? I think that its not entirely fair to compare the CMs to the Cray vector products. although somewhat similar, the programming model for the Crays was basically loop mining fortran and the CM really presented a model of 2^n fine grained processors. the precedent I always heard was the Iliac Note that Blelloch's favorite(?) primitive- the parallel prefix operation was used quite widely in the CM world, but equivalent 'horizontal' operations are pretty much completely lacking on the Intel/AMD instruction sets. Haven't looked at the GPU ones, but its absence is pretty frustrating

by dahart 6y ago

dragontamer 6y ago

> Thanks for the C-star link. At first glance shapes look similar to what have become launch dimensions. I think it'd be pretty interesting to hear a comparison from someone who's used both C-star and CUDA, about the plusses and minuses of each... have you used both? Nope. I'm looking at C-Star through a historical lens, I was curious at the overall development of CUDA and was researching the "line of influence", so to speak. > Right; I'm saying the "SIMT" of 10 years ago has to do with masking and maybe other per-thread control (even though the Connection Machines might have had it as well), and that per-thread counters are now going beyond what we used to know as SIMT, perhaps deserving of some other acronym. Or maybe SIMT is becoming more appropriate and more differentiated from SIMD over time? Is it possible that CM5 should be called SIMT and we simply didn't have that acronym at the time? CM2 was just another SIMD machine, there were plenty of others before CM2 (its just that CM2 was one of the more popular ones). IIRC, Cray had similar SIMD machines that competed against them. The overall SIMD-methodology reaches back until the 70s at least, maybe earlier. Just noting that CM5 was MIMD at the lowest level, trying to correct a mistake I made a few posts back. ------------ I think what "happened" was that Intel adopted SWAR: "SIMD With A Register" when Intel created the MMX / SSE / AVX instruction sets. (compared to dedicated SIMD-computers like the CM-2). At some point, Intel's SWAR approach became colloquially known as SIMD (even though SWAR was much more limited compared to "proper" SIMD computers like the CM2). NVidia creates a new acronym called SIMT to differentiate Intel's SIMD (aka: SWAR) from a proper SIMD machine, even though the 80s-style SIMD was substantially similar to NVidia's new SIMT term.

convolvatron 6y ago
just a few minor comments: the cm-1 and cm-2 both supported up to 64k single bit processors, and as many 'virtual' processors as you wanted until their memories got too small the cm-5, while it did have a potentially mimd model, each mimd node was a sparc that was lashed to 4 simd vector units, with their own memory bandwidth. so you would be giving up a lot to just run on the sparcs. it was also pretty clumsy - I think there was just a message library one called from C? I think that its not entirely fair to compare the CMs to the Cray vector products. although somewhat similar, the programming model for the Crays was basically loop mining fortran and the CM really presented a model of 2^n fine grained processors. the precedent I always heard was the Iliac Note that Blelloch's favorite(?) primitive- the parallel prefix operation was used quite widely in the CM world, but equivalent 'horizontal' operations are pretty much completely lacking on the Intel/AMD instruction sets. Haven't looked at the GPU ones, but its absence is pretty frustrating