3 ms·

Our theoretical double precision FLOPs (based on running at 1GHz) per core is 1GFLOP per second, but that is based on doing an add or a multiply. In the case of

Our theoretical double precision FLOPs (based on running at 1GHz) per core is 1GFLOP per second, but that is based on doing an add or a multiply. In the case of just doing FMAs, you would be doing 1.5GFLOP. This gives you (theoretical peak) 256 to 384 double precision GFLOPs per chip. Our bandwidth between cores is 16GB/s, while the required bandwidth for doing 1GFLOP is 8GB/s. Our single precision numbers are actually double that of double precision (compared to a GPU or most other SIMD systems, which have independent FP32 and FP64 FPUs, we have a single combined unit). While our ISA is pure 64 bit, we have packed 32 bit FPU instructions for doing two single precision FLOPs per cycle.

trsohmers 12y ago
I forgot to add why the separate FP32 and FP64 floating point units are "bad" (from our PoV)... having the separate units just increase complexity, and NVIDIA GPUs, for instance, have 4x the number of FP32 units compared to FP64 units, thus making their single precision numbers (which are what they typically advertise) 4x that of their double precision numbers.
vardump 12y ago
So how many I can do if for each FMA I also receive 16 bytes of data from a neighboring node and send 16 bytes of data to a neighboring node? Or is the data transfer free, is the neighboring node memory mapped? If so, how does synchronization work? Edit: didn't notice same was asked before too. Regardless, how many FMAs can be executed in the scenario I gave, also sending 16 bytes and receiving 16 bytes for each FMA?
- trsohmers 12y ago
  The superscalar design means Load/Store unit can operate independently from FMA and DMA units, i.e. FMA operation on register operands will not interfere with other operations elsewhere on the chip. Synchronization should be handled in a dataflow-driven program design. Any sort of mutex/semaphore/etc. will have to be software defined or interrupt-driven. In regards to neighboring node being memory mapped... are you asking about another compute chip or another node (with the 16 compute chips + GaMMU)? All of the compute chips in a grid are part of the same flat global memory map, and have DMA capabilities between each other. Once you get out of the compute grid (that is managed by the GaMMU), then that is a separate memory space, but can be accessed through some other layer through something like MPI, for instance.
  - vardump 12y ago
    I guess this is one of those cases where you just need to get your hands dirty to really understand it. Can you do reasonably efficient [arbitrary size] fixed point arithmetic on your hardware? Do you have 64-bit add with carry and 64-bit multiply with 128 bit results? I'm interested in 64.64 and 128.128. Needed operations are add, sub and multiply. I think compute grids like these could very well be an important part of computing in the future, maybe even the most important part. Ever since when I first saw an article about transputer. Grids or VLIWs, sadly software is always the pain point. I wish you luck, please get this working and right.

trsohmers 12y ago

I forgot to add why the separate FP32 and FP64 floating point units are "bad" (from our PoV)... having the separate units just increase complexity, and NVIDIA GPUs, for instance, have 4x the number of FP32 units compared to FP64 units, thus making their single precision numbers (which are what they typically advertise) 4x that of their double precision numbers.

vardump 12y ago

So how many I can do if for each FMA I also receive 16 bytes of data from a neighboring node and send 16 bytes of data to a neighboring node? Or is the data transfer free, is the neighboring node memory mapped? If so, how does synchronization work? Edit: didn't notice same was asked before too. Regardless, how many FMAs can be executed in the scenario I gave, also sending 16 bytes and receiving 16 bytes for each FMA?

trsohmers 12y ago
The superscalar design means Load/Store unit can operate independently from FMA and DMA units, i.e. FMA operation on register operands will not interfere with other operations elsewhere on the chip. Synchronization should be handled in a dataflow-driven program design. Any sort of mutex/semaphore/etc. will have to be software defined or interrupt-driven. In regards to neighboring node being memory mapped... are you asking about another compute chip or another node (with the 16 compute chips + GaMMU)? All of the compute chips in a grid are part of the same flat global memory map, and have DMA capabilities between each other. Once you get out of the compute grid (that is managed by the GaMMU), then that is a separate memory space, but can be accessed through some other layer through something like MPI, for instance.
- vardump 12y ago
  I guess this is one of those cases where you just need to get your hands dirty to really understand it. Can you do reasonably efficient [arbitrary size] fixed point arithmetic on your hardware? Do you have 64-bit add with carry and 64-bit multiply with 128 bit results? I'm interested in 64.64 and 128.128. Needed operations are add, sub and multiply. I think compute grids like these could very well be an important part of computing in the future, maybe even the most important part. Ever since when I first saw an article about transputer. Grids or VLIWs, sadly software is always the pain point. I wish you luck, please get this working and right.