7 ms·

They're proposing "new hardware architectures" to take advantage of this idea. Anybody with a background in GPU floating point math comment on how realistic th

by oofbey 2y ago

They're proposing "new hardware architectures" to take advantage of this idea. Anybody with a background in GPU floating point math comment on how realistic this is?

winterismute 2y ago
The paper is about integer multiplication, not float
- oofbey 2y ago
  Why would that matter? I understood the point was to speed up matrix multiplication by doing the adds and multiplies in a different order. Shouldn't matter whether the datatype is int, float, complex, whatever.
  - sifar 2y ago
    One can use this techniques to optimize the multiplier inside the FP FMA unit. However this cannot be used to multiply two floating point numbers as FP arithmetic is not associative.
    - saagarjha 2y ago
      I'm pretty sure a lot of people doing matrix multiplications are ok with non-associative floating point arithmetic?
emacs28 2y ago
First author here. The hardware architectures are realistic - we developed & evaluated real example hardware implementations for them, validated on FPGA, and they achieved state-of-the-art ResNet performance in a deep learning accelerator system implementation compared to prior accelerators evaluated on similar FPGAs. See the associated accelerator system source code here: https://github.com/trevorpogue/algebraic-nnhw https://github.com/trevorpogue/algebraic-nnhw The hardware architectures focused on in the paper are systolic array designs, an efficient type of hardware design for matrix multiplication (e.g., the Google TPU uses this), as opposed to more SIMD-like vector architectures like GPUs. It may be possible to extend the proposed KMM algorithm to other types of hardware architectures also in future work. Regarding floating point - this work is applicable for integer matrix multiplication acceleration, it may be possible to extend the concept to floating point data types in future work also.
- almostgotcaught 2y ago
  > systolic array designs, an efficient type of hardware design for matrix multiplication (e.g., the Google TPU uses this), as opposed to more SIMD-like vector architectures like GPUs this is wrong. TPUv4 has tensor cores just like NVIDIA has tensor cores just like AMD has tensor cores. no one uses a systolic array because bandwidth/connectivity is much scarcer than compute. the only people that keep talking about them are academics that don't actually fab/sell chips. https://cloud.google.com/tpu/docs/v4 https://cloud.google.com/tpu/docs/v4 https://www.nvidia.com/en-us/data-center/tensor-cores/ https://www.nvidia.com/en-us/data-center/tensor-cores/ https://rocm.docs.amd.com/projects/rocWMMA/en/latest/what-is-rocwmma.html https://rocm.docs.amd.com/projects/rocWMMA/en/latest/what-is... ninja edit: before you gotcha me with "a tensor core is a systolic array!!!" - most tensor cores are actually outerproduct engines not riffle shuffle engines (or whatever you wanna call the topology corresponding to a systolic array).
  - imtringued 2y ago
    https://cloud.google.com/tpu/docs/system-architecture-tpu-vm?hl=en https://cloud.google.com/tpu/docs/system-architecture-tpu-vm... >The primary task for TPUs is matrix processing, which is a combination of multiply and accumulate operations. TPUs contain thousands of multiply-accumulators that are directly connected to each other to form a large physical matrix. This is called a systolic array architecture. Cloud TPU v3, contain two systolic arrays of 128 x 128 ALUs, on a single processor.
    - robinhouston 2y ago
      I don't see any contradiction between your claim that TPU v3 uses systolic arrays and the parent post's claim that TPU v4 does not.
  - FL33TW00D 2y ago
    The TPU obviously uses a systolic array: https://jax-ml.github.io/scaling-book/tpus/ https://jax-ml.github.io/scaling-book/tpus/
deleted 2y ago
[deleted]

by oofbey 2y ago

They're proposing "new hardware architectures" to take advantage of this idea. Anybody with a background in GPU floating point math comment on how realistic this is?

winterismute 2y ago

The paper is about integer multiplication, not float

oofbey 2y ago
Why would that matter? I understood the point was to speed up matrix multiplication by doing the adds and multiplies in a different order. Shouldn't matter whether the datatype is int, float, complex, whatever.
- sifar 2y ago
  One can use this techniques to optimize the multiplier inside the FP FMA unit. However this cannot be used to multiply two floating point numbers as FP arithmetic is not associative.
  - saagarjha 2y ago
    I'm pretty sure a lot of people doing matrix multiplications are ok with non-associative floating point arithmetic?

emacs28 2y ago

First author here. The hardware architectures are realistic - we developed & evaluated real example hardware implementations for them, validated on FPGA, and they achieved state-of-the-art ResNet performance in a deep learning accelerator system implementation compared to prior accelerators evaluated on similar FPGAs. See the associated accelerator system source code here: https://github.com/trevorpogue/algebraic-nnhw https://github.com/trevorpogue/algebraic-nnhw The hardware architectures focused on in the paper are systolic array designs, an efficient type of hardware design for matrix multiplication (e.g., the Google TPU uses this), as opposed to more SIMD-like vector architectures like GPUs. It may be possible to extend the proposed KMM algorithm to other types of hardware architectures also in future work. Regarding floating point - this work is applicable for integer matrix multiplication acceleration, it may be possible to extend the concept to floating point data types in future work also.

almostgotcaught 2y ago
> systolic array designs, an efficient type of hardware design for matrix multiplication (e.g., the Google TPU uses this), as opposed to more SIMD-like vector architectures like GPUs this is wrong. TPUv4 has tensor cores just like NVIDIA has tensor cores just like AMD has tensor cores. no one uses a systolic array because bandwidth/connectivity is much scarcer than compute. the only people that keep talking about them are academics that don't actually fab/sell chips. https://cloud.google.com/tpu/docs/v4 https://cloud.google.com/tpu/docs/v4 https://www.nvidia.com/en-us/data-center/tensor-cores/ https://www.nvidia.com/en-us/data-center/tensor-cores/ https://rocm.docs.amd.com/projects/rocWMMA/en/latest/what-is-rocwmma.html https://rocm.docs.amd.com/projects/rocWMMA/en/latest/what-is... ninja edit: before you gotcha me with "a tensor core is a systolic array!!!" - most tensor cores are actually outerproduct engines not riffle shuffle engines (or whatever you wanna call the topology corresponding to a systolic array).
- imtringued 2y ago
  https://cloud.google.com/tpu/docs/system-architecture-tpu-vm?hl=en https://cloud.google.com/tpu/docs/system-architecture-tpu-vm... >The primary task for TPUs is matrix processing, which is a combination of multiply and accumulate operations. TPUs contain thousands of multiply-accumulators that are directly connected to each other to form a large physical matrix. This is called a systolic array architecture. Cloud TPU v3, contain two systolic arrays of 128 x 128 ALUs, on a single processor.
  - robinhouston 2y ago
    I don't see any contradiction between your claim that TPU v3 uses systolic arrays and the parent post's claim that TPU v4 does not.
- FL33TW00D 2y ago
  The TPU obviously uses a systolic array: https://jax-ml.github.io/scaling-book/tpus/ https://jax-ml.github.io/scaling-book/tpus/

deleted 2y ago

[deleted]