4 ms·

Because DL is 90% Matrix multiplication?

by extasia 3y ago

blowski 3y ago
Which begs the question… why are GPUs good at matrix multiplication?
- NikkiA 3y ago
  Because 3d graphics are mostly matrix multiplications.
  - vorticalbox 3y ago
    This is like saying "cars are fast because cars are good at going fast"
    - blowski 3y ago
      I guess it's like "cars are fast because they use similar engines to motorbikes, which also go fast". Technically correct, but doesn't really provide any useful information we can learn from.
    - flohofwoe 3y ago
      GPUs haven't been developed in a vacuum, they were mostly created to accelerate rendering for 3D games, and 3D rendering is mostly about vector operations in up to 4 dimensions (to process points and vectors in 3D space, and pixels in an RGB+transparency 'color space'). The other interesting property of 3D rendering is that it is an 'embarrassingly parallel' problem. Vertices or pixels (mostly) don't depend on each other, so the GPU can process them in parallel. That's how GPUs turned over time into big arrays of very simple cores which are good at running a huge number of vector operations in parallel.
- throwawaymaths 3y ago
  They're actually kind of not, but they're good at vector multiplication. Matrix multiplication is just stacked vector multiplication, so GPUs are okay at matrix multiplication (but not as good as one could imagine they could be). I think TPUs were the first thing really designed to do matrix multiplication
- dragontamer 3y ago
  Because matrix multiplication parallelizes easily. 1. Matrix Multiplication caches easily: you can program the data-fetches to occur in blocks and recycle your memory-fetches across a huge number of operations. 2. Matrix Multiplication is innately parallel, with every square of the output matrix possibly computed in parallel. ----------- That being said: GPUs are not the best at this. Systolic processors are the best matrix multipliers and that's where NVidia Tensor Cores and/or Google TPUs come in. ------------ GPU / SIMD architecture was chosen for GPUs because video games have a large number of 4x4 matrix multiplications per frame. Every single vertex, of which a typical video game could have ~100 million vertexes, needs to be 4x4 matrix multiplied to determine where on the screen it has moved. GPUs need to perform this operation at 60 FPS for video gamers to be pleased, though some video gamers demand 120 FPS or even 200+ FPS. 4x4 is the operation of an affine transformation in 3d space (translation, rotation, scaling, etc. etc.). Most commonly for 'camera movement'. (Player moved to the right 5 ticks and then looked to the north-west by 27 degrees, where should all the objects in the screen move to? This is solved by matrix-multiplying all 100-million verticies with the camera-position matrix) 4x4 Matrix Multiplication isn't quite big enough to "deserve" a systolic processor yet.
ThemalSpan 3y ago
First paragraph of the article: "As many have said GPUs are so fast because they are so efficient for matrix multiplication and convolution, but nobody gave a real explanation for why this is so. The real reason for this is memory bandwidth and not necessarily parallelism."
colordrops 3y ago
Perhaps the deeper question is why both visual representation and deep learning both rely heavily on matrix multiplication.
- pazimzadeh 3y ago
  What part of visual representation is the most graphics intensive? The least?
  - flohofwoe 3y ago
    One part is vertex transformation which transforms 3D points into the 2D screen space, the other is computing the per-pixel colors. Both work on 4D vector data though. Early GPUs had different processing units for vertex and pixel "shading", but over time those have merged into generic cores which can be dynamically assigned to vertex-, pixel- or 'compute'-tasks.
- throwawaymaths 3y ago
  Matrices are just tabulated numbers dude.
- dragontamer 3y ago
  Because 4x4 Matrixes are a linear transformation of 3 dimensional space. Deep Learning chose TPUs / Tensors because GPUs (and other computers) are faster at matrix multiplication than any other operation. Many many other representations of neural networks have existed over the decades, its the matrix-based "Tensor" that performed the best and took off.
casualscience 3y ago
Yes, and graphics cards are optimized to do matrix multiplications because most of what they do is projective geometry.

Because DL is 90% Matrix multiplication?

blowski 3y ago

Which begs the question… why are GPUs good at matrix multiplication?

NikkiA 3y ago
Because 3d graphics are mostly matrix multiplications.
- vorticalbox 3y ago
  This is like saying "cars are fast because cars are good at going fast"
  - blowski 3y ago
    I guess it's like "cars are fast because they use similar engines to motorbikes, which also go fast". Technically correct, but doesn't really provide any useful information we can learn from.
  - flohofwoe 3y ago
    GPUs haven't been developed in a vacuum, they were mostly created to accelerate rendering for 3D games, and 3D rendering is mostly about vector operations in up to 4 dimensions (to process points and vectors in 3D space, and pixels in an RGB+transparency 'color space'). The other interesting property of 3D rendering is that it is an 'embarrassingly parallel' problem. Vertices or pixels (mostly) don't depend on each other, so the GPU can process them in parallel. That's how GPUs turned over time into big arrays of very simple cores which are good at running a huge number of vector operations in parallel.
throwawaymaths 3y ago
They're actually kind of not, but they're good at vector multiplication. Matrix multiplication is just stacked vector multiplication, so GPUs are okay at matrix multiplication (but not as good as one could imagine they could be). I think TPUs were the first thing really designed to do matrix multiplication
dragontamer 3y ago
Because matrix multiplication parallelizes easily. 1. Matrix Multiplication caches easily: you can program the data-fetches to occur in blocks and recycle your memory-fetches across a huge number of operations. 2. Matrix Multiplication is innately parallel, with every square of the output matrix possibly computed in parallel. ----------- That being said: GPUs are not the best at this. Systolic processors are the best matrix multipliers and that's where NVidia Tensor Cores and/or Google TPUs come in. ------------ GPU / SIMD architecture was chosen for GPUs because video games have a large number of 4x4 matrix multiplications per frame. Every single vertex, of which a typical video game could have ~100 million vertexes, needs to be 4x4 matrix multiplied to determine where on the screen it has moved. GPUs need to perform this operation at 60 FPS for video gamers to be pleased, though some video gamers demand 120 FPS or even 200+ FPS. 4x4 is the operation of an affine transformation in 3d space (translation, rotation, scaling, etc. etc.). Most commonly for 'camera movement'. (Player moved to the right 5 ticks and then looked to the north-west by 27 degrees, where should all the objects in the screen move to? This is solved by matrix-multiplying all 100-million verticies with the camera-position matrix) 4x4 Matrix Multiplication isn't quite big enough to "deserve" a systolic processor yet.

ThemalSpan 3y ago

First paragraph of the article: "As many have said GPUs are so fast because they are so efficient for matrix multiplication and convolution, but nobody gave a real explanation for why this is so. The real reason for this is memory bandwidth and not necessarily parallelism."

colordrops 3y ago

Perhaps the deeper question is why both visual representation and deep learning both rely heavily on matrix multiplication.

pazimzadeh 3y ago
What part of visual representation is the most graphics intensive? The least?
- flohofwoe 3y ago
  One part is vertex transformation which transforms 3D points into the 2D screen space, the other is computing the per-pixel colors. Both work on 4D vector data though. Early GPUs had different processing units for vertex and pixel "shading", but over time those have merged into generic cores which can be dynamically assigned to vertex-, pixel- or 'compute'-tasks.
throwawaymaths 3y ago
Matrices are just tabulated numbers dude.
dragontamer 3y ago
Because 4x4 Matrixes are a linear transformation of 3 dimensional space. Deep Learning chose TPUs / Tensors because GPUs (and other computers) are faster at matrix multiplication than any other operation. Many many other representations of neural networks have existed over the decades, its the matrix-based "Tensor" that performed the best and took off.

casualscience 3y ago

Yes, and graphics cards are optimized to do matrix multiplications because most of what they do is projective geometry.