5 ms·

How many people does it take to implement this? A 10% gain in performance could pay for a lot of people's salaries when your company is spending hundreds of mil

How many people does it take to implement this? A 10% gain in performance could pay for a lot of people's salaries when your company is spending hundreds of millions on GPU clusters.

saagarjha 2y ago
I mean it’s not a significant change so one? But that isn’t to say anyone could do it.
- rvz 2y ago
  Just a reminder, this is the third of many open source releases from DeepSeek that they are willing to release, and that release is a very trivial low bar for them to find optimizations when it is needed. I guess since the majority here are blown away by the very low-level code involved, it tells me that they're likely not ready to use it or have been stuck on very high level tools that abstract this away.
  - randomNumber7 2y ago
    I tell you a secret. Most devs do something wrong when they start rolling out their own linear algebra library. Thats why people use LAPAC, BLAS, etc...
    - KeplerBoy 2y ago
      The thing is most people don't use Lapack or Blas. Most people are at higher levels of abstraction than torch.matmul.
fulafel 2y ago
If you think how many people who looked and failed to realize this optimization in the preceding performance efforts of the community, you could argue for quite a big number.
rfoo 2y ago
Uh, three? I worked at $CORP where we had a three people sub-team, they reverse engineered most of Volta's SASS instruction encoding, built a working SASS assembler (before the open source one of course), with the ultimate goal of making GEMM / Conv faster. And they did it. Though it wasn't applied to a high-profile enough big picture so we never heard about it :> If you don't believe me, previous open source SASS assemblers were mostly from university, they surely didn't have that many people.
- bjourne 2y ago
  Did $CORP also release the im0lementation to make it trivial for others to replicate their work?
  - rfoo 2y ago
    I think we did release some of the optimized kernels but I don't think we have released any one with SASS black magic, at least not before I left. Already been sanctioned by BIS, better not annoy NVIDIA furthermore.
  - DannyBee 2y ago
    Actually, a number of them did. Even Google did.
rowanG077 2y ago
Just a few of highly skilled people.

How many people does it take to implement this? A 10% gain in performance could pay for a lot of people's salaries when your company is spending hundreds of millions on GPU clusters.

saagarjha 2y ago

I mean it’s not a significant change so one? But that isn’t to say anyone could do it.

rvz 2y ago
Just a reminder, this is the third of many open source releases from DeepSeek that they are willing to release, and that release is a very trivial low bar for them to find optimizations when it is needed. I guess since the majority here are blown away by the very low-level code involved, it tells me that they're likely not ready to use it or have been stuck on very high level tools that abstract this away.
- randomNumber7 2y ago
  I tell you a secret. Most devs do something wrong when they start rolling out their own linear algebra library. Thats why people use LAPAC, BLAS, etc...
  - KeplerBoy 2y ago
    The thing is most people don't use Lapack or Blas. Most people are at higher levels of abstraction than torch.matmul.

fulafel 2y ago

If you think how many people who looked and failed to realize this optimization in the preceding performance efforts of the community, you could argue for quite a big number.

rfoo 2y ago

Uh, three? I worked at $CORP where we had a three people sub-team, they reverse engineered most of Volta's SASS instruction encoding, built a working SASS assembler (before the open source one of course), with the ultimate goal of making GEMM / Conv faster. And they did it. Though it wasn't applied to a high-profile enough big picture so we never heard about it :> If you don't believe me, previous open source SASS assemblers were mostly from university, they surely didn't have that many people.

bjourne 2y ago
Did $CORP also release the im0lementation to make it trivial for others to replicate their work?
- rfoo 2y ago
  I think we did release some of the optimized kernels but I don't think we have released any one with SASS black magic, at least not before I left. Already been sanctioned by BIS, better not annoy NVIDIA furthermore.
- DannyBee 2y ago
  Actually, a number of them did. Even Google did.

rowanG077 2y ago

Just a few of highly skilled people.