4 ms·

It's possible you're underestimating the open source community. If there's a competing platform that hobbyists can tinker with, the ecosystem can improve quite

It's possible you're underestimating the open source community. If there's a competing platform that hobbyists can tinker with, the ecosystem can improve quite rapidly, especially when the competing platform is completely closed and hobbyists basically are locked out and have no alternative.

throwaway48476 2y ago
Innovation is a bottom up process. If they sell the hardware the community will spring up to take advantage.
bayindirh 2y ago
> It's possible you're underestimating the open source community. On the contrary. You really don't know how I love and prefer open source and love a more leveling playing field. > If there's a competing platform that hobbyists can tinker with... AMD's cards are better from hardware and software architecture standpoint, but the performance is not there yet. Plus, ROCm libraries are not that mature, but they're getting there. Developing high performance, high quality code is deceivingly expensive, because it's very heavy in theory, and you fly very close to the metal. I did that in my Ph.D., so I know what it entails. So it requires more than a couple (hundred) hobbyists to pull off (see the development of Eigen linear algebra library, or any high end math library). Some big guns are pouring money into AMD to implement good ROCm libraries, and it started paying off (Debian has a ton of ROCm packages now, too). However, you need to be able to pull it off in the datacenter to be able to pull it off on the desktop. AMD also needs to be able to enable ROCm on desktop properly, so people can start hacking it at home. > especially when the competing platform is completely closed... NVIDIA gives a lot of support to universities, researchers and institutions who play with their cards. Big cards may not be free, but know-how, support and first steps are always within reach. Plus, their researchers dogfood their own cards, and write papers with them. So, as long as papers got published, researchers do their research, and something got invented, many people don't care about how open source the ecosystem is. This upsets me a ton, but when closed source AI companies and researchers who forget to add crucial details to their papers so what they did can't be reproduced don't care about open source, because they think like NVIDIA. "My research, my secrets, my fame, my money". It's not about sharing. It's about winning, and it's ugly in some aspects.
- light_hue_1 2y ago
  You're writing as if AMD cares about open source. If they would only actually open source their driver the community would have made their cards better than nvidia ones long ago. I'm one of those academics. You've got it all wrong. So many people care about open source. So many people carefully release their code and make everything reproducible. We desperately just want AMD to open up. They just refuse. There's nothing secret going on and there's no conspiracy. There's just a company that for some inexplicable reason doesn't want to make boatloads of money for free. AMD is the worst possible situation. They're hostile to us and they refuse to invest to make their stuff work.
  - talldayo 2y ago
    > If they would only actually open source their driver the community would have made their cards better than nvidia ones long ago. Software wise, maybe. But you can't change AMD's hardware with a magic wand, and that's where a lot of CUDA's optimizations come from. AMD's GPU architecture is optimized for raster compute, and it's been that way for decades. I can assure you that AMD does not have a magic button to press that would make their systems competitive for AI. If that was possible it would have been done years ago, with or without their consent. The problem is deeper and extends to design decisions and disagreement over the complexity of GPU designs. If you compare AMD's cards to Nvidia on "fair ground" (eg. no CUDA, only OpenCL) the GPGPU performance still leans in Nvidia's favor.
    - anticensor 2y ago
      RDNA and UDNA have progressed more towards compute-unified architecture than Vega&CDNA's raster-first architecture.
- btown 2y ago
  Yep, this thread has a good compilation of ROCm woes: https://news.ycombinator.com/item?id=34832660 https://news.ycombinator.com/item?id=34832660 That said, for hobbyist inference on large pretrained models, I think there is an interesting set of possibilities here: maybe a number of operations aren't optimized, and it takes 10x as long to load the model into memory... but all that might not matter if AMD were to be the first to market for 128GB+ VRAM cards that are the only things that can run next-generation open-weight models in a desktop environment, particularly those generating video and images. The hobbyists don't need to optimize all the linear algebra operations that researchers need to be able to experiment with when training; they just need to implement the ones used by the open-weight models. But of course this is all just wishful thinking, because as others have pointed out, any developments in this direction would require a level of foresight that AMD simply hasn't historically shown.
kevin_thibedeau 2y ago
That would require competently produced documentation. Intel can't do that for any of their side projects because their MBAs don't get a bonus if the tech writers are treated as a valuable asset.

throwaway48476 2y ago

Innovation is a bottom up process. If they sell the hardware the community will spring up to take advantage.

bayindirh 2y ago

> It's possible you're underestimating the open source community. On the contrary. You really don't know how I love and prefer open source and love a more leveling playing field. > If there's a competing platform that hobbyists can tinker with... AMD's cards are better from hardware and software architecture standpoint, but the performance is not there yet. Plus, ROCm libraries are not that mature, but they're getting there. Developing high performance, high quality code is deceivingly expensive, because it's very heavy in theory, and you fly very close to the metal. I did that in my Ph.D., so I know what it entails. So it requires more than a couple (hundred) hobbyists to pull off (see the development of Eigen linear algebra library, or any high end math library). Some big guns are pouring money into AMD to implement good ROCm libraries, and it started paying off (Debian has a ton of ROCm packages now, too). However, you need to be able to pull it off in the datacenter to be able to pull it off on the desktop. AMD also needs to be able to enable ROCm on desktop properly, so people can start hacking it at home. > especially when the competing platform is completely closed... NVIDIA gives a lot of support to universities, researchers and institutions who play with their cards. Big cards may not be free, but know-how, support and first steps are always within reach. Plus, their researchers dogfood their own cards, and write papers with them. So, as long as papers got published, researchers do their research, and something got invented, many people don't care about how open source the ecosystem is. This upsets me a ton, but when closed source AI companies and researchers who forget to add crucial details to their papers so what they did can't be reproduced don't care about open source, because they think like NVIDIA. "My research, my secrets, my fame, my money". It's not about sharing. It's about winning, and it's ugly in some aspects.

light_hue_1 2y ago
You're writing as if AMD cares about open source. If they would only actually open source their driver the community would have made their cards better than nvidia ones long ago. I'm one of those academics. You've got it all wrong. So many people care about open source. So many people carefully release their code and make everything reproducible. We desperately just want AMD to open up. They just refuse. There's nothing secret going on and there's no conspiracy. There's just a company that for some inexplicable reason doesn't want to make boatloads of money for free. AMD is the worst possible situation. They're hostile to us and they refuse to invest to make their stuff work.
- talldayo 2y ago
  > If they would only actually open source their driver the community would have made their cards better than nvidia ones long ago. Software wise, maybe. But you can't change AMD's hardware with a magic wand, and that's where a lot of CUDA's optimizations come from. AMD's GPU architecture is optimized for raster compute, and it's been that way for decades. I can assure you that AMD does not have a magic button to press that would make their systems competitive for AI. If that was possible it would have been done years ago, with or without their consent. The problem is deeper and extends to design decisions and disagreement over the complexity of GPU designs. If you compare AMD's cards to Nvidia on "fair ground" (eg. no CUDA, only OpenCL) the GPGPU performance still leans in Nvidia's favor.
  - anticensor 2y ago
    RDNA and UDNA have progressed more towards compute-unified architecture than Vega&CDNA's raster-first architecture.
btown 2y ago
Yep, this thread has a good compilation of ROCm woes: https://news.ycombinator.com/item?id=34832660 https://news.ycombinator.com/item?id=34832660 That said, for hobbyist inference on large pretrained models, I think there is an interesting set of possibilities here: maybe a number of operations aren't optimized, and it takes 10x as long to load the model into memory... but all that might not matter if AMD were to be the first to market for 128GB+ VRAM cards that are the only things that can run next-generation open-weight models in a desktop environment, particularly those generating video and images. The hobbyists don't need to optimize all the linear algebra operations that researchers need to be able to experiment with when training; they just need to implement the ones used by the open-weight models. But of course this is all just wishful thinking, because as others have pointed out, any developments in this direction would require a level of foresight that AMD simply hasn't historically shown.

kevin_thibedeau 2y ago

That would require competently produced documentation. Intel can't do that for any of their side projects because their MBAs don't get a bonus if the tech writers are treated as a valuable asset.