3 ms·

There's more to machine learning and AI than just training the latest LLM's though. Speaking for myself, I am very interested in supporting AMD and the ROCm eco

There's more to machine learning and AI than just training the latest LLM's though. Speaking for myself, I am very interested in supporting AMD and the ROCm ecosystem, whatever AMD's past sins may be. I'm building a machine now which will be based on a high-end consumer GPU from AMD. Not going for something like this as it's almost certainly way out of my budget, but perhaps in the future. So basically, I'm betting that AMD has had (or is having) a change of heart and is genuinely committed to AI/ML on their products. Time may ultimately prove me wrong, but so be it if that proves to be the case. And FWIW, one reason for my optimism is that, whatever you think about the state of ROCm today, they are clearly investing heavily into the platform and continually working on it. You can see that just from looking at the activity on their Github repos: https://github.com/orgs/ROCmSoftwarePlatform/repositories https://github.com/orgs/ROCmSoftwarePlatform/repositories There is constant activity and has been for some time, which I take as a good sign. Yes, it's just one signal among many that one could consider, but I think it's an important one.

zwaps 3y ago
Yeah but you see, we have been here before. "This time we are serious about ML/AI..." And if you went and bought an AMD card then, you'd have been wrong. My example about LLMs was just to show that AMD is simply not part of the conversation. Three month before you could have made the same point about another approach. And still, if you'd actually have to risk money, you and me both know you'd never invest in AMD hardware for AI or start developing on it something high stakes. I mean, look at geohot he tried and just gave up entirely and AMD.
- mindcrime 3y ago
  And still, if you'd actually have to risk money, you and me both know you'd never invest in AMD hardware for AI or start developing on it something high stakes. I assume you meant "you" in the royal sense. Since I personally am, in point of fact, investing in AMD hardware for AI. Yeah, it's a gamble, but that's my style. And I have to admit, part of it is driven by ideological reasons (ROCm being open source) and a simple desire to support AMD since I want them to become a serious competitor to NVIDIA. I believe that's an outcome that would benefit everybody. I mean, look at geohot he tried and just gave up entirely and AMD. I have to admit, I don't find that particularly compelling in any regard. Nothing against geohot, he's clearly a smart dude, but... I'm not judging a company based on his interactions with them. shrug
  - zwaps 3y ago
    I wish you luck then. Right now, you need CUDA to run things like bf16 or quantization, which are truly enablers in this space. So for now, I don't see AMD getting any traction. And apparently, the quality of the drivers has not been improving. Time will tell.
    - mindcrime 3y ago
      Agreed. And FWIW, I'm not saying I use AMD kit exclusively. I'm fine with using NVIDIA hardware and CUDA when needed. In addition to the AMD based box I'm building for the lab at home, I'm also building a sister box which will be nearly identical, except using an NVIDIA GPU. I am an ideologue but I'm also at least a little bit pragmatic. :-)
jhanschoo 3y ago
Speaking for yourself, what kind of workloads have you planned on running on ROCm when your machine is built?
- mindcrime 3y ago
  Tough to say, as my interests are pretty spread around. Definitely some fairly vanilla neural networks in different domains, as a lot of my interest is in multi-modal AI. So I'll be working on combining vision, audio, language and "other" where "other" corresponds to things that aren't senses we humans have but which my "AI bot" does - GPS, temperature, altitude, barometric pressure, etc. Beyond that, it's just going to be wherever the research takes me. And not everything is necessarily going to be particularly suited for running on ROCm. I'll use CUDA on NVIDIA hardware as and when needed, and I'm also open to dabbling in low-level hardware stuff and playing around with DSP's, analog computing, FPGA's, etc. One thing I want to work with a bit is some Spiking Neural Network stuff, neuromorphic approaches, etc. So not sure where this will end up. Heck, it's possible that some of the stuff I work with will turn out to be well suited for running on a plain old CPU, which is one reason I spec's a relatively high-end Ryzen CPU and 64GB of system RAM for these machines, which otherwise wouldn't necessarily be all that important for "pure" GPU computation. Maybe I'll even wind up going "old school" and building myself a Beowulf cluster! So I can finally answer the age old question about "imagine a Beowulf cluster of these..."

zwaps 3y ago

Yeah but you see, we have been here before. "This time we are serious about ML/AI..." And if you went and bought an AMD card then, you'd have been wrong. My example about LLMs was just to show that AMD is simply not part of the conversation. Three month before you could have made the same point about another approach. And still, if you'd actually have to risk money, you and me both know you'd never invest in AMD hardware for AI or start developing on it something high stakes. I mean, look at geohot he tried and just gave up entirely and AMD.

mindcrime 3y ago
And still, if you'd actually have to risk money, you and me both know you'd never invest in AMD hardware for AI or start developing on it something high stakes. I assume you meant "you" in the royal sense. Since I personally am, in point of fact, investing in AMD hardware for AI. Yeah, it's a gamble, but that's my style. And I have to admit, part of it is driven by ideological reasons (ROCm being open source) and a simple desire to support AMD since I want them to become a serious competitor to NVIDIA. I believe that's an outcome that would benefit everybody. I mean, look at geohot he tried and just gave up entirely and AMD. I have to admit, I don't find that particularly compelling in any regard. Nothing against geohot, he's clearly a smart dude, but... I'm not judging a company based on his interactions with them. shrug
- zwaps 3y ago
  I wish you luck then. Right now, you need CUDA to run things like bf16 or quantization, which are truly enablers in this space. So for now, I don't see AMD getting any traction. And apparently, the quality of the drivers has not been improving. Time will tell.
  - mindcrime 3y ago
    Agreed. And FWIW, I'm not saying I use AMD kit exclusively. I'm fine with using NVIDIA hardware and CUDA when needed. In addition to the AMD based box I'm building for the lab at home, I'm also building a sister box which will be nearly identical, except using an NVIDIA GPU. I am an ideologue but I'm also at least a little bit pragmatic. :-)

jhanschoo 3y ago

Speaking for yourself, what kind of workloads have you planned on running on ROCm when your machine is built?

mindcrime 3y ago
Tough to say, as my interests are pretty spread around. Definitely some fairly vanilla neural networks in different domains, as a lot of my interest is in multi-modal AI. So I'll be working on combining vision, audio, language and "other" where "other" corresponds to things that aren't senses we humans have but which my "AI bot" does - GPS, temperature, altitude, barometric pressure, etc. Beyond that, it's just going to be wherever the research takes me. And not everything is necessarily going to be particularly suited for running on ROCm. I'll use CUDA on NVIDIA hardware as and when needed, and I'm also open to dabbling in low-level hardware stuff and playing around with DSP's, analog computing, FPGA's, etc. One thing I want to work with a bit is some Spiking Neural Network stuff, neuromorphic approaches, etc. So not sure where this will end up. Heck, it's possible that some of the stuff I work with will turn out to be well suited for running on a plain old CPU, which is one reason I spec's a relatively high-end Ryzen CPU and 64GB of system RAM for these machines, which otherwise wouldn't necessarily be all that important for "pure" GPU computation. Maybe I'll even wind up going "old school" and building myself a Beowulf cluster! So I can finally answer the age old question about "imagine a Beowulf cluster of these..."