6 ms·

Well AMD really needs to work on its software stack to support AI.

dpflan 3y ago
Indeed, are you aware of the current state of AMD's equivalent to CUDA? And how far behind would be, and what would need to be done to near equivalence? It seems like the opportunity and differentiator. I wonder how Apple silicon software is doing...
- llm_nerd 3y ago
  Instead of trying to integrate the whole stack of, say, pytorch, Apple's primary approach has been converting models to work with Apple's stack. https://github.com/apple/coremltools https://github.com/apple/coremltools Clearly no one is going to be doing training or even fine tuning on Apple hardware at any scale (it competes at the low end, but at scale you invariably will be using nvidia hardware), but once you have a decent model it's a robust way of using it on Apple devices.
  - coredog64 3y ago
    I’ve tried a few models and none have worked. It’s not that they need more resources, just that it freezes and then dies with an inscrutable stack trace somewhere in the OS. If someone from AMD sees the parent comment, please don’t copy Apple!
    - dpflan 3y ago
      Mind sharing which ones and your approach? Haven't there been shared posts of success to follow? Also, yes, anyone from AMD, I think we all want more variety in the market, what do you need help with?
    - llm_nerd 3y ago
      Not every model feature and op is supported (though it is growing with every release), but personally I've had surprising success with it. It has allowed me to leverage some models efficiently on both Intel, Apple Silicon and iPhone/iPad devices. I mean...the number of people using PyTorch models ported to CoreML is probably several magnitude greater than the number of people actually having success with AMD's initiatives in the space thus far...
polski-g 3y ago
ATI drivers have been terrible for over 20 years.
- bryanlarsen 3y ago
  But on Linux they're superior to NVidia's closed source garbage.
  - angry_octet 3y ago
    How so? You mean graphics or compute? The whole concept of the way graphics drivers works is bonkers, thanks to IP issues and game patching, but in production the NVIDIA drivers are very reliable and sophisticated.
    - bryanlarsen 3y ago
      From my perspective as an admin of several GPU clusters.
      - angry_octet 3y ago
        You have GPU clusters with AMD devices?
- fulafel 3y ago
  They were at some point but they're better than NVidia now for normal desktop etc use at least on Linux. They went open source and upstreamed the drivers, like Intel. On the GPGPU stack front it may be different but CUDA is also really low level and abstracted away by ML stacks. And some of them also had OpenCL / SYCL backends at one point, I wonder what's the story there.
  - mikepavone 3y ago
    Unfortunately, the ROCm stack seems to use a fair bit of separate kernel code and it's not nearly as stable in my experience. I have a 5700 XT (not officially supported by ROCm, but close enough to some other cards that some have had success). It's perfectly stable for demanding games, but just straightforward DMA is enough to cause GPU hangs with ROCm for some reason.
moffkalast 3y ago
If AMD throws twice the VRAM onto their cards they'd be a no brainer for that regardless. OpenCL is pretty well supported now.
- paulmd 3y ago
  AMD is trying to get people to pay $5000 for a workstation card too. The days of Radeon VII being $699 are long past. Part of it is that as you shrink, the PHYs don't shrink much, so everyone is incentivized to minimize the number of memory channels and reduce the PCIe bus size/etc on lower tier products. And in turn, since GDDR6 tops out at 16 Gbit (=2 Gigabyte) per chip, that means a 4-PHY card tops out at 8GB, etc. And while you can do clamshell... you want to be selling those cards to workstation users, not giving them away to gamers! It is the same problem NVIDIA faces, the fact that GDDR density has not increased leaves them with one single move (clamshell) and they've traditionally reserved that for workstation cards (and 3090) to increase margins.
  - williamDafoe 3y ago
    Rx 7900xt is $760 and it's an incredible bargain now, IMHO ...
    - paulmd 3y ago
      Yeah if you don't need CUDA the 7900XT is better right now than people give it credit for. People are super mega butthurt about prices right now, to probably an unreasonable degree, and they're ignoring some of the actually decent options that exist. 4070 at $600 (rip microcenter steam GC deal) is a pretty ok deal too, much better than people give it credit for. For less than a 6950XT you get 4GB less VRAM but it pulls 200W less power which is very noticeable, and gets DLSS2 (even if you don't like DLSS3!) which is significantly better at 1440p and 1080p, which is a big perf and perf/w boost, with better quality than FSR2. Even HUB now likes the 4070 over 6900XT for a generalist kinda build: https://youtu.be/Iy3ikm8MxOM?t=875 https://youtu.be/Iy3ikm8MxOM?t=875 Or yes, the 7900XT on top, the launch MSRP sucked but $700-750 is OK for what you get. If you are otherwise getting 6900XT/6950XT I'd probably just recommend spending up and getting the 7900XT (or 4070), they really are a lot more efficient and have better featureset etc. It'll be worth it, suck it up and do it. 6800XT makes sense at like $450, that'd give it space vs the 4070, but people are getting irrational over the whole situation. 3090 is also an underappreciated competitor. What if there was a 3080 Ti 24GB, with DLSS2 and stuff but not $1600? There is, it's $700 on ebay. However, I am uncomfortable with the VRAM on the back with how hot GDDR6X runs, a lot of those cards mined for a lot of years... if you can get 3090 Ti it has only VRAM on the front, or get an evga one, or something. It's ok, at $700 it's a similar proposition to 6900XT/6950XT at $600 but a bit more VRAM and you get DLSS2. 6700/6700XT for $300 is a screamer of a deal and it's not really surprising AMD can't beat it. They're in the "1080 Ti vs 2070" situation, they over-cut on the old stuff and the new stuff can't really edge past it nor is the value great against a deeply cutdown older card (6700XT is a $480 card for $300!). It's not gonna last forever, if that's the featureset you want I'd consider buying. There will probably be a decent replacement eventually but that's clearly the value peak of what they can do with RDNA2 and it makes RDNA3 look poor in many ways. The successor in this price segment is, best case imo, 7600 16GB at $299-329, and that's a bit slower and more limited in a lot of ways, and not really more efficient either. N32 may not really compete favorably with it in either cost or perf/mm2. 4060 Ti is junk but 4060 8GB at $299 for basically 3060 Ti performance (4060 Ti is closer to 3070 at 1080p and 1440p) is reasonable imo, and DLSS will put it clearly over the top of the 7600 at 1080p (FSR sucks). I'm guessing AMD has to get $279 down to $249 or $229 by the time the 4060 launches. I'm guessing 7600 16GB will probably launch at $329 and drop to $299, and that'll be a decent option vs 4060 8GB too (and cost viable). Probably there will be a 7500XT 8GB or 6GB cutdown at $199 (12GB at $250?), doubt they can go too much below that (die cost isn't the problem/cutdowns don't help memory bus size). It'll be a bit slower than a 6700XT for sure, and not all that much more efficient, and you go down to a PCIe x8 bus, but it does have some newer stuff. I think the 16GB versions of both 4060 and 4060 Ti are DOA, the 4070 is a lot faster and has enough. 6800XT is potentially still compelling in this segment too. AMD really needs to figure their shit out with the N32 die though. They actually do need a competitor to 4070 besides just RDNA2 rebrands. I think with the unexpectedly (apparently) poor performance of RDNA3 it's just not worth it, like I just haven't heard any rumor mill shit about N32 at all. The MCDs alone would use as much 6nm silicon as a whole 7600 and then you have a chunk of N5P too. With how short wafers are for other products (IO dies, Epyc, N31, etc) it may just not be worth bringing N32 to market. Who knows, but, I'm getting more and more curious. Numbers: https://www.reddit.com/r/hardware/comments/13vm5ti/geforce_rtx_4060_ti_radeon_rx_7600_meta_review/ https://www.reddit.com/r/hardware/comments/13vm5ti/geforce_r... Tim from HUB talks candidly about FSR: https://www.youtube.com/watch?v=ycXkvVfc2yw&t=2541s https://www.youtube.com/watch?v=ycXkvVfc2yw&t=2541s (the quality difference at 1080p and 1440p is significant, and that's where $200-300 cards will be running. Even in the 4070 segment... starting to become an issue for 7900XT etc)

Indeed, are you aware of the current state of AMD's equivalent to CUDA? And how far behind would be, and what would need to be done to near equivalence? It seems like the opportunity and differentiator. I wonder how Apple silicon software is doing...

llm_nerd 3y ago
Instead of trying to integrate the whole stack of, say, pytorch, Apple's primary approach has been converting models to work with Apple's stack. https://github.com/apple/coremltools https://github.com/apple/coremltools Clearly no one is going to be doing training or even fine tuning on Apple hardware at any scale (it competes at the low end, but at scale you invariably will be using nvidia hardware), but once you have a decent model it's a robust way of using it on Apple devices.
- coredog64 3y ago
  I’ve tried a few models and none have worked. It’s not that they need more resources, just that it freezes and then dies with an inscrutable stack trace somewhere in the OS. If someone from AMD sees the parent comment, please don’t copy Apple!
  - dpflan 3y ago
    Mind sharing which ones and your approach? Haven't there been shared posts of success to follow? Also, yes, anyone from AMD, I think we all want more variety in the market, what do you need help with?
  - llm_nerd 3y ago
    Not every model feature and op is supported (though it is growing with every release), but personally I've had surprising success with it. It has allowed me to leverage some models efficiently on both Intel, Apple Silicon and iPhone/iPad devices. I mean...the number of people using PyTorch models ported to CoreML is probably several magnitude greater than the number of people actually having success with AMD's initiatives in the space thus far...

polski-g 3y ago

ATI drivers have been terrible for over 20 years.

bryanlarsen 3y ago
But on Linux they're superior to NVidia's closed source garbage.
- angry_octet 3y ago
  How so? You mean graphics or compute? The whole concept of the way graphics drivers works is bonkers, thanks to IP issues and game patching, but in production the NVIDIA drivers are very reliable and sophisticated.
  - bryanlarsen 3y ago
    From my perspective as an admin of several GPU clusters.
    - angry_octet 3y ago
      You have GPU clusters with AMD devices?
fulafel 3y ago
They were at some point but they're better than NVidia now for normal desktop etc use at least on Linux. They went open source and upstreamed the drivers, like Intel. On the GPGPU stack front it may be different but CUDA is also really low level and abstracted away by ML stacks. And some of them also had OpenCL / SYCL backends at one point, I wonder what's the story there.
- mikepavone 3y ago
  Unfortunately, the ROCm stack seems to use a fair bit of separate kernel code and it's not nearly as stable in my experience. I have a 5700 XT (not officially supported by ROCm, but close enough to some other cards that some have had success). It's perfectly stable for demanding games, but just straightforward DMA is enough to cause GPU hangs with ROCm for some reason.

moffkalast 3y ago

If AMD throws twice the VRAM onto their cards they'd be a no brainer for that regardless. OpenCL is pretty well supported now.

paulmd 3y ago
AMD is trying to get people to pay $5000 for a workstation card too. The days of Radeon VII being $699 are long past. Part of it is that as you shrink, the PHYs don't shrink much, so everyone is incentivized to minimize the number of memory channels and reduce the PCIe bus size/etc on lower tier products. And in turn, since GDDR6 tops out at 16 Gbit (=2 Gigabyte) per chip, that means a 4-PHY card tops out at 8GB, etc. And while you can do clamshell... you want to be selling those cards to workstation users, not giving them away to gamers! It is the same problem NVIDIA faces, the fact that GDDR density has not increased leaves them with one single move (clamshell) and they've traditionally reserved that for workstation cards (and 3090) to increase margins.
- williamDafoe 3y ago
  Rx 7900xt is $760 and it's an incredible bargain now, IMHO ...
  - paulmd 3y ago
    Yeah if you don't need CUDA the 7900XT is better right now than people give it credit for. People are super mega butthurt about prices right now, to probably an unreasonable degree, and they're ignoring some of the actually decent options that exist. 4070 at $600 (rip microcenter steam GC deal) is a pretty ok deal too, much better than people give it credit for. For less than a 6950XT you get 4GB less VRAM but it pulls 200W less power which is very noticeable, and gets DLSS2 (even if you don't like DLSS3!) which is significantly better at 1440p and 1080p, which is a big perf and perf/w boost, with better quality than FSR2. Even HUB now likes the 4070 over 6900XT for a generalist kinda build: https://youtu.be/Iy3ikm8MxOM?t=875 https://youtu.be/Iy3ikm8MxOM?t=875 Or yes, the 7900XT on top, the launch MSRP sucked but $700-750 is OK for what you get. If you are otherwise getting 6900XT/6950XT I'd probably just recommend spending up and getting the 7900XT (or 4070), they really are a lot more efficient and have better featureset etc. It'll be worth it, suck it up and do it. 6800XT makes sense at like $450, that'd give it space vs the 4070, but people are getting irrational over the whole situation. 3090 is also an underappreciated competitor. What if there was a 3080 Ti 24GB, with DLSS2 and stuff but not $1600? There is, it's $700 on ebay. However, I am uncomfortable with the VRAM on the back with how hot GDDR6X runs, a lot of those cards mined for a lot of years... if you can get 3090 Ti it has only VRAM on the front, or get an evga one, or something. It's ok, at $700 it's a similar proposition to 6900XT/6950XT at $600 but a bit more VRAM and you get DLSS2. 6700/6700XT for $300 is a screamer of a deal and it's not really surprising AMD can't beat it. They're in the "1080 Ti vs 2070" situation, they over-cut on the old stuff and the new stuff can't really edge past it nor is the value great against a deeply cutdown older card (6700XT is a $480 card for $300!). It's not gonna last forever, if that's the featureset you want I'd consider buying. There will probably be a decent replacement eventually but that's clearly the value peak of what they can do with RDNA2 and it makes RDNA3 look poor in many ways. The successor in this price segment is, best case imo, 7600 16GB at $299-329, and that's a bit slower and more limited in a lot of ways, and not really more efficient either. N32 may not really compete favorably with it in either cost or perf/mm2. 4060 Ti is junk but 4060 8GB at $299 for basically 3060 Ti performance (4060 Ti is closer to 3070 at 1080p and 1440p) is reasonable imo, and DLSS will put it clearly over the top of the 7600 at 1080p (FSR sucks). I'm guessing AMD has to get $279 down to $249 or $229 by the time the 4060 launches. I'm guessing 7600 16GB will probably launch at $329 and drop to $299, and that'll be a decent option vs 4060 8GB too (and cost viable). Probably there will be a 7500XT 8GB or 6GB cutdown at $199 (12GB at $250?), doubt they can go too much below that (die cost isn't the problem/cutdowns don't help memory bus size). It'll be a bit slower than a 6700XT for sure, and not all that much more efficient, and you go down to a PCIe x8 bus, but it does have some newer stuff. I think the 16GB versions of both 4060 and 4060 Ti are DOA, the 4070 is a lot faster and has enough. 6800XT is potentially still compelling in this segment too. AMD really needs to figure their shit out with the N32 die though. They actually do need a competitor to 4070 besides just RDNA2 rebrands. I think with the unexpectedly (apparently) poor performance of RDNA3 it's just not worth it, like I just haven't heard any rumor mill shit about N32 at all. The MCDs alone would use as much 6nm silicon as a whole 7600 and then you have a chunk of N5P too. With how short wafers are for other products (IO dies, Epyc, N31, etc) it may just not be worth bringing N32 to market. Who knows, but, I'm getting more and more curious. Numbers: https://www.reddit.com/r/hardware/comments/13vm5ti/geforce_rtx_4060_ti_radeon_rx_7600_meta_review/ https://www.reddit.com/r/hardware/comments/13vm5ti/geforce_r... Tim from HUB talks candidly about FSR: https://www.youtube.com/watch?v=ycXkvVfc2yw&t=2541s https://www.youtube.com/watch?v=ycXkvVfc2yw&t=2541s (the quality difference at 1080p and 1440p is significant, and that's where $200-300 cards will be running. Even in the 4070 segment... starting to become an issue for 7900XT etc)