3 ms·

Looks like you can get an MI60 with 32gb vram for ~$500 on ebay - if you're vram limited might be a good option for less than a 3090 seems to be going for, and

by kimixa 3y ago

Looks like you can get an MI60 with 32gb vram for ~$500 on ebay - if you're vram limited might be a good option for less than a 3090 seems to be going for, and seems supported by rocm (being an mi50 just with more hbm). Not sure how well it will perform as it lacks some of the smaller data type acceleration available on newer cores, though.

superkuh 3y ago
AMD has a bad habit of dropping ROCm support for their devices after 4 years. Trying to work with an old dependency chain is something I wouldn't wish on an enemy. I don't know about the MI60 but much of the MI* line has already had support for compute under ROCm dropped.
- kimixa 3y ago
  It's probably not surprising that cheaper options sacrifice some level of software or hardware support - that's why they're cheaper. Software support is why the "pro" versions exist as separate (more expensive) SKUs, after all. Of course you'll get a better service if you pay more. Expecting a high performance, easy, turnkey solution for a low price is expecting to have your cake and eat it. It's never happened on any other technology with this level of demand, and expecting it now seems naive at best. There's too much money (and hype) sloshing around in the sector, and limited supply. Enthusiasts and experimenters have always traded their time for cost - if only because their time investment and learning (and maybe fun) is the whole point, not the end results of their experiments. If you expect financial returns due to your incredible idea, you should be looking for investors not old hardware.
  - brucethemoose2 3y ago
    AMD Instinct cards are very expensive Pro products. The thing is, they are more geared towards HPC/Scientific computing. The only thing that really makes them appealing for ML is that they have somewhat more price depreciation than Nvidia cards and M1 Maxes, for the moment. > Expecting a high performance, easy, turnkey solution for a low price is expecting to have your cake and eat it. But I don't agree with this. This is not a normal "premium" hardware market, its a greedy pseudo monopoly that also dramatically supply constrained at the moment, until maybe next year (as the article points out). Business can pay a premium, but that's different than a lack of competition and supply.
  - PeterisP 3y ago
    Your arguments are technically valid, and explain why AMD is doing this, but the eventual outcome is that for the (many!) non-corporate users of would-be-ML-hwardware you get better service by avoiding AMD/ROCm and buying nVidia hardware instead, because there you do get appropriate software support also for old cards, gaming cards, and old gaming cards And afterwards the AMD community is wondering why researchers and toolmakers have limited support their ecosystem - this is why.
- 65a 3y ago
  Probably officially dropped, but gfx900-series works fine on ROCm 5.7 here, which actually goes back to vega10 (IIRC mi50/mi60 are vega20).
brucethemoose2 3y ago
Its kinda funky, and AMD dropped the MI50 already. See this very interesting reddit discussion: https://old.reddit.com/r/LocalLLaMA/comments/17vcsf9/something_wrong_with_older_amd_high_vram_cards/k9a05wh/ https://old.reddit.com/r/LocalLLaMA/comments/17vcsf9/somethi... Another thing that jumps out: > I also have a couple W6800's and they are actually as fast or faster than the MI100s with the same software... That's insane. The MI100 should be so much faster than the W6800 (a ~6900XT) that its not even funny.
65a 3y ago
I grabbed a 32gb mi60 on a lark, because at the time it was ~$300. It works fine with recent ROCm etc, but it has the following potentially severe drawbacks: 1. It is not a recent GPU, so don't expect blazing inference speeds 2. It is designed for external forced air, so you will need to push air through it somehow 3. The one I got is actually not a normal card, I think it was a firmware testbed, and has no VBIOS and reports its product name as "TBD" It does work and have 32gb of VRAM though.

by kimixa 3y ago

superkuh 3y ago

AMD has a bad habit of dropping ROCm support for their devices after 4 years. Trying to work with an old dependency chain is something I wouldn't wish on an enemy. I don't know about the MI60 but much of the MI* line has already had support for compute under ROCm dropped.

kimixa 3y ago
It's probably not surprising that cheaper options sacrifice some level of software or hardware support - that's why they're cheaper. Software support is why the "pro" versions exist as separate (more expensive) SKUs, after all. Of course you'll get a better service if you pay more. Expecting a high performance, easy, turnkey solution for a low price is expecting to have your cake and eat it. It's never happened on any other technology with this level of demand, and expecting it now seems naive at best. There's too much money (and hype) sloshing around in the sector, and limited supply. Enthusiasts and experimenters have always traded their time for cost - if only because their time investment and learning (and maybe fun) is the whole point, not the end results of their experiments. If you expect financial returns due to your incredible idea, you should be looking for investors not old hardware.
- brucethemoose2 3y ago
  AMD Instinct cards are very expensive Pro products. The thing is, they are more geared towards HPC/Scientific computing. The only thing that really makes them appealing for ML is that they have somewhat more price depreciation than Nvidia cards and M1 Maxes, for the moment. > Expecting a high performance, easy, turnkey solution for a low price is expecting to have your cake and eat it. But I don't agree with this. This is not a normal "premium" hardware market, its a greedy pseudo monopoly that also dramatically supply constrained at the moment, until maybe next year (as the article points out). Business can pay a premium, but that's different than a lack of competition and supply.
- PeterisP 3y ago
  Your arguments are technically valid, and explain why AMD is doing this, but the eventual outcome is that for the (many!) non-corporate users of would-be-ML-hwardware you get better service by avoiding AMD/ROCm and buying nVidia hardware instead, because there you do get appropriate software support also for old cards, gaming cards, and old gaming cards And afterwards the AMD community is wondering why researchers and toolmakers have limited support their ecosystem - this is why.
65a 3y ago
Probably officially dropped, but gfx900-series works fine on ROCm 5.7 here, which actually goes back to vega10 (IIRC mi50/mi60 are vega20).

brucethemoose2 3y ago

Its kinda funky, and AMD dropped the MI50 already. See this very interesting reddit discussion: https://old.reddit.com/r/LocalLLaMA/comments/17vcsf9/something_wrong_with_older_amd_high_vram_cards/k9a05wh/ https://old.reddit.com/r/LocalLLaMA/comments/17vcsf9/somethi... Another thing that jumps out: > I also have a couple W6800's and they are actually as fast or faster than the MI100s with the same software... That's insane. The MI100 should be so much faster than the W6800 (a ~6900XT) that its not even funny.

65a 3y ago

I grabbed a 32gb mi60 on a lark, because at the time it was ~$300. It works fine with recent ROCm etc, but it has the following potentially severe drawbacks: 1. It is not a recent GPU, so don't expect blazing inference speeds 2. It is designed for external forced air, so you will need to push air through it somehow 3. The one I got is actually not a normal card, I think it was a firmware testbed, and has no VBIOS and reports its product name as "TBD" It does work and have 32gb of VRAM though.