4 ms·

Does this also ship only in x8 batches? I really liked MI300 and could afford one of them for my research, but they only come in batches of x8 in a server rack

Does this also ship only in x8 batches? I really liked MI300 and could afford one of them for my research, but they only come in batches of x8 in a server rack, so I decided to buy an RTX Pro 6000.

latchkey 1y ago
It isn't 8x batches. It is 8 OAMs on a UBB. The UBB is what enables the 8x to communicate with each other over infinity fabric. If you don't need 8, then that's exactly why we offer 1xMI300x VM's.
- ryao 1y ago
  A number of people want to purchase their own hardware, not rent cloud hardware. I recently purchased a RTX PRO 6000 for the same reason, despite having the option of renting a B200 VM for $1.49 an hour from DeepInfra until the end of June.
  - latchkey 1y ago
    True, but as time goes on, it will become a wider and wider gap between what is deployed in DC’s and what you can run at home. We see it now with 8x UBB and it will get worse with direct liquid cooling and larger power requirements. Mi300x is 700w. Mi355 is 1200w. Mi450 will be even more. Certainly amd should make some consumer grade stuff, but they won’t stop on the enterprise side either. Your only option to get super computer level compute, will be to rent it.
    - ryao 1y ago
      The higher power requirements of datacenter GPUs are mostly from running hardware well past a reasonable point on the efficiency curve to eke slightly higher generational increases in performance. Nvidia for example has three versions of the RTX PRO 6000. One runs at 600W while the other two run at 300W. The main differences are the power target and the cooling solutions. If I recall correctly, the performance difference between them is less than 20%, despite a 50% reduction in power consumption, for what is effectively the same hardware. This can be confirmed by changing the power target of the 600W version to 300W and benchmarking the before and after. Plenty of people have made similar observations of AMD’s hardware. That said, I am confident that Nvidia will continue serve those of us who want our own hardware.
      - latchkey 1y ago
        They all bin their chips after testing. 355x is 35% faster than 300x and uses almost 2x more power. But 355 has fp4/6 added in, which until udma comes out, likely won’t get emulated. It is fine if you dont need the features of newer hardware, but if you do… then desktop won’t help you.
        ryao 1y ago
        Your comment does not make much sense to me. The 355x and 300x are two different chips, not binned versions of one another. The RTX PRO 6000 has fp4/fp6 support too, so there is no need to use datacenter exclusive hardware to get that.
        latchkey 1y ago
        I wasn’t referring to them being binned to each other. Oh nice, I didn’t know that about the 6k.
        ryao 1y ago
        The consumer 50 series GPUs also have FP4/FP6 support. Nvidia talks about it a little here: https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf https://images.nvidia.com/aem-dam/Solutions/geforce/blackwel... FP4 is given a full page advertising it while FP6 support in “RTX Blackwell” is a footnote.
jiggawatts 1y ago
Of course not. AMD stubbornly refuses to recognise the huge numbers of low- or medium- budget researchers, hobbyists, and open source developers. This ignorance of how software development is done has resulted in them losing out on a multi-trillion-dollar market. It's incredible to me how obstinate certain segments of the industry (such as hardware design) can be.
- gdiamos 1y ago
  startups and researchers are broke, just like Geoff Hinton in 2006 - https://blog.waqasrana.me/assets/papers/hinton2006.pdf https://blog.waqasrana.me/assets/papers/hinton2006.pdf
  - behnamoh 1y ago
    no we're not broke! we constantly write grants and receive funding from various sources. guess what hardware we recommend the University to purchase? it's 99.9% Nvidia, and sometimes Mac Studio just to play with MLX.
    - gdiamos 1y ago
      I mean broke compared to Meta or OpenAI. AI research used to be fringe and not well funded. Back in those days, 99.9% of hardware was Xeon.
      - ryao 1y ago
        It has gone through many boom and busy cycles. If you go far back enough, it was very well funded. In particular, I recall reading about the US government investing 1 to 2 billion dollars during the Cold War into AI research to translate Russian into English. It had some very impressive demos on preselected Russian texts that had justified the investments. However, it failed to yield results on arbitrary texts. The translation problem has only been mostly solved in recent years.
- rfv6723 1y ago
  These ppl are very loud online, but they don't make decisions for hyperscalers which are biggest spenders on AI chips. AMD is doing just fine, Oracle just announced an AI cluster with up to 131,072 of AMD's new MI355X GPUs. AMD needs to focus on bringing rack-scale mi400 as quickly as possible to market, rather than those hobbyists always find something to complain instead of spending money.

Does this also ship only in x8 batches? I really liked MI300 and could afford one of them for my research, but they only come in batches of x8 in a server rack, so I decided to buy an RTX Pro 6000.

latchkey 1y ago

It isn't 8x batches. It is 8 OAMs on a UBB. The UBB is what enables the 8x to communicate with each other over infinity fabric. If you don't need 8, then that's exactly why we offer 1xMI300x VM's.

ryao 1y ago
A number of people want to purchase their own hardware, not rent cloud hardware. I recently purchased a RTX PRO 6000 for the same reason, despite having the option of renting a B200 VM for $1.49 an hour from DeepInfra until the end of June.
- latchkey 1y ago
  True, but as time goes on, it will become a wider and wider gap between what is deployed in DC’s and what you can run at home. We see it now with 8x UBB and it will get worse with direct liquid cooling and larger power requirements. Mi300x is 700w. Mi355 is 1200w. Mi450 will be even more. Certainly amd should make some consumer grade stuff, but they won’t stop on the enterprise side either. Your only option to get super computer level compute, will be to rent it.
  - ryao 1y ago
    The higher power requirements of datacenter GPUs are mostly from running hardware well past a reasonable point on the efficiency curve to eke slightly higher generational increases in performance. Nvidia for example has three versions of the RTX PRO 6000. One runs at 600W while the other two run at 300W. The main differences are the power target and the cooling solutions. If I recall correctly, the performance difference between them is less than 20%, despite a 50% reduction in power consumption, for what is effectively the same hardware. This can be confirmed by changing the power target of the 600W version to 300W and benchmarking the before and after. Plenty of people have made similar observations of AMD’s hardware. That said, I am confident that Nvidia will continue serve those of us who want our own hardware.
    - latchkey 1y ago
      They all bin their chips after testing. 355x is 35% faster than 300x and uses almost 2x more power. But 355 has fp4/6 added in, which until udma comes out, likely won’t get emulated. It is fine if you dont need the features of newer hardware, but if you do… then desktop won’t help you.
      - ryao 1y ago
        Your comment does not make much sense to me. The 355x and 300x are two different chips, not binned versions of one another. The RTX PRO 6000 has fp4/fp6 support too, so there is no need to use datacenter exclusive hardware to get that.
        latchkey 1y ago
        I wasn’t referring to them being binned to each other. Oh nice, I didn’t know that about the 6k.
        ryao 1y ago
        The consumer 50 series GPUs also have FP4/FP6 support. Nvidia talks about it a little here: https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf https://images.nvidia.com/aem-dam/Solutions/geforce/blackwel... FP4 is given a full page advertising it while FP6 support in “RTX Blackwell” is a footnote.

jiggawatts 1y ago

Of course not. AMD stubbornly refuses to recognise the huge numbers of low- or medium- budget researchers, hobbyists, and open source developers. This ignorance of how software development is done has resulted in them losing out on a multi-trillion-dollar market. It's incredible to me how obstinate certain segments of the industry (such as hardware design) can be.

gdiamos 1y ago
startups and researchers are broke, just like Geoff Hinton in 2006 - https://blog.waqasrana.me/assets/papers/hinton2006.pdf https://blog.waqasrana.me/assets/papers/hinton2006.pdf
- behnamoh 1y ago
  no we're not broke! we constantly write grants and receive funding from various sources. guess what hardware we recommend the University to purchase? it's 99.9% Nvidia, and sometimes Mac Studio just to play with MLX.
  - gdiamos 1y ago
    I mean broke compared to Meta or OpenAI. AI research used to be fringe and not well funded. Back in those days, 99.9% of hardware was Xeon.
    - ryao 1y ago
      It has gone through many boom and busy cycles. If you go far back enough, it was very well funded. In particular, I recall reading about the US government investing 1 to 2 billion dollars during the Cold War into AI research to translate Russian into English. It had some very impressive demos on preselected Russian texts that had justified the investments. However, it failed to yield results on arbitrary texts. The translation problem has only been mostly solved in recent years.
rfv6723 1y ago
These ppl are very loud online, but they don't make decisions for hyperscalers which are biggest spenders on AI chips. AMD is doing just fine, Oracle just announced an AI cluster with up to 131,072 of AMD's new MI355X GPUs. AMD needs to focus on bringing rack-scale mi400 as quickly as possible to market, rather than those hobbyists always find something to complain instead of spending money.