6 ms·

And this is the investment case for AMD, models fit entirely in a single chassis, and side benefit: less tariffed network equipment to interconnect compute. Map

And this is the investment case for AMD, models fit entirely in a single chassis, and side benefit: less tariffed network equipment to interconnect compute. Map/reduce instead of clustered compute. Edit: when downvoting, please offer some insight why you disagree

dragonwriter 1y ago
How is the a unique advantage for AMD?
- latchkey 1y ago
  AMD is consistently stacking more HBM. H100 80GB HBM3 H200 141GB HBM3e B200 192GB HBM3e MI300x 192GB HBM3 MI325x 256GB HBM3e MI355x 288GB HBM3e This means that you can fit larger and larger models into a single node, without having to go out over the network. The memory bandwidth on AMD is also quite good.
  - krapht 1y ago
    So the MI300x has 8 different memory domains, and although you can treat it as one flat memory space, if you want to reach their advertised peak memory bandwidth you have to work with it like an 8-socket board.
    - latchkey 1y ago
      Here is a good article on it: https://rocm.blogs.amd.com/software-tools-optimization/compute-memory-modes/README.html https://rocm.blogs.amd.com/software-tools-optimization/compu...
  - ryao 1y ago
    It really does not matter how much memory AMD has if the drivers and firmware are unstable. To give one example from last year: https://www.tomshardware.com/pc-components/gpus/amds-lisa-su-steps-in-to-fix-driver-issues-with-new-tinybox-ai-servers-tiny-corp-calls-for-amd-to-make-its-radeon-7900-xtx-gpu-firmware-open-source https://www.tomshardware.com/pc-components/gpus/amds-lisa-su... They are currently developing their own drivers for AMD hardware because of the headaches that they had with ROCm.
    - latchkey 1y ago
      "driver" is such a generic word. tinygrad works on mi300x. If you want to use it, you can. Negates your point. Additionally, ROCm is a giant collection of a whole bunch of libraries. Certainly there are issues, as with any large collection of software, but the critical thing is whether or not AMD is responsive towards getting things fixed. In the past, it was a huge issue, AMD would routinely ignore developers and bugs would never get fixed. But, after that SA article, Lisa lit a fire under Anush's butt and he's taking ownership. It is a major shift in the entire culture at the company. They are extremely responsive and getting things fixed. You can literally tweet your GH issue to him and someone will respond. What is true a year ago isn't today. If you're paying attention like I am, and experiencing it first hand, things are changing, fast.
      - ryao 1y ago
        I have been hearing this about AMD/ATI drivers for decades. Every year, someone says that it is fixed, only for new evidence to come out that they are not. I have no reason to believe it is fixed given the history. Here is evidence to the contrary: If ROCm actually was in good shape, tinygrad would use it instead of developing their own driver.
        latchkey 1y ago
        We have all been hearing things for decades. Things are noticeably different now. Live in the present, not in the past. Tinygrad isn’t a driver. It is a framework. It is being developed by George however he wants. If he wants to build something that gives him more direct control over things. Fine. Others might write PTX instead if using higher level abstractions. Fact is that tinygrad runs not only on AMD, but also Nvidia and others. You might want to reassess your beliefs because you’re reading into things and coming up with the wrong conclusions.
        ryao 1y ago
        I read tinygrad’s website: https://tinygrad.org/#tinygrad https://tinygrad.org/#tinygrad Under driver quality for AMD, they say “developing” and point to their git repository. If AMD had fixed the issues, they would instead say the driver quality is great and get more sales. They can still get sales even if they are honest about the state of AMD hardware, since they sell Nvidia hardware too, while your company would risk 0 sales if you say anything other than “everything is fine”, since your business is based on leasing AMD GPUs: https://hotaisle.xyz/pricing/ https://hotaisle.xyz/pricing/ Given your enormous conflict of interest, I will listen to what George Hotz and others are saying over what you say on this matter.
  - dragonwriter 1y ago
    MI355X isn't out yet, and the upcoming B300 also has 288GB HBM3e
    - latchkey 1y ago
      June 12th. B300 is Q4 2025. Yes, they keep leapfrogging each other. AMD is still ahead in vram.
rixed 1y ago
> when downvoting, please offer some insight why you disagree And remind that (down)voting is not for (dis)agreement.

dragonwriter 1y ago

How is the a unique advantage for AMD?

latchkey 1y ago
AMD is consistently stacking more HBM. H100 80GB HBM3 H200 141GB HBM3e B200 192GB HBM3e MI300x 192GB HBM3 MI325x 256GB HBM3e MI355x 288GB HBM3e This means that you can fit larger and larger models into a single node, without having to go out over the network. The memory bandwidth on AMD is also quite good.
- krapht 1y ago
  So the MI300x has 8 different memory domains, and although you can treat it as one flat memory space, if you want to reach their advertised peak memory bandwidth you have to work with it like an 8-socket board.
  - latchkey 1y ago
    Here is a good article on it: https://rocm.blogs.amd.com/software-tools-optimization/compute-memory-modes/README.html https://rocm.blogs.amd.com/software-tools-optimization/compu...
- ryao 1y ago
  It really does not matter how much memory AMD has if the drivers and firmware are unstable. To give one example from last year: https://www.tomshardware.com/pc-components/gpus/amds-lisa-su-steps-in-to-fix-driver-issues-with-new-tinybox-ai-servers-tiny-corp-calls-for-amd-to-make-its-radeon-7900-xtx-gpu-firmware-open-source https://www.tomshardware.com/pc-components/gpus/amds-lisa-su... They are currently developing their own drivers for AMD hardware because of the headaches that they had with ROCm.
  - latchkey 1y ago
    "driver" is such a generic word. tinygrad works on mi300x. If you want to use it, you can. Negates your point. Additionally, ROCm is a giant collection of a whole bunch of libraries. Certainly there are issues, as with any large collection of software, but the critical thing is whether or not AMD is responsive towards getting things fixed. In the past, it was a huge issue, AMD would routinely ignore developers and bugs would never get fixed. But, after that SA article, Lisa lit a fire under Anush's butt and he's taking ownership. It is a major shift in the entire culture at the company. They are extremely responsive and getting things fixed. You can literally tweet your GH issue to him and someone will respond. What is true a year ago isn't today. If you're paying attention like I am, and experiencing it first hand, things are changing, fast.
    - ryao 1y ago
      I have been hearing this about AMD/ATI drivers for decades. Every year, someone says that it is fixed, only for new evidence to come out that they are not. I have no reason to believe it is fixed given the history. Here is evidence to the contrary: If ROCm actually was in good shape, tinygrad would use it instead of developing their own driver.
      - latchkey 1y ago
        We have all been hearing things for decades. Things are noticeably different now. Live in the present, not in the past. Tinygrad isn’t a driver. It is a framework. It is being developed by George however he wants. If he wants to build something that gives him more direct control over things. Fine. Others might write PTX instead if using higher level abstractions. Fact is that tinygrad runs not only on AMD, but also Nvidia and others. You might want to reassess your beliefs because you’re reading into things and coming up with the wrong conclusions.
        ryao 1y ago
        I read tinygrad’s website: https://tinygrad.org/#tinygrad https://tinygrad.org/#tinygrad Under driver quality for AMD, they say “developing” and point to their git repository. If AMD had fixed the issues, they would instead say the driver quality is great and get more sales. They can still get sales even if they are honest about the state of AMD hardware, since they sell Nvidia hardware too, while your company would risk 0 sales if you say anything other than “everything is fine”, since your business is based on leasing AMD GPUs: https://hotaisle.xyz/pricing/ https://hotaisle.xyz/pricing/ Given your enormous conflict of interest, I will listen to what George Hotz and others are saying over what you say on this matter.
- dragonwriter 1y ago
  MI355X isn't out yet, and the upcoming B300 also has 288GB HBM3e
  - latchkey 1y ago
    June 12th. B300 is Q4 2025. Yes, they keep leapfrogging each other. AMD is still ahead in vram.

rixed 1y ago

> when downvoting, please offer some insight why you disagree And remind that (down)voting is not for (dis)agreement.