3 ms·

Sarcasm aside... Can we drop the "Nvidia is the only self-interested evil company in existence" schtick? I'm not being "fooled" by anyone. I've been trying to

Sarcasm aside... Can we drop the "Nvidia is the only self-interested evil company in existence" schtick? I'm not being "fooled" by anyone. I've been trying to use ROCm since the initial release six years ago (on Vega at the time). I've spent thousands of dollars on AMD hardware over the years hoping to see progress for myself. I've burned untold amounts of time fighting with ROCm, hoping it's even remotely a viable competitor to CUDA/Nvidia. Here we are in 2024 and they're still doing braindead stuff like dropping a new ROCm release to support their flagship $1000 consumer card a full year after release... ROCm 6 looks good? Check the docker containers[0]. Their initial release for ROCm only supported Python 3.9 for some strange reason even though the previous ROCm 5.7 containers were based on Python 3.10. Python 3.10 is more-or-less the minimum for nearly anything out there. It took them 1.5 months to address this... This is merely one example, spend some time actually working with this and you will find dozens of similar "WTF?!?" bombs all over the place. I suggest you put your money and time where your mouth is (as I have) to actually try to work with ROCm. You will find that it is nowhere near the point of actually being a viable competitor to CUDA/Nvidia for anyone who's trying to get work done. > Tinygrad are doing god's work Tinygrad is packaging hardware with off the shelf components plus substantial markup. There is nothing special about this hardware and they aren't doing anything you couldn't have done in the past year. They have been vocal on calling out AMD but show me their commits to ROCm and I'll agree they are "doing god's work". We'll save the work being done on their framework for another thread. [0] - https://hub.docker.com/r/rocm/pytorch/tags https://hub.docker.com/r/rocm/pytorch/tags

latchkey 3y ago
> There is nothing special about this hardware and they aren't doing anything you couldn't have done in the past year. What they are doing is all of the hardware engineering work that it takes to build something like this. You're dismissing the amount of time they spent on figuring stuff like this out: "Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this."
- kkielhofner 3y ago
  > "Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this." Define "hard". The crypto mining community has had this working for at least half a decade with AMD cards. With Nvidia it's a non-issue. I'd be very, very curious to get more technical details on what new work they did here.
  - latchkey 3y ago
    I ran 150,000 AMD cards for mining and we didn't run into that problem because we bought systems with PCIe baseboards (12x cards) instead of dumb risers. I'd be interested in finding out more details as well, but it seems he doesn't want to share that in public. That said, if you think any of this is easy, you're the one who should define that word.
    - kkielhofner 3y ago
      I never used the word easy, I never used the word hard. He used the word hard, you used the word easy. With that said. Easy: Assembling off the shelf PC components to provide what is fundamentally no different than what gamers/miners build every day. Six cards in a machine and two power supplies is low-end mining. Also see the x8 GPU machines with multiple power supplies that have been around forever. I'm not quite sure why you're arguing this so hard, you're more than familiar with these things. Hard: Show me something with a BOM. Some manufacturing? PCB? Fab? Anything. FWIW for someone that is frequently promoting their startup here you come across as pretty antagonistic. I'm not attacking you, just saying that for someone like myself that has been intrigued by what you're working on it gives me pause in terms of what I'd charitably refer to as potential personality/relationship issues. Everyone has those days, just thought it was worth mentioning.
      - latchkey 3y ago
        I totally get where you're coming from with the mining use case being straightforward. However, when it comes to AI, it's a different ball game. Each use case has its own set of requirements and optimizations, which is why many big mining operations find it challenging to shift towards AI. It's not just about assembling parts; it requires a deeper technical know-how. For mining, the focus is mainly on GPUs, and the specifics like bus speed or other components aren't as critical. You could get by with a basic setup - a $35 CPU, 4GB of RAM, 100meg network, and PXE booting without any local storage. Even older GPUs like the RX470s did the job perfectly until the very end. But what George is working on is something else entirely. It's not just about the number of GPUs; it's about creating a cohesive system where every component plays its part and is configured correctly. This complexity of tying everything together, is what makes it challenging. George is incredibly talented, and the fact that he's been dedicating himself to the tinybox project for a year now really speaks volumes about the intricacies involved. Please don't think I'm trying to be confrontational - that's not my intention in the slightest. I appreciate your perspective, but I'm just trying to offer a different angle based on my own experience in this field. While it might seem that this hardware isn't groundbreaking or that the developments could have been achieved earlier, it's important to recognize the innovation and hard work behind it. This isn't just about putting together existing pieces; it's about creating something that works better as a whole than the sum of its parts. I'm confident that if we were to talk in person, we'd get along just fine.
fragmede 3y ago
isn't tinygrad's value add the software they provide on top of the open source drivers to make it all work? why would should they commit to ROCm if that's the product they're trying to sell?

latchkey 3y ago

> There is nothing special about this hardware and they aren't doing anything you couldn't have done in the past year. What they are doing is all of the hardware engineering work that it takes to build something like this. You're dismissing the amount of time they spent on figuring stuff like this out: "Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this."

kkielhofner 3y ago
> "Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this." Define "hard". The crypto mining community has had this working for at least half a decade with AMD cards. With Nvidia it's a non-issue. I'd be very, very curious to get more technical details on what new work they did here.
- latchkey 3y ago
  I ran 150,000 AMD cards for mining and we didn't run into that problem because we bought systems with PCIe baseboards (12x cards) instead of dumb risers. I'd be interested in finding out more details as well, but it seems he doesn't want to share that in public. That said, if you think any of this is easy, you're the one who should define that word.
  - kkielhofner 3y ago
    I never used the word easy, I never used the word hard. He used the word hard, you used the word easy. With that said. Easy: Assembling off the shelf PC components to provide what is fundamentally no different than what gamers/miners build every day. Six cards in a machine and two power supplies is low-end mining. Also see the x8 GPU machines with multiple power supplies that have been around forever. I'm not quite sure why you're arguing this so hard, you're more than familiar with these things. Hard: Show me something with a BOM. Some manufacturing? PCB? Fab? Anything. FWIW for someone that is frequently promoting their startup here you come across as pretty antagonistic. I'm not attacking you, just saying that for someone like myself that has been intrigued by what you're working on it gives me pause in terms of what I'd charitably refer to as potential personality/relationship issues. Everyone has those days, just thought it was worth mentioning.
    - latchkey 3y ago
      I totally get where you're coming from with the mining use case being straightforward. However, when it comes to AI, it's a different ball game. Each use case has its own set of requirements and optimizations, which is why many big mining operations find it challenging to shift towards AI. It's not just about assembling parts; it requires a deeper technical know-how. For mining, the focus is mainly on GPUs, and the specifics like bus speed or other components aren't as critical. You could get by with a basic setup - a $35 CPU, 4GB of RAM, 100meg network, and PXE booting without any local storage. Even older GPUs like the RX470s did the job perfectly until the very end. But what George is working on is something else entirely. It's not just about the number of GPUs; it's about creating a cohesive system where every component plays its part and is configured correctly. This complexity of tying everything together, is what makes it challenging. George is incredibly talented, and the fact that he's been dedicating himself to the tinybox project for a year now really speaks volumes about the intricacies involved. Please don't think I'm trying to be confrontational - that's not my intention in the slightest. I appreciate your perspective, but I'm just trying to offer a different angle based on my own experience in this field. While it might seem that this hardware isn't groundbreaking or that the developments could have been achieved earlier, it's important to recognize the innovation and hard work behind it. This isn't just about putting together existing pieces; it's about creating something that works better as a whole than the sum of its parts. I'm confident that if we were to talk in person, we'd get along just fine.

fragmede 3y ago

isn't tinygrad's value add the software they provide on top of the open source drivers to make it all work? why would should they commit to ROCm if that's the product they're trying to sell?