11 ms·
AMD GPUs Go Brrr
- avidphantasm 1y agoThis is great, but why does the write-up read like it was written by someone with brain damage?
- gortok 1y agoThe writing is laughably bad. I can’t tell if it’s someone that over relied on AI or if they just mimic the structure and mannerisms of AI produced writing because that’s what they see. A few choice examples: > Checkout part one of this series for an intro to HipKittens and checkout this post for a technical deep dive. > Unsurprisingly, making AMD GPUs go brr boils down to keeping the “matrix cores” (tensor cores on NVIDIA) fed. > These two patterns tradeoff programmability and performance, where 8-wave and its large tile primitives lead to compact code and 4-wave fine-grained interleaving expands code size. Surprisingly, the 8-wave schedule is sufficient to achieve SoTA-level performance on GEMMs and attention forwards. For GQA non-causal attention backwards, 8-wave also outperforms all AMD baselines by 1.8 × 1.8×, and our HK 4-wave further outperforms by 2.3 × 2.3×. And I could go on. And on. But overall besides the overuse of cliche/memespeak places it doesn’t make sense, the entire section that deals with the hot loop describes something that should be explained in a graph and instead explained in 100 lines of source code.
- beepbooptheory 1y agoAm I crazy what is wrong with any of those quotes.
- homarp 1y agosee https://news.ycombinator.com/item?id=45923188 https://news.ycombinator.com/item?id=45923188 for HipKittens discussion
- alex1138 1y agoIt's not my favorite internet meme but I'm tickled to see "go brr" on a website/university like Stanford
- microtonal 1y agoThey already "went brr" when they announced ThunderKittens a year ago: https://hazyresearch.stanford.edu/blog/2024-05-12-tk https://hazyresearch.stanford.edu/blog/2024-05-12-tk
- Skunkleton 1y agoThis meme is tired. Let it rest boss.
- rightbyte 1y agoUsually a sign that it is not cool anymore and the kids need to make something new up.
- DeathArrow 1y agoI think many people tried making AMD GPU go brrr for the mass of the developers but no one succeeded. I don't get why AMD doesn't solve their own software issues. Now they have a lot of money so not having money to pay for developers is not an excuse. And data centers GPUs are not the worst. Using GPU compute for things like running inference at home is a much, much better experience with Nvidia. My 5 years old RTX 3090 is better than any consumer GPU AMD released up to this date, at least for experimenting with ML and AI.
- jacobgorm 1y agoAnd the developer experience is horrible when working with AMD. They don’t even accept driver crash bug reports.
- donaldihunter 1y agoPeople say that as if the Nvidia experience is better. Nvidia also has a horrible developer experience.
- kg 1y agoYMMV but I reported a crash in Nvidia's vulkan driver and they responded promptly and fixed it.
- _zoltan_ 1y agoHuh? I've been developing against the Nvidia ecosystem for years. Just build a container and you're done. They even provide base containers. Anything specific related to DC level computing?
- logicchains 1y ago> Now they have a lot of money so not having money to pay for developers is not an excuse. NVidia is the exception to the rule when it comes to hardware companies paying competitive salaries for software engineers. I imagine AMD is still permeated by the attitude that software "isn't real work" and doesn't deserve more compensation, and that kind of inertia is very hard to overcome.
- skeptrune 1y agoI appreciate that there are people in academia working on this problem, but it seems like something AMD would have to fix internally if they were serious.
- aabhay 1y agoExcept that this same team built a similarly named software package for Nvidia GPUs as well. It’s bright researchers doing what they do best if you ask me.
- sigmoid10 1y agoExcept that this other package also only came out last year and has contributed zero to Nvidia's current status. If AMD ever wants to be taken seriously in this market, they will need to start making their own software good instead of relying on "open source" in the mistaken belief that someone else will fix their bad code for free. Nvidia spent more than a decade hiring top talent and getting their proprietary software environment right before they really took off. And some of the older ML researchers here will certainly remember it wasn't pain-free either. But they didn't just turn the ship around, they turned it into a nuclear aircraft carrier that dominates the entire world.
- stingraycharles 1y agoYeah honestly I’m dumbfounded why all these years AMD still doesn’t have an internal “code red” and get their developer experience up to par with CUDA.
- jstummbillig 1y agoYes. Why is that? Somebody here must have an informed opinion. It seems ludicrous, but also too obvious. What's up?
- stingraycharles 1y ago
- colordrops 1y agoIt's insane to me that AMD is not spending billions and billions trying to fix their software. Nvidia is the most valuable company in the world and AMD is the only one poised to compete.
- david-gpu 1y agoI worked at at a number of GPU vendors, and it felt like Nvidia was the only one that took software as an asset worth investing in, rather than as a cost center. Massively different culture.
- aabhay 1y agoThey are, but the problem is that shifting an organization whose lifeblood is yearly hardware refreshes and chip innovation towards a ship-daily software culture is challenging. And software doesn’t “make money” the way hardware does so it can get deprioritized by executives. And vendors are lining up to write and even open source lots of software for your platform in exchange for pricing, preference, priority (great on paper but bad for long term quality). And your competitors will get ahead of you if you miss even a single hardware trend/innovation.
- keyringlight 1y agoThere was a podcast episode linked here a while ago about how the software industry in Japan never took off as it did in America and it was a similar conclusion. According to the host, the product being sold was hardware, and software was a means to fulfill and then conclude the contract. After that you want the customer to buy the new model, primarily for the hardware and software comes along for the ride. It should be obvious by now though that there's symbiosis between software and hardware, and that support timescales are longer. Another angle is that it's more than just AMD's own software developers, also the developers making products for their customers who in turn buy AMD's if everyone works together to make them run well and it's those second developers they need to engage with in a way their efforts will be welcomed.
- nikanj 1y agoHardware is a profit center, software is a cost center, and they get treated accordingly
- dylanpaker 1y ago[dead]
- badgersnake 1y agoI quite like that the AMD aren’t so popular with the AI bubble. It means I can play games without getting a mortgage.
- x3n0ph3n3 1y agoTheir linux driver support isn't so great, though. I really considered an AMD GPU for my most recent build, and based on the driver support for just the integrated graphics on my new AMD CPU (7900X), I opted for an NVidia card instead.
- mkayokay 1y agoI'm running a 6900XT on Arch and have no problems so far. Steam, Heroic launcher and every game i tried so far worked like a charm. You can even OC with LACT [1] if you want to. [1] https://github.com/ilya-zlobintsev/LACT https://github.com/ilya-zlobintsev/LACT
- esseph 1y agoI have a 9060 in one PC and a 9070 in another, on Fedora 43. It runs great. Run all my steam stuff through them. Those days to mention have been long gone for quite awhile.
- abenga 1y agoHow so? Switching from an Nvidia card to an AMD one I am now able to upgrade my kernel whenever without getting a blinking cursor after reboot. How are in-tree drivers worse than whatever Nvidia does?
- blactuary 1y agoQuite the opposite these days. AMD just works and Nvidia is a crapshoot
- J_Shelby_J 1y agoHow those AMD crashes though. All my friends in AMD CPUs have had a hell of the last two years with constant crashes in unreal engine games. Meanwhile, I made fun of myself for buying an ancient 11 series which is a decade old arch at this point but is rock solid.
- LarsDu88 1y agoFrom this writeup it does sound like the architecture of the AMD gpu makes it a bit harder to optimize. It also seems like long term, the AMD approach may scale better in the long run. 8 chiplets rather than 2 for the nvidia offering, along with all the associated cache and memory locality woes. The future will probably see more chiplets rather than less, so I wonder if dealing with complexity here will pay more dividends in the long run
- WithinReason 1y agoAMD doesn't need warp specialisation for high performance while nvidia does, which simplifies programming AMD
- bsaul 1y agoside question : how is mojo doing in that regard ? i thought their ideas was to improve devX on amds gpu ?
- sorenjan 1y agoSee also this post about the same work: HipKittens: Fast and furious AMD kernels [0], with comments from George Hotz and AMD employees. [0] https://news.ycombinator.com/item?id=45923188 https://news.ycombinator.com/item?id=45923188
- boxerab 1y agoThis is a great project, but the bigger question is: why isn't AMD doing this themselves? It continues to boggle my mind how much they don't seem to get the importance of a mature software stack when it is so obviously the key to the success of team red. A stack that can be used for EVERY card they produce, like CUDA, not just a select few. I used to believe that AMD the underdog would catch up some day, but I've more or less given up on them.