5 ms·
AMD Instinct MI300 Data Center APU – 146B Transistors, Shipping H2’23
- pclmulqdq 4y agoThat is amazing and incredible, but still won't make up for the fact that rocm is a garbage fire and opencl is slow. Nvidia has more than good hardware.
- buildbot 4y agoRoCM works okay for me in casual use, what issues do you have?
- posguy 4y agoROCm's software stack runs on what feels like Ubuntu LTS only (good luck on Debian or derivatives!), you need very specific PyTorch versions that work with ROCm (no running the bleeding edge easily), and running an old UEFI version (eg: 2 or more years old) often causes a no boot/no POST situation. ROCm is a fragile stack that works on a very small subset of AMD's hardware with 3 blessed Linux distros. It doesn't have to be this way, most APUs and GPUs from AMD should be able to run ROCm even on Debian, Nix, or a distro of the user's choosing. Thankfully some Instinct hardware is super underpriced, so if your willing to make the effort you can run things really fast for rather cheap :D
- pclmulqdq 4y agoAnd God help you if you want to program in actual ROCm rather than using a library like PyTorch or BLAS. There are basically no good manuals or usage guides, and the accompanying tooling is much harder to use than the CUDA equivalents (if they exist at all). On top of that, nobody has any examples outside of major libraries or HPC code (which were largely written by developers at AMD).
- ioedward 4y agoWhere did you buy the Instinct hardware and what's a good price?
- pclmulqdq 4y agoWhen I tried it, ebay was the place to go, and the price was about 1/10th the nvidia equivalent when comparing computing power. You pay for it in time instead.
- incrudible 4y agoMaybe comparing raw specs, not accounting for software running slower, if at all. You pay for it in time, but also sanity.
- mappu 4y agoHIP/ROCm has been packaged in debian - all I had to do was `apt install libamdhip64-{5,dev}; pip install torch` and then Stable Diffusion would run, on amdgpu 6600XT, with no third party repos at all. It should be fine for AMD's packages to only work on their own blessed distros, that's normal behaviour for upstream. Then downstream distros will then repackage it to make it a good citizen in their distro.
- makomk 4y agoThose packages are only in Debian testing, and I think RDNA2 GPUs are the only consumer-level GPUs with reasonably good support in ROCm (and then probably only some of them). AMD also dragged their feet for a long time on supporting RDNA after it launched, and actually managed to break support for the only consumer GPUs that worked reasonably well with ROCm before that (the RX570 and similar) quite a while before they added support for RDNA.
- buildbot 4y agoNot discounting any of that, but it seems like a similar situation to early versions of cuda. Those really sucked to install.
- teruakohatu 4y agoI would argue cuda is still painful when you find yourself in some sort of edge case... The thought of using a stack that is worse gives me the creeps.
- pabs3 4y agoDebian bookworm will ship with a few bits of ROCm, and there are more things coming https://qa.debian.org/developer.php?email=debian-ai%40lists.debian.org https://qa.debian.org/developer.php?email=debian-ai%40lists.... https://tracker.debian.org/teams/rocm-team/ https://tracker.debian.org/teams/rocm-team/ https://salsa.debian.org/rocm-team https://salsa.debian.org/rocm-team https://lists.debian.org/debian-ai/ https://lists.debian.org/debian-ai/
- gnufx 4y ago> runs on what feels like Ubuntu LTS only It surely runs in the Cray environment (I assume still based on SuSE) on Frontier. There is ROCm packaging in Fedora/EPEL, provided by an AMD employee, but I don't know anything about it.
- brucethemoose2 4y agoArch is packaging ROCm natively in testing, and PyTorch has official-ish builds of it now (and maybe Arch will build it too). But you should also check out MLIR which, for instance, is used very sucessfully here and is benching way faster than rocm: https://github.com/nod-ai/SHARK/ https://github.com/nod-ai/SHARK/
- bayindirh 4y agoDebian also has a lot of ROCm related packages from base library to software packages. Looks like AMD is in a push for making it widely available and easy to work with.
- pclmulqdq 4y agoI think we should clarify that this push is for making it easy to use, not to develop. They are trying to capture the python-based ML community, but care a lot less about people who actually want to write code in ROCm than Nvidia cares about CUDA coders.
- brucethemoose2 4y agoAnd the datacenter cards (which are even less accessible than an A100) are still the first class citizens, while CUDA performance on the desktop cards is quite excellent.
- sitkack 4y agoAre you saying OpenCl is architecturally slow or a particular implementation is slow? Moritz is able to extract theoretical peak hardware perf out of his OpenCl code.
- buildbot 4y ago146B transistors - we are probably only a few years away from 1 trillion in a single design. Pretty awe inspiring.
- glowingly 4y agoIf MI300 counts at 146 billion transistors, than I think we can count Cerebras' Wafer Scale Engine 2. It sits at 2.6 trillion transistors. https://www.anandtech.com/show/16626/cerebras-unveils-wafer-scale-engine-two-wse2-26-trillion-transistors-100-yield https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...
- baybal2 4y agoFlash memory chips have passed 1 trillion long time ago. The first one was a 512GB Samsung flash chip
- rektide 4y agoSure there's a ton of compute chips, gobs of HBM... but honestly the super interesting weird challenge here, besides making this monster boot, is connecting all the chips. Will CPU & GPU chiplets talk with each other? Or will all communication travel through glue chips like the IO Die? Where does the hbm attach? Is there ram too? On RDNA3, AMD has 6 small Memory Channel Die (and a huge Graphic Compute Die in the center)... will we see similar with HBM be attached to individual GCD?
- DeepYogurt 4y agoGiven that AMD has been making combined cpu+gpus for over a decade I think it's safe to assume that the cpu(s) and gpu(s) will have an efficient communication path with shared access to memory.
- mastax 4y agoI'm going to guess that the 6nm chiplets are analogous to the EPYC IO die, and have the IO, memory controllers, and last level cache. The compute dies are stacked on the IO dies, which are stacked on an interposer. The interposer connects the IO dies to each other and to the HBM. The HBM is not stacked on the chiplets because I don't think any current HBM supports that and AMD considers it a future technology: https://www.techpowerup.com/305060/amd-envisions-stacked-dram-on-top-of-compute-chiplets-in-the-near-future https://www.techpowerup.com/305060/amd-envisions-stacked-dra...
- rektide 4y agoThis is a fun theory! I like the view of the IO dies connecting to each other across interposer. That interposer is going to be moving a lot of traffic!
- mastax 4y agoHBM requires a pretty expensive and dense silicon interposer, which should be sufficient to connect the IO dies. They're probably just using an infinity fabric network between the dies anyway so it's pretty high level.
- phkahler 4y agoCan we get an APU with 16 or 32GB of HBM to build tiny PCs? That and an SSD and some IO would make an awesome SFF. Of course do this all at low power so air cooling is possible.
- whazor 4y agoAPU with 24 or 32 GB would be really awesome, this would be perfect for a game console (new steam machine?). The extra memory is very useful as the GPU and CPU would be one. Similarly, the M1/M2 Mac mini could, hardware wise, also be a great game console. So we are maybe not that far away.
- EntrePrescott 4y agothat would be really great… if AMD had a working software stack that supported consumer (as opposed to: datacenter) APUs. I'm writing this on a machine with a Ryzen APU… and it's pretty much unusable for compute. And that's not a hardware problem (the hardware specs would be nice enough) but AMD not giving a shit about having a working software stack, especially if it doesn't fall into one of the two domains: "gamers" or "datacenter".
- slavik81 4y agoWhich APU do you have?
- brucethemoose2 4y agoToo expensive. PC OEMs dont even like moderately sized IGPs, as they largely rejected Intel's Iris Broadwell eDRAM chips, Kaby Lake G, and Van Gogh (the Steam Deck chip[1]): [1] https://videocardz.com/newz/amd-mobile-apus-for-2021-2022-detailed-first-navi2-igpu-coming-with-vangogh https://videocardz.com/newz/amd-mobile-apus-for-2021-2022-de...
- sliken 4y agoI was trying to shop various higher end intel iGPUs. Generally the premiums were so high than a low end Nvidia was much faster AND cheaper. They were somewhat lower power, but it was a pretty narrow space where Intel had the advantage.
- tromp 4y agoNo mention of SRAM. Does anyone know how much is present in total across all caches?
- imhoguy 4y agoI want that in my next laptop :) seems to be Apple M1/M2 competitor.
- deleted 4y ago[deleted]