10 ms·
AMD’s RX 7600: Small RDNA 3 Appears
- speed_spread 3y agoThe 8GB VRAM size limitation is the biggest downside for cheap local AI. My feelings is that current LLMs start producing interesting results from 12GB and above.
- ChuckNorris89 3y agoHere's where the unified memory architecture of M1 Macs, gaming consoles, APUs, and other custom SoCs can shine and shows the inefficiencies of the traditional ageing desktop PC architecture in the modern world where the graphics VRAM is completely separated from the system RAM. I'd like to see PCs designed more like consoles or M1 Macs, with GDDR shared as unified RAM between GPU and CPU. I have an laptop with a last gen AMD APU that's no slouch, but the VRAM slice of total RAM is still fixed as configured in BIOS between 512MB and 4GB, instead of fully unified and dynamically shred by the OS, which seems highly ineficient and wasteful in the modern age. It's why PS5 and Xbox with their 16GB of fully unified VRAM can compete with gaming PCS witch need 16GB of system RAM and over 8GB of VRAM. Why have two separate memory zones where one sists empty most of the time and when needed is actually too small while the other is half empty, when you can unify them and make use of the whole pie as needed?
- coffeebeqn 3y agoI can’t imagine the SOCs are too far away for PCs. It does go against the upgradeability philosophy but I’m not sure how many users really care about that
- kramerger 3y agoIs this different from integerated GPUs from Intel and AMD?
- ChuckNorris89 3y agoAs exemplified in my post above, AMD integrated GPUs don't have unified memory, at least my 5000 series I bought in 2022. Maybe the new 7000 series changes this.
- fulafel 3y agoThey have had it for about 10 years, since Kaveri. See eg https://hexus.net/tech/news/cpu/54709-amds-heterogeneous-uniform-memory-access-huma-detailed/ https://hexus.net/tech/news/cpu/54709-amds-heterogeneous-uni... or https://www.guru3d.com/articles-pages/amd-a10-7800-kaveri-apu-review,3.html https://www.guru3d.com/articles-pages/amd-a10-7800-kaveri-ap...
- ChuckNorris89 3y agoNot exactly. While that may have been the case way back in the past when AMD APUs were designed as a homonegnous CPU-GPU unit from the start, I can tell you that's not the case with the relatively modern 5000 series. I need to go in the Bios and specify explicitly how much of the system RAM I want allocated exclusively of the integrated GPU and the rest stays available for system RAM. The reason is because Ryzen 5000 APUs seem to be a job rushed out the door so they're just a Zen 3 CPU with a separate Vega GPU glued together on the same die, but they're not a homogenous design, designed to work as one unit, like the APU of the PS5, so memory wise they're unaware of each other, even though AMD calls them APUs, they're not really, but more like separate CPU and GPU on the same die. I wish I knew about this limitation at the time, as Intel chips with integrated graphics have unified memory.
- yjftsjthsd-h 3y agoI wonder if a clever enough driver+firmware could make that hot-adjustable (or truly unified, rather) with nothing but a software patch. It sounds like the pieces are all there.
- 3y ago
- throw9away6 3y agoWho really upgrades pcs anymore. By the time you need to upgrade everything from socket to power standard is outdated
- bee_rider 3y agoA mid-lifetime upgrade where you pop in a new GPU, fill up any unpopulated ram slots, or add in a new drive, can be nice. CPUs last a really long time nowadays.
- throw9away6 3y agoSo do gpus though. Are you really going to pop in a 800$ gpu into 200$ worth of components?
- coffeebeqn 3y agoDepends how good the original was if it’s worthwhile. GPUs don’t need that much from the rest of the machine as long as it’s somewhat sensible for gaming. Especially at 1440p or 4k the GPU is a huge portion of the performance
- yjftsjthsd-h 3y ago> Are you really going to pop in a 800$ gpu into 200$ worth of components? Why not? For the right workloads, the GPU is the important thing anyways. It's not that different from rotating disks through a machine.
- bee_rider 3y agoThe main argument I can think of for separating their memory is that, of course, GDDR can be optimized for bandwidth and regular ram can be optimized for latency or somewhere in between. But, memory latency continues to make poor progress compared to CPU speeds. So, since we’re already going to need a complicated system of caches on the CPU side, maybe it is not such a big deal if CPU memory acts more like GDDR.
- codedokode 3y agoIt doesn't explain why GPUs that are integrated into a CPU (like AMD's ones) and use the same memory, cannot use it all. If they could use all memory, you could cheaply run neural networks with 64Gb of RAM without buying a professional GPU. No wonder manufacturers don't want you to be able to do that.
- Dalewyn 3y agoBoth discrete and integrated GPUs can use as much system RAM as you allow them in your BIOS or operating system. The problem is that system RAM is slow compared to onboard VRAM on discrete GPUs. Size isn't everything, speed and latency are also factors that have to be considered.
- tsss 3y agoI imagine if that ever happens it will only be used to nickel and dime consumers and you'll end up paying even more through forced upselling than with the inefficient VRAM we currently have.
- pezezin 3y agoThe problem is memory bandwidth. The M2 has a bandwidth of 100 GB/s, whereas this RX 7600 has 288 GB/s, and top of the line RX 7900 and RTX 4090 reach almost 1 TB/s. Reaching such speeds with standard DDR5 would require a massive number of channels that is impractical for desktop PCs. The M1, M2, PS5 and Xbox get away with it by being closed systems without any possibility for expansion, but gaming PCs are expected to be expandable.
- hotstickyballs 3y agoPC gaming is responsible for most of the bugs. With Nvidia going all in on cloud compute, gaming is going either console or cloud IMO.
- pezezin 3y agoSure, look at the huge success of Stadia /s Cloud gaming is not an option for most gamers for a very simple reason: latency. By the way, people have been predicting the death of PC gaming since at least the PSX/Saturn/N64 era, yet here we are...
- dontlaugh 3y agoCertainly PC gaming has terrible ergonomics and is very annoying. Sadly, some things you can't play on a console. Cloud gaming is not even close to being an option, the latency is disastrous. Even streaming locally on gigabit network has poor latency, let alone over the internet.
- windowsrookie 3y agoThe Xbox Series X uses GDDR6 and has a memory bandwidth of 560GB/s. I'm assuming apple is using LPDDR5 for power efficiency. Theoretically they could make a desktop specific chip with GDDR6.
- lukevp 3y agoThe M1 ultra has 800 GB/s memory bandwidth. That’s not very far off from the RTX 4090.
- shaklee3 3y agoCUDA has had a single pointer that will handle paging between host and device for quite a while.
- kramerger 3y agoIt's "RDNA lite" for somewhat cheaper GPUs. It is not supposed to have tons of VRAM. Besides, the _really_ interesting LLMs use far far more than 12GB. But that sort of this changes from day to day here...
- cypress66 3y agoYou're expecting too much from a cheap card. AMD offers many 16GB cards, and used RTX 3090s (24GB) are very affordable and are the go to for AI.
- tigeroil 3y agoWhat do you consider to be affordable? In the UK used RTX 3090s still go for well over 1k.
- LaurensBER 3y agoFor a company that's very affordable, for an individual we're not there yet but I've seen some impressive demos running on a M1/M2 laptop so no doubt time will bring down the hardware requirements even more (and hopefully the price of used 3090s as well).
- LoganDark 3y agoRWKV is also fairly performant on CPU. disclaimer: recently did a bunch of work on rwkv.cpp.
- skippyboxedhero 3y agoYou're getting ripped off kid. I can get you one for £600.
- theaussiestew 3y agoHow?
- topspin 3y agoScrew around on ebay, bidding for used cards from individual sellers, as opposed to 'top' sellers that get substantially more for their listings. They may or may not have been burning up the GPU for the last couple years overclocking. Eventually you can land a $600-ish 3090 [1]. Good luck. [1] https://www.ebay.com/itm/266275573529 https://www.ebay.com/itm/266275573529 (read the listing...)
- throwaway485 3y agoI'm suddenly curious if this is a fairly static limit: 8GB for LLMs. Is there any paper or posting somewhere describing how this will move up or down with technology changes? Does it depend entirely on the "thickness" or density of the data being analyzed to produce a model? I'm imagining a spreadsheet or database table with 100 rows vs one with 1000. I guess it depends entirely on how thoroughly data was collected? Not all attributes are dependent, etc.
- jacooper 3y agoYou are not going to be running any AI project on it, ROCm won't support it at all.
- mastax 3y ago> It uses TSMC’s 6 nm process, which won’t provide the performance and power savings that the cutting edge 5 nm process would. In prior generations, AMD fabbed x60 and x600 series cards on the same cutting edge node as their higher end counterparts, and used the same architecture too. However, they clearly felt a sharp need to save costs with this generation, compromising the RX 7600 a bit more than would be expected for a midrange GPU. It's also so they only need to design one memory controller for 6nm. I believe I remember this corroborated from an AMD engineer interview around the 7900XTX launch. Memory controllers aren't just logic that can be "compiled" to whatever target node. They have specific electrical requirements that take substantial design work. For this generation AMD has a 6nm memory controller that they use both in this 6nm monolithic design and in their 6nm memory controller chiplets on the larger designs.
- rowanG077 3y agoIt's rf circuitry at the chip boundary. It isn't like the high-level digital code like Verilog/VHDL which is relatively straightforward to port. The people who do that are the true gurus honestly.
- gyudin 3y agoYet their drivers still cause kernel panic and crashes even on their own demos lol
- HeWhoLurksLate 3y agoI have an RX 6700XT that I bought used that's been flawless since I got it, even though I've been doing cursed stuff with it. Is this specifically a 7000 series gpu problem?
- stepbeek 3y ago5700xt on launch was bad for this, but my anecdotal experience has been that AND have really improved drivers in the last few years.
- rektide 3y agoAlmost twice the bandwidth of the rx580, from April 2017. About the same msrp. It'll be interesting to see how this holds up versus Intel, whose been doing pretty ok at this price point. Intel's initial launch was pretty rocky but the drivers have gotten much much faster already.
- rektide 3y agoI don't know how I messed this up but the actual bandwidth gain is much less. 288GBps vsow 200's? My bad sorry. And yikes.
- SECProto 3y agoAs an RX580 user, I was hoping for a bit more memory - as it stands, this doesn't feel compelling for me to upgrade. Maybe there'll be a 7600xt or something, but I'll be holding out another year or three (and turning graphics settings down to medium)
- tomcam 3y agoI couldn’t see any prices… did I miss that information?
- iforgotpassword 3y agoAny benchmarking sites started comparing cards by "performance per Watt" yet? Asking as a German...
- bullen 3y agoThey can't because then it will be obvious we have peaked and people will panic.
- stepbeek 3y agoHardware Unboxed and Gamers Nexus both show frames per watt iirc.
- heavypixels 3y agoDigital Foundry (hosted on Eurogamer) has started doing that for all their GPU reviews, e.g.: https://www.eurogamer.net/digitalfoundry-2023-nvidia-geforce-rtx-4060-ti-review https://www.eurogamer.net/digitalfoundry-2023-nvidia-geforce...
- distances 3y agoPhoronix has both frames per second per watt and framer per second per dollar. 7600 review at https://www.phoronix.com/review/amd-radeon-rx-7600-linux https://www.phoronix.com/review/amd-radeon-rx-7600-linux