2 ms·

Using DDR5 as VRAM however means you're only getting 50GB/s to 100GB/s read/write speed instead of the 500GB/s available on a proper GPU. That might be a fine

Using DDR5 as VRAM however means you're only getting 50GB/s to 100GB/s read/write speed instead of the 500GB/s available on a proper GPU. That might be a fine tradeoff for some kernels. But my understanding is that Stable Diffusion is very VRAM-bandwidth heavy and actually benefits from the higher-speed GDDR6 or HBM RAM on a proper high-end GPU.

thulle 3y ago
With a 4090 nvidia-smi reports ~60-70% mem bandwidth usage while at 99% gpu usage, so that'd be 650-750GB/s. Considering how slow the inference is on the APU having 10% of the 4090 mem bandwidth maybe isn't that much of an issue? edit: comment from the reddit thread: > LLM isn't all that great as it is primarily memory bandwidth bound, ie almost no difference from a CPU if your memory bw is mere 12/25Gb/s. SD needs far more compute for inference - APU with slow memory helps. - https://old.reddit.com/r/Amd/comments/15t0lsm/i_turned_a_95_amd_apu_into_a_16gb_vram_gpu_and_it/jwjr5j3/ https://old.reddit.com/r/Amd/comments/15t0lsm/i_turned_a_95_...