3 ms·
NVidia and AMD make $$$ on datacenter GPUs so it makes sense they don't want to discount their own high-end. Intel has nothing there so they can happily go for
by treprinum 2y ago
NVidia and AMD make $$$ on datacenter GPUs so it makes sense they don't want to discount their own high-end. Intel has nothing there so they can happily go for commodization of AI hardware like what Meta did when releasing LLaMA to the wild.
- beAbU 2y agoIs nVidia or AmD offering 128gb cards in any configuration?
- latchkey 2y agoThey aren't "cards" but MI300x has 192GB and MI325x has 256GB.
- phkahler 2y agoYou can run an AMD APU with 128GB of shared RAM.
- treprinum 2y agoIt's too slow and not very compatible. Most BIOSes also don't allow sharing that much memory with GPU (max like 16GB).
- ryao 2y agoIsn’t that setting just a historical thing annd ann integrated GPU is able to access any system memory that is mapped by the IOMMU? I assume this is how it works for people using the NVIDIA Jetson AGX Orin 64GB Developer Kit to do inference. I do not know why it would be different for AMD APUs.
- treprinum 2y agoI remember somebody complaining about it on reddit, unable to overcome some BIOS limitation on an AMD G processor. Even on M3 Max one had to issue a special command to enable GPU to access more memory.
- ryao 2y agoThe command on the M3 Max is a sysctl command to adjust an operating system enforced limit. That is different than the aperture setting in the bios. The limitation on AMD is more interesting and more relevant.
- genewitch 2y agoyou can do that with nvidia too but it takes you from 6tok/s to 6s/token or worse (not even exaggerating)