4 ms·

Just a point of clarification. I believe the 128GB Strix Halo can only allocate up to 96GB of RAM to the GPU.

by cpburns2009 1y ago

geerlingguy 1y ago
108 GB or so under Linux. The BIOS allows pre-allocating 96 GB max, and I'm not sure if that's the maximum for Windows, but under Linux, you can use `amdttm.pages_limit` and `amdttm.page_pool_size` [1] [1] https://www.jeffgeerling.com/blog/2025/increasing-vram-allocation-on-amd-ai-apus-under-linux?fjs https://www.jeffgeerling.com/blog/2025/increasing-vram-alloc...
- amstan 1y ago
  I have been doing a couple of tests with pytorch allocations, it let me go as high as 120GB [1] (assuming the allocations were small enough) without crashing. The main limitation was mostly remaining system memory: htpc@htpc:~% free -h total used free shared buff/cache available Mem: 125Gi 123Gi 920Mi 66Mi 1.6Gi 1.4Gi Swap: 19Gi 4.0Ki 19Gi [1] https://bpa.st/LZZQ https://bpa.st/LZZQ
- cpburns2009 1y ago
  Thanks for the correction. I was under the impression the GPU memory had to be preallocated in the BIOS, and 96 GB was the maximum number I read about.
  - sliken 1y ago
    Some older software stacks require static allocation in BIOS, but things are moving pretty quickly and allow dynamic allocation. Newer versions (or patches to) pytorch, ollama, and related, which I think might depend on a newer kernel (6.13 or so). Does seem like there's been quite a bit of progress in the last month.
lhl 1y ago
In Linux, you can allocate as much as you want with `ttm`: In 4K pages for example: options ttm pages_limit=31457280 options ttm page_pool_size=15728640 This will allow up to 120GB to be allocated and pre-allocate 60GB (you could preallocate none or all depending on your needs and fragmentation size. I believe `amdgpu.vm_fragment_size=9` (2MiB) is optimal.

Just a point of clarification. I believe the 128GB Strix Halo can only allocate up to 96GB of RAM to the GPU.

108 GB or so under Linux. The BIOS allows pre-allocating 96 GB max, and I'm not sure if that's the maximum for Windows, but under Linux, you can use `amdttm.pages_limit` and `amdttm.page_pool_size` [1] [1] https://www.jeffgeerling.com/blog/2025/increasing-vram-allocation-on-amd-ai-apus-under-linux?fjs https://www.jeffgeerling.com/blog/2025/increasing-vram-alloc...

amstan 1y ago
I have been doing a couple of tests with pytorch allocations, it let me go as high as 120GB [1] (assuming the allocations were small enough) without crashing. The main limitation was mostly remaining system memory: htpc@htpc:~% free -h total used free shared buff/cache available Mem: 125Gi 123Gi 920Mi 66Mi 1.6Gi 1.4Gi Swap: 19Gi 4.0Ki 19Gi [1] https://bpa.st/LZZQ https://bpa.st/LZZQ
cpburns2009 1y ago
Thanks for the correction. I was under the impression the GPU memory had to be preallocated in the BIOS, and 96 GB was the maximum number I read about.
- sliken 1y ago
  Some older software stacks require static allocation in BIOS, but things are moving pretty quickly and allow dynamic allocation. Newer versions (or patches to) pytorch, ollama, and related, which I think might depend on a newer kernel (6.13 or so). Does seem like there's been quite a bit of progress in the last month.

lhl 1y ago

In Linux, you can allocate as much as you want with `ttm`: In 4K pages for example: options ttm pages_limit=31457280 options ttm page_pool_size=15728640 This will allow up to 120GB to be allocated and pre-allocate 60GB (you could preallocate none or all depending on your needs and fragmentation size. I believe `amdgpu.vm_fragment_size=9` (2MiB) is optimal.