4 ms·
And here I am with 128GB Strix Halo longingly eyeing the Blackwell cards that spit tokens 10-20x the speed. The question is ultimate shape of knowledge compres
by seemaze 4mo ago
And here I am with 128GB Strix Halo longingly eyeing the Blackwell cards that spit tokens 10-20x the speed.
The question is ultimate shape of knowledge compression and bandwidth optimization at which we arrive I suppose.
- canyp 4mo agoIf you haven't already, check/increase the GPU memory carve-out on your UEFI. More details: https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimization/strixhalo.html#memory-settings https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimi...
- electroglyph 4mo agothat link actually recommends not doing it from UEFI and doing it via software
- seemaze 4mo agoCurrently utilizing 126GB GTT on a headless host