2 ms·
does this work similar to airllm? i am wondering how it would handle something like quantizing kimi k3 on a budget of 8 gbs, or is that something you are not at
by mbuchel-hn 2mo ago
does this work similar to airllm? i am wondering how it would handle something like quantizing kimi k3 on a budget of 8 gbs, or is that something you are not attempting to solve yet?
- rhgraysonii 2mo agoYes that is exactly what this does.
- kennywinker 2mo agoCould you explain what happens when you try to shoehorn a 2.4T parameter model into a 24gb m4 mac?
- akshay_akula 2mo agoWondering the same thing but for 48gb M5 Max.
- metalliqaz 2mo agoextreme divergence would be my guess