4 ms·
Nope. Just the weights would take 88GB at 4 bit. 128GB MBP ought to be able to run it. If I were to guess, a version for Apple MLX should be available within a
by ein0p 3y ago
Nope. Just the weights would take 88GB at 4 bit. 128GB MBP ought to be able to run it. If I were to guess, a version for Apple MLX should be available within a few days, for those of us fortunate enough to own such a thing.
- Art9681 3y agoIt’s already available. I had it running yesterday morning in an M3 MAX 128GB. I get about 6tps. https://www.reddit.com/r/LocalLLaMA/s/MSsrqWHYga https://www.reddit.com/r/LocalLLaMA/s/MSsrqWHYga