2 ms·
I don’t have a good answer but would like one. I’ve got a not too old desktop with a 4090 and 64gb of ram, I’d like a local model also.
by hightrix 1mo ago
I don’t have a good answer but would like one. I’ve got a not too old desktop with a 4090 and 64gb of ram, I’d like a local model also.
- amumu 1mo agoI haven't tried with Claudette but did evaluate https://github.com/zachahn/vomit https://github.com/zachahn/vomit and https://github.com/gvzdv/claudish-to-english https://github.com/gvzdv/claudish-to-english earlier today. I ended up using claudish-to-english with a prompt derived from the vomit one and some of my existing instructions. I ran a few local models through a test harness to see how they did, and the gemma4 ones added bad behavior back in less than any others I tested. So for the parent's Macbook question `gemma4-26b-mlx` should work well. For you with 24 GB VRAM, `gemma4-26b-a4b`. I tried higher VRAM models and they slowed down while still doing just as well or slightly worse. If someone else tests and finds a better performing model though, please update me here, I'd love to try it.