3 ms·
Not a ton. I'd say 64 GB minimal to play, 96-128 GB better.
by abalashov 4mo ago
Not a ton. I'd say 64 GB minimal to play, 96-128 GB better.
- throwawayffffas 4mo agoNah, you can run the 24b - 35b class with between 90k and 256k of context with about 40GB and they are pretty good. Especially the MOE variants fit neatly in 40GB.
- abalashov 4mo agoYeah, but then you need RAM for the rest of your OS and applications. I'd say 64 to be comfortable in the sense to which most HN users are accustomed.
- throwawayffffas 4mo agoSure sure, if you plan to run it on system ram instead of dedicated gpus then yeah you need an extra overhead to run your own stuff.