3 ms·
200 GB is an unfathomable amount of main memory for a CPU (with apologies for snark,) give gpt-oss-120b a try. It’s not fast at all, but it can generate on CPU
by kristjansson 9mo ago
200 GB is an unfathomable amount of main memory for a CPU
(with apologies for snark,) give gpt-oss-120b a try. It’s not fast at all, but it can generate on CPU.
- awestroke 9mo agoBut it's incredibly incapable compared to SOTA models. OP wants high quality output but doesn't need it fast. Your suggestion would mean slow AND low quality output.
- kristjansson 9mo agoSet your parameters to make that point then. “Yeah just run a 1T+ model on CPU”