3 ms·
Of course efficiency matters, but a lot of people either have cheap electricity or efficient hardware. My AMD strix halo home server can serve Gemma4-26B at lik
by sosodev 3mo ago
Of course efficiency matters, but a lot of people either have cheap electricity or efficient hardware. My AMD strix halo home server can serve Gemma4-26B at like 70 TPS (rough estimate, I don’t remember the exact speed buts its fast af) while only using 100W.