4 ms·
Power isn't proportional to parameters. It may be vaguely proportional to tokens/s although batching screws that up. Claude Sonnet is probably running on a 8 G
by wmf 5mo ago
Power isn't proportional to parameters. It may be vaguely proportional to tokens/s although batching screws that up.
Claude Sonnet is probably running on a 8 GPU box that consumes 10 kW while Opus might use more like 50 kW but that's shared by a bunch of users thanks to batching.