3 ms·
It's likely that a stack of Macs will draw more power for slower prefill/decode than equivalently priced Nvidia GPUs. If power efficient inference is the goal,
by bigyabai 5d ago
It's likely that a stack of Macs will draw more power for slower prefill/decode than equivalently priced Nvidia GPUs. If power efficient inference is the goal, Macs are a non-starter.
- _hugerobots_ 5d agoSo if it isn't a comparative ability, now it's a power cost issue? This reads like goal post moving.