3 ms·
Maybe that’s how models trained on luxury, bountiful H100s feel when being quantized and run on the near-barren earth of consumer CPU inference.
by jl6 1y ago
Maybe that’s how models trained on luxury, bountiful H100s feel when being quantized and run on the near-barren earth of consumer CPU inference.