5 ms·
The costs are falling 10-100x every few months. If you told people in 2022 that they could run 70b paramater models at 2 bit quantization on your home 4090, the
by Der_Einzige 3y ago
The costs are falling 10-100x every few months. If you told people in 2022 that they could run 70b paramater models at 2 bit quantization on your home 4090, they'd have laughed in your face. They're not laughing anymore. SFT/DPO are ultra efficient compared to RLHF, and innovation in this space is only just now getting started.