3 ms·
Only GPU-poors run Q-GaLore and similar tricks.
by storus 1y ago
Only GPU-poors run Q-GaLore and similar tricks.
- Twirrim 1y agoEven the large cloud AI services are focusing on this too, because it drives down the average "cost per query", or whatever you want to call it. For inference, arguably more even than training, the smaller and more efficient they can get it, the better their bottom line.
- storus 1y agoFor inference of course; the OP I replied to mentioned training though.