3 ms·
> If you have a large collection of mostly-dense vectors and can tolerate lossy compression, Bolt can probably save you 10-200x space and compute time. Space.
by Iv 4y ago
> If you have a large collection of mostly-dense vectors and can tolerate lossy compression, Bolt can probably save you 10-200x space and compute time.
Space. It can save space.
The main limitation of fast ML models nowadays is how much parameters you can load in your GPU memory, and these are usually matrices.
200x would allow me to run GPT-3 on my old GTX 1050.
Frameworks, please implement this NOW!