3 ms·
How does that fit into a 4090? The files on the repo look way too large. Do they mean a quant?
by Havoc 2y ago
How does that fit into a 4090? The files on the repo look way too large. Do they mean a quant?
- cbg0 2y ago> Mistral Small can be deployed locally and is exceptionally "knowledge-dense", fitting in a single RTX 4090 or a 32GB RAM MacBook once quantized.