3 ms·
This is not something that will be possible to run locally. If you had 1 bit per parameter (not realistic), it would still take ~100 GB of RAM just to load int
by whimsicalism 5y ago
This is not something that will be possible to run locally.
If you had 1 bit per parameter (not realistic), it would still take ~100 GB of RAM just to load into memory.
- The_rationalist 5y agoYou could technically dynamically offload the RAM overload to disk but this would probably be too slow?
- whimsicalism 5y agoI mean, theoretically if you can get the model weights onto disk then you should be able to do the computation - but it might takes days or months on commodity hardware. It would also require creating a system that can do this and I doubt there is much demand.