3 ms·
Would love an answer on this too. It would be even better not just to try using this, but also be able to run it locally, something that has been impossible for
by gjstein 4y ago
Would love an answer on this too. It would be even better not just to try using this, but also be able to run it locally, something that has been impossible for GPT-3.
- arkano 4y agoDoes it look like it would be possible to run locally?
- whimsicalism 4y agoThis is not something that will be possible to run locally. If you had 1 bit per parameter (not realistic), it would still take ~100 GB of RAM just to load into memory.
- The_rationalist 4y agoYou could technically dynamically offload the RAM overload to disk but this would probably be too slow?
- whimsicalism 4y agoI mean, theoretically if you can get the model weights onto disk then you should be able to do the computation - but it might takes days or months on commodity hardware. It would also require creating a system that can do this and I doubt there is much demand.