4 ms·
Yes indeed. The models are too computationally expensive to run locally (7.5Billion parameters). Though you could in-principle swap in any local model.
by shishirpatil 3y ago
Yes indeed. The models are too computationally expensive to run locally (7.5Billion parameters). Though you could in-principle swap in any local model.
- Tostino 3y agoWhat are you talking about? 7b parameter models run insanely fast if you can offload to gpu, and are entirely reasonable speed if CPU only.
- kajecounterhack 3y agoDo y'all have plans to release the model for those who have 16gb graphics cards? (I'm assuming the model is fp16?)