4 ms·
It can't be run locally,can it ? I see that the training need 8xA100 80G and running need cuda but I doubt it need 8xA100 to run.
by _ache_ 3y ago
It can't be run locally,can it ?
I see that the training need 8xA100 80G and running need cuda but I doubt it need 8xA100 to run.
- orbital-decay 3y agoIt's a model with 7B parameters, it runs on a potato.
- valine 3y agoWith quantization you can run this on a Raspberry Pi.