4 ms·
Can one of thse models be run on a single machine? What specs do you need?
by sk11001 2y ago
Can one of thse models be run on a single machine? What specs do you need?
- Y_Y 2y agoAbsolutely! They have a billion-parameter model that will run on my first computer if we quantize it to 1.5 bits. But realistically yes, if you can fit in total ram you can run it slowly, if you can fit it in gpu ram you can probably run it fast enough to chat.
- sumedh 2y agoThe 8B models run fine on a M1 pro 16GB.