3 ms·
Wow that's so incredible. Thanks for putting this together! Do you have any machine specs associated with this? Can an old-ish Macbook Pro run this service? I
by yawnxyz 4y ago
Wow that's so incredible. Thanks for putting this together!
Do you have any machine specs associated with this? Can an old-ish Macbook Pro run this service?
I'm also curious, since I'm new to all this — is it possible to run something like this on Fly.io or does it take up way too much space?
- sp332 4y ago7B is the default. If it's quantized to 4 bits, that's a 3.9 GB file.