3 ms·
In theory quantized weights of smaller models are under a gigabyte. If you are looking for megabytes, yeah, those "chat" llms are pretty unusable at that size.
by brucethemoose2 3y ago
In theory quantized weights of smaller models are under a gigabyte.
If you are looking for megabytes, yeah, those "chat" llms are pretty unusable at that size.