4 ms·
I'm guessing these small models are not meant to be used for writing whole blocks of code but rather to add more intelligent autocomplete for a few characters a
by diggan 3y ago
I'm guessing these small models are not meant to be used for writing whole blocks of code but rather to add more intelligent autocomplete for a few characters ahead, then they could probably provide a bit of help at least. I've had the same experience as you when trying anything locally below 30B parameters.