3 ms·
Then the author needs to be more specific because there are special models for code autocomplete that are much faster.
by stingraycharles 3mo ago
Then the author needs to be more specific because there are special models for code autocomplete that are much faster.
- jboss10 3mo agoFor people who saw this and might want a recomendation, I like running a tiny qwen model with llama cpp. Qwen2.5 coder 0.5B or 1.5B (not the instruct version) On a modern-ish GPU these should run really fast with little latency. They cost nothing and don't send your data to anyone.