20 ms·
I've been trying my hands at implementing an emacs package for inline completions with this. I have it mostly working and performance is good enough but I haven
by cmrdporcupine 9mo ago
I've been trying my hands at implementing an emacs package for inline completions with this. I have it mostly working and performance is good enough but I haven't been blown away by the quality of its suggestions unfortunately. Which I guess is expected from a 1.5B model.
I'd love to see them making a larger model in the 10-20b range maybe? I know most people wouldn't be able to run that on their machines, but some could.
Running on ollama locally on NVIDIA Spark GB10. Tried it also with vLLM. Pretty fast.
- mijoharas 9mo agoDo you care to share your implementation?
- cmrdporcupine 9mo agoIf I can make it clean and decent I will. I might look at again after work and see if I can tune it up. It was a bit flake and I wasn't blown away by the interaction.
- kevinlu1248 9mo agoAre you using the right format? https://huggingface.co/sweepai/sweep-next-edit-1.5B/blob/main/run_model.py#L30-L103 https://huggingface.co/sweepai/sweep-next-edit-1.5B/blob/mai...
- cmrdporcupine 9mo agoYea, I tweaked it a bunch to try to follow what was described there