4 ms·
Show HN: TurboPilot: Copilot clone runs code completion LLM on your CPU
Hi HN,
I spent my easter weekend stuck in the house with COVID and I decided to play with llama.cpp [1] and fauxpilot [2] to see if I could get LLM code assist working on pure CPU.
As a proof of concept I'd say I've proven that it's possible. However there's still a lot to do. The auto complete is quite slow at the moment. PRs welcome.
[1] https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp
[2] https://github.com/fauxpilot/fauxpilot https://github.com/fauxpilot/fauxpilot
- jslakro 3y agohttps://github.com/TabbyML/tabby https://github.com/TabbyML/tabby
- danduma 3y agoNice, but "Tabby requires Pascal or newer NVIDIA GPU." As I understand the whole point of TurboPilot is it runs on CPU :D
- ranguna 3y agoTabby infers on the gpu and is slow, I can only imagine how slow truepilot is on the cpu. If people want auto complete, it needs to be super fast. For slow inference, a better application would be a chatbot that reads your code and answers questions, like cody from sourcegraph.
- wsxiaoys 3y agoNice work! Added to https://github.com/wsxiaoys/awesome-ai-coding https://github.com/wsxiaoys/awesome-ai-coding