3 ms·
I use a local LLaMA API as my code inference endpoint. No way I'm just shipping all my code to OpenAI.
by MacsHeadroom 4y ago
I use a local LLaMA API as my code inference endpoint. No way I'm just shipping all my code to OpenAI.
- nico 4y agoYou can also use GPT4All https://github.com/nomic-ai/gpt4all https://github.com/nomic-ai/gpt4all
- MacsHeadroom 4y agoI use Vicuna[0]. It's much better than GPT4All. Vicuna is based on 13B (not 7B) and its training data includes humans chatting with GPT-4 vs GPT4All's purely synthetic dataset generated by GPT-3.5. [0] https://github.com/lm-sys/FastChat https://github.com/lm-sys/FastChat
- nico 4y agoThank you! Could Vicuña be used to further fine-tune GPT4All to make it better?
- MacsHeadroom 4y agoI think GPT4All's inferior quality dataset would make a worse combined model than strict Vicuna. Vicuna-30B will likely be better than GPT-3.5 level and approaching GPT-4 level when it's done training, but run slow on CPU.