4 ms·
I assume you haven't tried Alpaca (which hasn't been released), only Llama. See the instruction fine tuning section in the article.
by yunyu 4y ago
I assume you haven't tried Alpaca (which hasn't been released), only Llama. See the instruction fine tuning section in the article.
- karmasimida 4y agoThey currently only supports a single input/response format of input right? Multi turns will be more challenging to handle. I am optimistic for 30B or 66B to catch up with OpenAI, but 7B is unlikely to have the same quality.
- enlyth 4y agoHere is a sample chatbot style conversation I had locally with LLAMA 30B 4bit quantization on a 3090 RTX (generation speed 5 tokens per second): https://i.imgur.com/V4lzLz7.png https://i.imgur.com/V4lzLz7.png It does quite well at simple reasoning. More complex stuff, it does struggle sometimes, but I am impressed by the output and the fact that it even runs on my computer. I used the oobabooga repository on Ubuntu 22.