3 ms·
Anyone know if these LLaMA models can have a large pile of context fed in? Eg to have the "AI" act like ChatGPT with a specific knowledge base you feed in? Ie
by unshavedyak 4y ago
Anyone know if these LLaMA models can have a large pile of context fed in? Eg to have the "AI" act like ChatGPT with a specific knowledge base you feed in?
Ie imagine you feed in the last year of chatlogs of yours, and then ask the Assistant queries about the chatlogs. Compound that with your Wiki, itinerary, etc. Is this possible with LLaMA? Where might it fail in doing this?
(and yes, i know this is basically autocomplete on steroids. I'm still curious hah)
- kevin42 4y agoYou can feed in a lot of context, but the memory requirements go up and up. It was crashing on me in some experiments, but I saw a pull request that let's you set the context size on the command line. I was using the 30B model and with 500 words of context it was using over 100GB of RAM. You probably would be better off fine-tuning the LLaMA model like they did with Alpaca though.
- deleted 4y ago[deleted]
- grepLeigh 4y agoI cracked up while reading the README. To the author: may you always get as much joy as you're giving with these projects. Nice work!