34 ms·
I'm quite impressed with Llama 2 13B - the more time I spend with it the more I think it might be genuinely useful for more than just playing around with local
by simonw 3y ago
I'm quite impressed with Llama 2 13B - the more time I spend with it the more I think it might be genuinely useful for more than just playing around with local LLMs.
I'm using the MLC version (since that works with a GPU on my M2 Mac) via my https://github.com/simonw/llm-mlc https://github.com/simonw/llm-mlc plugin.
- gsuuon 3y agoEven the 7B model is shockingly good! I've been hacking on a project also built on MLC (but the web runtime) and the completions I'm seeing from Llama 2 7B, just running on my laptop's browser, have been really impressive. There's a demo page here: https://ad-llama.vercel.app/ https://ad-llama.vercel.app/
- simonw 3y agoThat demo is really cool!
- moneywoes 3y agoWhat are your use cases
- simonw 3y agoThe thing I really want to get working is retrieval augmented generation - so effectively answering questions based on a blob of context that I pass in, and being able to do good-enough summarization. I haven't quite proved this to myself yet but I think it's going to work pretty well.
- nl 3y agoNot simonw, but I've been using Llama2-13B for search re-ranking very successfully.