I was playing around w/ a lot of these models as well, and was surprised by how bad LLaMA performed vs it's benchmark scores [1][2][3]. However, recently @tyfon

I was playing around w/ a lot of these models as well, and was surprised by how bad LLaMA performed vs it's benchmark scores [1][2][3]. However, recently @tyfon mentioned he had great success w/ LLaMA and shared his prompt [4] (based off of more recent work by llama.cpp contributors and it performed much better in my own personal testing. There's basically a new fine tune a day and while some I don't like (Alpaca, Vicuna, Baize, Koala are all fine-tuned to be too limiting IMO), I'm interested in what gpt4-x-alpaca and OA (Open Assistant) are doing, and the various un-filtered fine tunes (especially w/ lighter weight adapter/LoRA training which would let you personalize/specialize). GPTQ-for-LLaMa let's me load the 4-bit quantized 30B model (~17GiB) onto my GPU in about 5 seconds (and I know llama.cpp's mmap improvements have also made it quite a lot quicker) so I think it's perfectly reasonable to switch between tuned models for tasks in code assistance, correspondence, etc. I have access to ChatGPT 4, and agree it's signficantly better than what's out there atm, and it can basically do anything I've thrown at it (here's it helping me with my WM yak shaving: https://sharegpt.com/c/Xv73Vwl https://sharegpt.com/c/Xv73Vwl or discussing MAPS/psychedelics for clinical applications https://sharegpt.com/c/N3VXFxS https://sharegpt.com/c/N3VXFxS - it's amazing what it can pull from memory and it hallucinates much less than 3.5). That being said, I've found the Browsing 3.5 model to be quite useful for doing things like catching up on the last few years of LLM advancements: https://sharegpt.com/c/JFexqvm https://sharegpt.com/c/JFexqvm [1] https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md https://github.com/facebookresearch/llama/blob/main/MODEL_CA... [2] https://github.com/ggerganov/llama.cpp/discussions/406 https://github.com/ggerganov/llama.cpp/discussions/406 [3] https://paperswithcode.com/sota/language-modelling-on-wikitext-2 https://paperswithcode.com/sota/language-modelling-on-wikite... [4] https://news.ycombinator.com/item?id=35484341 https://news.ycombinator.com/item?id=35484341