3 ms·

ChatGPT has 175B weights if I'm not mistaken. Llama 7B would not be in any way comparable.

• superkuh 4y ago
  For people who think the number of parameters determines LLM coherence, well, that's a good rule of thumb. But there's an optimal training set data size to parameters count and gpt3 was trained on too little data. The LLM coming out now trained on more data with fewer parameters and achieve something close. Sure, the 7 billion parameter can't do long outputs. But the 13 billion one is not too bad. They're not a full replacement by any means but for many use cases a local service that is stupider is far preferable to a paid cloud service.
• v64 4y ago
  One finding in the LLaMA paper [1] is that our current large models are undertrained. LLaMA with 13B params outperforms GPT-3 175B (not ChatGPT), but an "instruct" version of LLaMA was finetuned over the 65B model and did quite well. [1] https://arxiv.org/pdf/2302.13971.pdf https://arxiv.org/pdf/2302.13971.pdf