3 ms·
Yi-34B, Llama 2, and common practices in LLM training
- helloericsf 3y agoIn short, all modern large language models (LLMs) are made from the same algorithmic building blocks. The architectural differences between Llama 2 and the original 2017 Transformer were not invented by Meta, and are all public owing to open access publishing being the norm in computer science. So, even though Yi-34B adopts Llama 2's architecture, Meta's model did not give 01.AI access to any previously inaccessible innovation.
- visarga 3y agoFunny, I never saw the claim Yi 34B was based on LLaMA, but now I did.
- deleted 3y ago[deleted]
- hinkley 3y agoYi-34B sounds like a stellar object.