This initial public alpha release of Lamini LLM fine tuning includes several advanced optimizations: 1. Chinchilla recipe smaller models pretrained on data inc

This initial public alpha release of Lamini LLM fine tuning includes several advanced optimizations: 1. Chinchilla recipe smaller models pretrained on data increases inference speed 2. Instruction fine tuning training on a small high quality set of instructions unlocks the knowledge learned during foundation model training. 3. Latency constrained batching achieves high utilization under load during token generation 4. Containerized SLURM combines fast scheduling of SLURM with LLM containers 5. Mixed precision training uses matrix operations for training There are so many low hanging fruits in LLM tuning, steering, and alignment. We are just getting started on this for enterprise and open source. For this reason I disagree with Sam Altman that the age of bigger models is over. We are still leaving orders of magnitude on the table, e.g. by not including optimizations like sparsity in these models. References for inspiration: [1] - https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556 [2] - https://arxiv.org/abs/1910.10683 https://arxiv.org/abs/1910.10683 [3] - https://www.usenix.org/system/files/osdi22-yu.pdf https://www.usenix.org/system/files/osdi22-yu.pdf [4] - https://www.schedmd.com/ https://www.schedmd.com/ [5] - https://arxiv.org/abs/1710.03740 https://arxiv.org/abs/1710.03740