Very good to hear that you are optimizing for inference rather than training. I’ve tried llama and its various instruction tuned siblings and have yet to get eq

Very good to hear that you are optimizing for inference rather than training. I’ve tried llama and its various instruction tuned siblings and have yet to get equivalent performance to gpt-3.5 on coding tasks. Seeing how the base model performed relative to gpt-3 on the various benchmarks gives me hope that the difference is just in RLHF or other fine tuning steps. I really hope the community is able to get there, Especially if the resulting model is able to be quantized with minimal loss.