5 ms·Inference speed is not a great metric given the horizontal scalability of LLMs.by gliched_robot 2y agoInference speed is not a great metric given the horizontal scalability of LLMs.