3 ms·

This is quite cool. Have you compared performance vs latency tradeoff between 1B/3B/7B models? What would you recommend as the ideal choice?