2 ms·The real test is throughput. I'd like to see tokens/sec at higher concurrency and with uneven hardware.by whatjustin 3mo agoThe real test is throughput. I'd like to see tokens/sec at higher concurrency and with uneven hardware.