3 ms·
that's incorrect in the presence of batching. it's tough work making it truly deterministic: https://x.com/FireworksAI_HQ/status/2069873437217276015 https://x.
by nok22kon 4mo ago
that's incorrect in the presence of batching. it's tough work making it truly deterministic:
https://x.com/FireworksAI_HQ/status/2069873437217276015 https://x.com/FireworksAI_HQ/status/2069873437217276015
- vidarh 4mo agoIt's not that hard. What is hard is making it truly deterministic and retain high throughput.