3 ms·
They significantly lowered latency compared to EPYC/Xeon, which is critical for streaming agents (e.g. text/audio/video agents).
by storus 7mo ago
They significantly lowered latency compared to EPYC/Xeon, which is critical for streaming agents (e.g. text/audio/video agents).
- stingraycharles 7mo agoWhat latency? How much is it compared to LLM inference speed?
- storus 7mo agoSee the Redpanda comment/link here.