3 ms·
This can currently already be done using a streaming capable LLM with a streaming input/output TTS model.
by cootsnuck 2y ago
This can currently already be done using a streaming capable LLM with a streaming input/output TTS model.
- lostmsu 2y agoAny LLM is "streaming capable".
- Xmd5a 2y agohttps://github.com/mit-han-lab/streaming-llm https://github.com/mit-han-lab/streaming-llm On a side node, and that's what led me to the link above, I wonder if it would be possible to chain N streaming LLMs in an agent workflow and get a final output stream almost instantaneously without waiting for N-1 LLM to complete their reply.