4 ms·
Yep, we support both evaluations and autocorrections for streaming as well.
by cmdalsanto 2y ago
Yep, we support both evaluations and autocorrections for streaming as well.
- internetter 2y agoAren't streams write-only? Can you delete parts of the stream if it is later deemed incorrect somehow?
- cmdalsanto 2y agoThere's some secret sauce here, but since we intercept each chunk as the LLM pushes them out, we can perform evaluations on them and decide what gets sent back to the client if we detect a fault.