3 ms·
This seems useful for issues early in the convo but what if the AI responses diverge from the recorded convo prior to the issue being hit?
by aeternum 2y ago
This seems useful for issues early in the convo but what if the AI responses diverge from the recorded convo prior to the issue being hit?
- zammitjames 2y agoThat’s a great point! While we do our best to simulate an identical case, if the agent responds differently, our focus is on whether the key evaluator or goal for that replay set passes or fails. We use that as the source of truth and flag the exact moment where the conversation diverges from the expected flow.