3 ms·
You don't need a super agent, you just need two LLM-based systems with errors that aren't too correlated.
by lukasb 3y ago
You don't need a super agent, you just need two LLM-based systems with errors that aren't too correlated.
- pphysch 3y agoHow do you "just" accurately evaluate the error state space of an LLM relative to a real business process? Sounds approximately impossible to me. If you already have the business process robustly defined as code, then the utility of LLM is unclear. The value prop of LLM is in fuzzy business processes like parsing arbitrary helpdesk tickets.
- lukasb 3y agoYou evaluate it the way we've evaluated production ML for years, with cheap QC layers sampled and checked by more expensive layers (with humans on top.) LLMs didn't invent stochastic process steps.