3 ms·
> You could skip the generative part for the bot reply entirely How could you remove the generative part from an LLM? (genuine question)
by flopriore 2y ago
> You could skip the generative part for the bot reply entirely
How could you remove the generative part from an LLM? (genuine question)
- TeMPOraL 2y agoSorry, bad phrasing. What I mean is, LLMs are more useful when they're asked to parse user input, into a more structured and narrow output, that is then fed to some business logic. Like, there's countless different ways the user could ask about their missing shipment; unlike earlier methods, LLMs are able to correctly classify almost all of them as "missing shipment". So do this, and then stop at that. The conversational output you send to the user does not need to be generated by an LLM. There's not much value added for the massive risk of hallucinations LLMs introduce.