3 ms·
Yeah, things like summarization and general chatbot use are some of the most impressive things LLMs can do, yet maybe the last things they should be used for in
by HarHarVeryFunny 3y ago
Yeah, things like summarization and general chatbot use are some of the most impressive things LLMs can do, yet maybe the last things they should be used for in a business setting where the accuracy of the output matters. Air Canada found this out by being dumb enough to put an LLM-based chatbot into production.
Better use cases for LLMs are either where:
a) There is a non-judging human in the loop who is using the LLM as a tool, and accepts it's errors/limitations. It's a bit like asking a co-worker for advice - they might have something useful on occasion, but you're not just going to take what they say unchecked and put it on the company web site or send it to your boss as accepted truth.
b) You can verify the output of the LLM in someway - maybe easier if it's a structured output of some sort (SQL query perhaps) rather than free-form text.
- cjk2 3y agoa) it is more effort to rationalise something that is written than it is to write something rationalised. b) if it was that deterministic, then you could generate it without an LLM.
- HarHarVeryFunny 3y agoAgreed. Other than just playing with them, the only place I've personally found any utility in LLMs is essentially as a kind of search to get a toehold in a subject matter I'm not familiar with.
- bluefirebrand 3y agoYeah, my thoughts are that the best usecase for LLMs is as an input to a process that is then controlled by a human For the summary thing, I think we eventually settled on a workflow that generates the summary using LLM, then we store the output in our dataset for the report. This also allows the human building the report to modify it if necessary