4 ms·
> Does Slack sell an LLM chatbot solution that is able to give me reliable answers to business/technical decisions made over the last 2 years in chat? We don't
by rsynnott 2y ago
> Does Slack sell an LLM chatbot solution that is able to give me reliable answers to business/technical decisions made over the last 2 years in chat? We don't have this yet - most likely because it's probably still too expensive to do this much inference with such high context window.
So, your problem there is 'reliable'. LLMs, fairly fundamentally, cannot do 'reliable'. If you're looking for reliable, you likely are looking at a different tech entirely.
- williamcotton 2y agoIf an LLM is tasked with translating a full text to a summary of that text then it is very reliable. This is akin to an analytic statement, eg, "all bachelors are married". The truth is completely within the definition of the statement. Compare this to a synthetic statement such as "it is raining outside". In this case the truth is contingent on facts outside of the statement itself. When LLMs are faced with an analytic statement they are more reliable. When they are faced with a synthetic statement they are prone to confabulate and are unreliable.
- rsynnott 2y ago> If an LLM is tasked with translating a full text to a summary of that text then it is very reliable. Hrm. I've found LLM summaries to be of... dubious reliability. When someone posts an article on this here orange website, these days someone will sometimes 'helpfully' post a summary generated by a magic robot. Have a look at these, sometime. They _often_ leave out key details, and sometimes outright make shit up. Interesting article on someones' experiences with this recently: https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/ https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actu...
- williamcotton 2y agoSure, anecdotal evidence. Here's another anecdote: Original article: https://osa1.net/posts/2024-10-09-oop-good.html https://osa1.net/posts/2024-10-09-oop-good.html LLM (ChatGPT o1-preview) results: https://chatgpt.com/share/67166301-00d8-8013-9cf5-e8a980aca775 https://chatgpt.com/share/67166301-00d8-8013-9cf5-e8a980aca7... LGTM! I'd like to know which model is used in the article you've referenced as well as the prompt. I also suspect that 50 pages is pushing up to the limits of the context window and has an impact on the results. --- As for the article itself... Use a language like F# or OCaml and you get a functional-first language that also supports OOP!