10 ms·

Real world industry wont stand for "hallucinated" outputs, unless there is more innovation around UI/UX on outputs. For example, no way lawyers/bankers/doctors

Real world industry wont stand for "hallucinated" outputs, unless there is more innovation around UI/UX on outputs. For example, no way lawyers/bankers/doctors are going to use LLM in their current forms and limitations if they can't trust the outputs.

captn3m0 3y ago
Professional norms differ between countries. What might be unthinkable for a doctor in US might be meh for a professional in India. For eg, a judge in India used ChatGPT for a bail hearing. https://www.livemint.com/news/india/this-indian-court-has-used-chatgpt-on-a-criminal-case/amp-11679977632552.html https://www.livemint.com/news/india/this-indian-court-has-us...
npsomaratna 3y ago
Disagree. As a lawyer I use LLMs with RAG to help me surface information all the time. Often, this allows me to find niche case law that I just wouldn't have had the time to find on my own. However, I double-check everything, and read all the original sources. LLMs are best treated as the AI equivalent of a human assistant who is knowledgeable and fast, but also inexperienced, and thus, prone to making mistakes. You won't throw out the work of such an assistant-it'll still save you hours of effort. However, you won't take the work at face value either.
- jebarker 3y ago
  I use LLMs for helping with coding it's proving invaluable. I see it very similarly, an inexperienced assistant with very broad knowledge. I also find that they don't make too many mistakes in tasks such as refactoring or finding bugs, it's when you ask it to just wholesale generate code for you that you hit problems. If I were to just take the code as is, not test it and not check I understand it then use it it's me that would be making the mistake not the LLM.
- arrowsmith 3y ago
  What's RAG?
  - simonw 3y ago
    Retrieval Augmented Generation. It's the trick where you take a question from a user, search for documents that match that question, stuff as many of the relevant chunks of content from those documents as you can into the prompt (usually 4,000 or 8,000 tokens, but Claude can go up to 100,000) and then say to the LLM "Based on this context, answer this question: QUESTION". I wrote about one way to implement that here: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer...
- squeaky-clean 3y ago
  I have a friend in law school at the moment, and while he obviously can't use AI for school, multiple professors of his have recommended he get familiar with using them now so he'll be efficient at using them after passing the bar.
naasking 3y ago
> Real world industry wont stand for "hallucinated" outputs Of course they will, if the other benefits are large enough. Checking factual accuracy of a large corpus can often be considerably simpler than generating the large corpus to begin with.
gryn 3y ago
funny you mention lawyers when just not long ago some of them made the news because they trusted chatGPT which gave them a imaginary precedent. just because a small percentage technically oriented people know the limitations of LLMs doesn't mean that the rest do. people have a tendency to anthropomorphize things, so they naturally think that LLMs think in the same way other humans they know think. edit: first link I found about it, there was also some posts about it here in HN. https://www.abc.net.au/news/2023-06-09/lawyers-blame-chatgpt-for-tricking-them-into-citing-fake-cases/102462028 https://www.abc.net.au/news/2023-06-09/lawyers-blame-chatgpt...
ryanklee 3y ago
Real world industry already relies heavily on hallucinated outputs. They are called humans. One of the main differences between people who see the astonishing value of LLMs right now and people who are some combo of skeptical, dismissive, and indignant, is the expectation that for something to be a valuable source of information, it has to be factually accurate every time. The entirety of civilization has been built on the back of inaccurate sources of information. That will never change and it absolutely can't, because (1) factual accuracy is not something that can be determined by consensus in a variety of significant cases, and (2) factual accuracy as a concept itself does not have a consensus definition, operationally or in the abstract. Absolutely frustrating to see these topics addressed as if thousands of years of intense thinking around truth and factual accuracy has not taken place. The results of those inquiries do not support the basic assumptions of these conversations (i.e. that factual accuracy is amenable to exhaustive algorithmic verification).
JPLeRouzic 3y ago
If you have employees, you also need to create transverse structures (make people work in teams, set check lists, QA, HR, accounting, create corporate charters on gender equality and many other topics, tell many "statements that ..." or "our mission is ...", create corporate culture, ) because you can't trust humans employees at 100%. Actually some of banks biggest failures were when only a few and in some cases only one person was in charge. https://en.wikipedia.org/wiki/List_of_corporate_collapses_and_scandals https://en.wikipedia.org/wiki/List_of_corporate_collapses_an... How is that different from LLMs?

captn3m0 3y ago

Professional norms differ between countries. What might be unthinkable for a doctor in US might be meh for a professional in India. For eg, a judge in India used ChatGPT for a bail hearing. https://www.livemint.com/news/india/this-indian-court-has-used-chatgpt-on-a-criminal-case/amp-11679977632552.html https://www.livemint.com/news/india/this-indian-court-has-us...

npsomaratna 3y ago

Disagree. As a lawyer I use LLMs with RAG to help me surface information all the time. Often, this allows me to find niche case law that I just wouldn't have had the time to find on my own. However, I double-check everything, and read all the original sources. LLMs are best treated as the AI equivalent of a human assistant who is knowledgeable and fast, but also inexperienced, and thus, prone to making mistakes. You won't throw out the work of such an assistant-it'll still save you hours of effort. However, you won't take the work at face value either.

jebarker 3y ago
I use LLMs for helping with coding it's proving invaluable. I see it very similarly, an inexperienced assistant with very broad knowledge. I also find that they don't make too many mistakes in tasks such as refactoring or finding bugs, it's when you ask it to just wholesale generate code for you that you hit problems. If I were to just take the code as is, not test it and not check I understand it then use it it's me that would be making the mistake not the LLM.
arrowsmith 3y ago
What's RAG?
- simonw 3y ago
  Retrieval Augmented Generation. It's the trick where you take a question from a user, search for documents that match that question, stuff as many of the relevant chunks of content from those documents as you can into the prompt (usually 4,000 or 8,000 tokens, but Claude can go up to 100,000) and then say to the LLM "Based on this context, answer this question: QUESTION". I wrote about one way to implement that here: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer...
squeaky-clean 3y ago
I have a friend in law school at the moment, and while he obviously can't use AI for school, multiple professors of his have recommended he get familiar with using them now so he'll be efficient at using them after passing the bar.

naasking 3y ago

> Real world industry wont stand for "hallucinated" outputs Of course they will, if the other benefits are large enough. Checking factual accuracy of a large corpus can often be considerably simpler than generating the large corpus to begin with.

gryn 3y ago

funny you mention lawyers when just not long ago some of them made the news because they trusted chatGPT which gave them a imaginary precedent. just because a small percentage technically oriented people know the limitations of LLMs doesn't mean that the rest do. people have a tendency to anthropomorphize things, so they naturally think that LLMs think in the same way other humans they know think. edit: first link I found about it, there was also some posts about it here in HN. https://www.abc.net.au/news/2023-06-09/lawyers-blame-chatgpt-for-tricking-them-into-citing-fake-cases/102462028 https://www.abc.net.au/news/2023-06-09/lawyers-blame-chatgpt...

ryanklee 3y ago

Real world industry already relies heavily on hallucinated outputs. They are called humans. One of the main differences between people who see the astonishing value of LLMs right now and people who are some combo of skeptical, dismissive, and indignant, is the expectation that for something to be a valuable source of information, it has to be factually accurate every time. The entirety of civilization has been built on the back of inaccurate sources of information. That will never change and it absolutely can't, because (1) factual accuracy is not something that can be determined by consensus in a variety of significant cases, and (2) factual accuracy as a concept itself does not have a consensus definition, operationally or in the abstract. Absolutely frustrating to see these topics addressed as if thousands of years of intense thinking around truth and factual accuracy has not taken place. The results of those inquiries do not support the basic assumptions of these conversations (i.e. that factual accuracy is amenable to exhaustive algorithmic verification).

JPLeRouzic 3y ago

If you have employees, you also need to create transverse structures (make people work in teams, set check lists, QA, HR, accounting, create corporate charters on gender equality and many other topics, tell many "statements that ..." or "our mission is ...", create corporate culture, ) because you can't trust humans employees at 100%. Actually some of banks biggest failures were when only a few and in some cases only one person was in charge. https://en.wikipedia.org/wiki/List_of_corporate_collapses_and_scandals https://en.wikipedia.org/wiki/List_of_corporate_collapses_an... How is that different from LLMs?