5 ms·

Disagree. As a lawyer I use LLMs with RAG to help me surface information all the time. Often, this allows me to find niche case law that I just wouldn't have ha

Disagree. As a lawyer I use LLMs with RAG to help me surface information all the time. Often, this allows me to find niche case law that I just wouldn't have had the time to find on my own. However, I double-check everything, and read all the original sources. LLMs are best treated as the AI equivalent of a human assistant who is knowledgeable and fast, but also inexperienced, and thus, prone to making mistakes. You won't throw out the work of such an assistant-it'll still save you hours of effort. However, you won't take the work at face value either.

jebarker 3y ago
I use LLMs for helping with coding it's proving invaluable. I see it very similarly, an inexperienced assistant with very broad knowledge. I also find that they don't make too many mistakes in tasks such as refactoring or finding bugs, it's when you ask it to just wholesale generate code for you that you hit problems. If I were to just take the code as is, not test it and not check I understand it then use it it's me that would be making the mistake not the LLM.
arrowsmith 3y ago
What's RAG?
- simonw 3y ago
  Retrieval Augmented Generation. It's the trick where you take a question from a user, search for documents that match that question, stuff as many of the relevant chunks of content from those documents as you can into the prompt (usually 4,000 or 8,000 tokens, but Claude can go up to 100,000) and then say to the LLM "Based on this context, answer this question: QUESTION". I wrote about one way to implement that here: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer...
squeaky-clean 3y ago
I have a friend in law school at the moment, and while he obviously can't use AI for school, multiple professors of his have recommended he get familiar with using them now so he'll be efficient at using them after passing the bar.

jebarker 3y ago

I use LLMs for helping with coding it's proving invaluable. I see it very similarly, an inexperienced assistant with very broad knowledge. I also find that they don't make too many mistakes in tasks such as refactoring or finding bugs, it's when you ask it to just wholesale generate code for you that you hit problems. If I were to just take the code as is, not test it and not check I understand it then use it it's me that would be making the mistake not the LLM.

arrowsmith 3y ago

What's RAG?

simonw 3y ago
Retrieval Augmented Generation. It's the trick where you take a question from a user, search for documents that match that question, stuff as many of the relevant chunks of content from those documents as you can into the prompt (usually 4,000 or 8,000 tokens, but Claude can go up to 100,000) and then say to the LLM "Based on this context, answer this question: QUESTION". I wrote about one way to implement that here: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer...

squeaky-clean 3y ago

I have a friend in law school at the moment, and while he obviously can't use AI for school, multiple professors of his have recommended he get familiar with using them now so he'll be efficient at using them after passing the bar.