3 ms·

This argument is disingenuous and distracts rather than addresses the point. Yes, it is possible for a compiler to have a bug. No, that is I’m mo way analogous

This argument is disingenuous and distracts rather than addresses the point. Yes, it is possible for a compiler to have a bug. No, that is I’m mo way analogous to AI producing buggy code. I’ve experienced maybe two compiler bugs in my twenty year career. I have experienced countless AI mistakes - hundreds? Thousands? Already. These are not the same and it has the whiff of sales patter trying to address objections. Please stop.

LiamPowell 9mo ago
I'm not arguing that LLMs are at a point today where we can blindly trust their outputs in most applications, I just don't think that 100% correct output is necessarily a requirement for that. What it needs to be is correct often enough that the cost of reviewing the output far outweighs the average cost of any errors in the output, just like with a compiler. This even applies to human written code and human mistakes, as the expected cost of errors goes up we spend more time on having multiple people review the code and we worry more about carefully designing tests.
- yourapostasy 9mo ago
  The challenge not addressed with this line of reasoning is the required sheer scale of output validation on the backend of LLM-generated code. Human hand-developed code was no great shakes at the validation front either, but the scale difference hid this problem. I’m hopeful what used to be tedious about the software development process (like correctness proving or documentation) becomes tractable enough with LLM’s to make the scale more manageable for us. That’s exciting to contemplate; think of the complexity categories we can feasibly challenge now!
- sampullman 9mo ago
  If natural language is used to specify work to the LLM, how can the output ever be trusted? You'll always need to make sure the program does what you want, rather than what you said.
  - rootnod3 9mo ago
    Just create a very specific and very detailed prompt that is so specific that it starts including instructions and you came up with the most expensive programming language.
    - reverius42 9mo ago
      It's not great that it's the most expensive (by far), but it's also by far the most expressive programming language.
      - rootnod3 9mo ago
        How is it more expressive? What is more expressive than Turing completeness?
        reverius42 9mo ago
        This is a non-sequitur. Almost all programming languages are Turing complete, but I think we'd all agree they vary in expressivity (e.g. x64 assembly vs. TypeScript). By expressivity I mean that you can say what you mean, and the more expressive the language is, the easier that is to do. It turns out saying what you mean is quite easy in plain English! The hard part is that English allows a lot of ambiguity. So the tradeoffs of how you express things are very different. I also want to note how remarkable it is that humans have built a machine that can effectively understand natural language.
  - signatoremo 9mo ago
    You trust your natural language instructions thousand times a day. If you ask for a large black coffee, you can trust that is more or less what you’ll get. Occasionally you may get something so atrocious that you don’t dare to drink, but generally speaking you trust the coffee shop knows what you want. It you insist on a specific amount of coffee brewed at a specific temperature, however, you need tools to measure. AI tools are similar. You can trust them because they are good enough, and you need a way (testing) to make sure what is produced meet your specific requirements. Of course they may fail for you, doesn’t mean they aren’t useful in other cases. All of that is simply common sense.

LiamPowell 9mo ago

I'm not arguing that LLMs are at a point today where we can blindly trust their outputs in most applications, I just don't think that 100% correct output is necessarily a requirement for that. What it needs to be is correct often enough that the cost of reviewing the output far outweighs the average cost of any errors in the output, just like with a compiler. This even applies to human written code and human mistakes, as the expected cost of errors goes up we spend more time on having multiple people review the code and we worry more about carefully designing tests.

yourapostasy 9mo ago
The challenge not addressed with this line of reasoning is the required sheer scale of output validation on the backend of LLM-generated code. Human hand-developed code was no great shakes at the validation front either, but the scale difference hid this problem. I’m hopeful what used to be tedious about the software development process (like correctness proving or documentation) becomes tractable enough with LLM’s to make the scale more manageable for us. That’s exciting to contemplate; think of the complexity categories we can feasibly challenge now!
sampullman 9mo ago
If natural language is used to specify work to the LLM, how can the output ever be trusted? You'll always need to make sure the program does what you want, rather than what you said.
- rootnod3 9mo ago
  Just create a very specific and very detailed prompt that is so specific that it starts including instructions and you came up with the most expensive programming language.
  - reverius42 9mo ago
    It's not great that it's the most expensive (by far), but it's also by far the most expressive programming language.
    - rootnod3 9mo ago
      How is it more expressive? What is more expressive than Turing completeness?
      - reverius42 9mo ago
        This is a non-sequitur. Almost all programming languages are Turing complete, but I think we'd all agree they vary in expressivity (e.g. x64 assembly vs. TypeScript). By expressivity I mean that you can say what you mean, and the more expressive the language is, the easier that is to do. It turns out saying what you mean is quite easy in plain English! The hard part is that English allows a lot of ambiguity. So the tradeoffs of how you express things are very different. I also want to note how remarkable it is that humans have built a machine that can effectively understand natural language.
- signatoremo 9mo ago
  You trust your natural language instructions thousand times a day. If you ask for a large black coffee, you can trust that is more or less what you’ll get. Occasionally you may get something so atrocious that you don’t dare to drink, but generally speaking you trust the coffee shop knows what you want. It you insist on a specific amount of coffee brewed at a specific temperature, however, you need tools to measure. AI tools are similar. You can trust them because they are good enough, and you need a way (testing) to make sure what is produced meet your specific requirements. Of course they may fail for you, doesn’t mean they aren’t useful in other cases. All of that is simply common sense.