3 ms·

This article compares the hardness of program synthesis to the utility of chatbots to show chatbots fall short. The difference is we don’t expect chatbots to b

This article compares the hardness of program synthesis to the utility of chatbots to show chatbots fall short. The difference is we don’t expect chatbots to be 100% right but we do expect program synthesis to be 100% right. For chatbots, 99% is amazing in terms of utility. That other 1% is really hard to get. Given the limitations of English being able to robustly specify a program, thus requiring constraints for program synthesis to be formal descriptions of specifications, the author is committing a category error in comparing two incommensurable solutions to two distinct problems.

jebarker 2y ago
> For chatbots, 99% is amazing in terms of utility. Is it really though? If it's 99% of generated tokens, then 1 out of 100 being wrong is not great for code generation since you're often going to generate more than that. But let's suppose it's 1 in 100 whole functions or programs that's wrong. Unless there's a way to automatically verify when a function is wrong the error rate basically makes full automation, e.g. automatically fixing GitHub issues, out of the question. So it seems like at 99% we're left with a system that requires constant vigilance from the user to check it's work. This is a far cry from the promise of true program synthesis. What would be amazingly useful is a 99% reliable system that can tell you when it can't answer the question correctly instead of providing a wrong answer.
- simonw 2y ago
  If you expect to be able to prompt a chatbot and then not TEST that the code it wrote for you works, you're going to have a bad time. If you're willing to put in that QA work yourself the productivity boost you can get them from is immense. And honestly, working with the best human programmer in the world won't guarantee that the software they produce is the exact software that you wanted. Details are always lost in the communication between the person who wants the software and the person who builds it What matters most is the dialog and speed of iteration that you can achieve to converge on the right product.
  - jebarker 2y ago
    I think the reason that I haven't felt the productivity boost is because my work is mostly debugging and fixing existing code in a complicated codebase. Rarely is it writing new greenfield code. LLMs have so far struggled to be useful in this setting as it's more like the QA part of the workflow you're suggesting.
  - robertlagrant 2y ago
    > If you're willing to put in that QA work yourself the productivity boost you can get them from is immense. Isn't that work more than half the work of programming? I seem to find myself spending much more time on test code than on application code.
  - YeGoblynQueenne 2y ago
    >> If you expect to be able to prompt a chatbot and then not TEST that the code it wrote for you works, you're going to have a bad time. Then what do you need the chatbot for? I have to do that with my own code all the time anyway. What's the benefit?

jebarker 2y ago

> For chatbots, 99% is amazing in terms of utility. Is it really though? If it's 99% of generated tokens, then 1 out of 100 being wrong is not great for code generation since you're often going to generate more than that. But let's suppose it's 1 in 100 whole functions or programs that's wrong. Unless there's a way to automatically verify when a function is wrong the error rate basically makes full automation, e.g. automatically fixing GitHub issues, out of the question. So it seems like at 99% we're left with a system that requires constant vigilance from the user to check it's work. This is a far cry from the promise of true program synthesis. What would be amazingly useful is a 99% reliable system that can tell you when it can't answer the question correctly instead of providing a wrong answer.

simonw 2y ago
If you expect to be able to prompt a chatbot and then not TEST that the code it wrote for you works, you're going to have a bad time. If you're willing to put in that QA work yourself the productivity boost you can get them from is immense. And honestly, working with the best human programmer in the world won't guarantee that the software they produce is the exact software that you wanted. Details are always lost in the communication between the person who wants the software and the person who builds it What matters most is the dialog and speed of iteration that you can achieve to converge on the right product.
- jebarker 2y ago
  I think the reason that I haven't felt the productivity boost is because my work is mostly debugging and fixing existing code in a complicated codebase. Rarely is it writing new greenfield code. LLMs have so far struggled to be useful in this setting as it's more like the QA part of the workflow you're suggesting.
- robertlagrant 2y ago
  > If you're willing to put in that QA work yourself the productivity boost you can get them from is immense. Isn't that work more than half the work of programming? I seem to find myself spending much more time on test code than on application code.
- YeGoblynQueenne 2y ago
  >> If you expect to be able to prompt a chatbot and then not TEST that the code it wrote for you works, you're going to have a bad time. Then what do you need the chatbot for? I have to do that with my own code all the time anyway. What's the benefit?