3 ms·

Bug fixing via gpt interface is going to be fun! It needs to be far better than humans to avoid the following situation, and at that point these models have tak

Bug fixing via gpt interface is going to be fun! It needs to be far better than humans to avoid the following situation, and at that point these models have taken over basically every office job, so at that point you wont have to go through this process anyway. You: Here is a bug: [explanation] GPT: Ok, fixed it! You: It is still there. Gpt: Ok, now fixed for real! You: It is still there...

PaulRobinson 3y ago
You're right, and I like the joke, but we should be clar that LLM != GPT != ChatGPT. There's a whole field emerging of prompt engineers, but I think what's desperately missing is good quality integrated UX engineering. Being able to write a comment and/or a method signature and having an LLM spit out a generated piece of code is kinda cool, but the interface for showing that the code is buggy needs refining. I think that's likely to be quickly showing a test case: given these inputs I'm expecting this output. LLM refactors against all tests, and you iterate together towards the correct solution. That's one small example of how we can start to think about the future UX that isn't about having a "chat" about your code. Similar UX thought is needed in other disciplines, it'll be interesting to see how this domain evolves.
- yunwal 3y ago
  We should probably also be clear that LLMs aren’t the only ML models that can learn to code. If hands-on programming is automated away, it’s not obvious that it’ll be an LLM that does it.
ilaksh 3y ago
I have already built a system that puts ChatGPT in a loop with the output of programs and running commands. It can be pretty effective. It's even more effective when you can limit the domain somewhat to things that can fit in the context window, which eliminates out of date API information. We will now shortly be able to add visual feedback of the applications into the loop.

PaulRobinson 3y ago

You're right, and I like the joke, but we should be clar that LLM != GPT != ChatGPT. There's a whole field emerging of prompt engineers, but I think what's desperately missing is good quality integrated UX engineering. Being able to write a comment and/or a method signature and having an LLM spit out a generated piece of code is kinda cool, but the interface for showing that the code is buggy needs refining. I think that's likely to be quickly showing a test case: given these inputs I'm expecting this output. LLM refactors against all tests, and you iterate together towards the correct solution. That's one small example of how we can start to think about the future UX that isn't about having a "chat" about your code. Similar UX thought is needed in other disciplines, it'll be interesting to see how this domain evolves.

yunwal 3y ago
We should probably also be clear that LLMs aren’t the only ML models that can learn to code. If hands-on programming is automated away, it’s not obvious that it’ll be an LLM that does it.

ilaksh 3y ago

I have already built a system that puts ChatGPT in a loop with the output of programs and running commands. It can be pretty effective. It's even more effective when you can limit the domain somewhat to things that can fit in the context window, which eliminates out of date API information. We will now shortly be able to add visual feedback of the applications into the loop.