5 ms·

I've been thinking about this for a while now, wrt two points: 1. This will be the end of traditional SWEs and the rise of the age of debuggers, human debugger

I've been thinking about this for a while now, wrt two points: 1. This will be the end of traditional SWEs and the rise of the age of debuggers, human debuggers who spend their days setting up breakpoints and figuring bugs in a sea of LLM generated code. 2. Hiring will switch from using Leetcode questions to "pull out your debugger and figure out what's wrong with this code".

w-m 3y ago
What makes you think the LLM couldn’t run a debugging session from the content of a JIRA ticket and the whole code base + documentation?
- eichin 3y ago
  Having never seen it, or anything even close to it. (Of course, I'm a little biased by seeing product demos that don't even get "add another item to this list of command line arguments" right; maybe if everyone already believes it works, nobody bothers to actually sell that?)
- spookie 3y ago
  If the codebase is anything more than a simple Python project... I don't think that'll happen. It just doesn't scale that well. Hell, GPT-4 can't make sense of my own projects.
- mike_hearn 3y ago
  Who can say what's possible but there are very few "debugging transcripts" for neural nets to train on out there. So it'd have to sort of work out how to operate a debugger via functions, understand the codebase (perhaps large parts of it), intuit what's going wrong, be able to modify the codebase, etc. Lots of work to do there.
- Workaccount2 3y ago
  Nobody wants to face the potential that their skills are going to greatly decrease in value.
eichin 3y ago
The first interview where I was handed some (intentionally) broken C code and gdb was about 15 years ago. I'm not sure that part is a change in developer workflow (this may apply more in systems and embedded though.) I've been paying attention to this too (mostly by following Simon Willison) and I'm still solidly in the "get back to me when this stuff can successfully review a pull request or even interpret a traceback" camp...
ukuina 3y ago
Yep, time to start adding source_file.prompt sidecar files next to each generated module so debugging sessions can start at the same initial condition.

w-m 3y ago

What makes you think the LLM couldn’t run a debugging session from the content of a JIRA ticket and the whole code base + documentation?

eichin 3y ago
Having never seen it, or anything even close to it. (Of course, I'm a little biased by seeing product demos that don't even get "add another item to this list of command line arguments" right; maybe if everyone already believes it works, nobody bothers to actually sell that?)
spookie 3y ago
If the codebase is anything more than a simple Python project... I don't think that'll happen. It just doesn't scale that well. Hell, GPT-4 can't make sense of my own projects.
mike_hearn 3y ago
Who can say what's possible but there are very few "debugging transcripts" for neural nets to train on out there. So it'd have to sort of work out how to operate a debugger via functions, understand the codebase (perhaps large parts of it), intuit what's going wrong, be able to modify the codebase, etc. Lots of work to do there.
Workaccount2 3y ago
Nobody wants to face the potential that their skills are going to greatly decrease in value.

eichin 3y ago

The first interview where I was handed some (intentionally) broken C code and gdb was about 15 years ago. I'm not sure that part is a change in developer workflow (this may apply more in systems and embedded though.) I've been paying attention to this too (mostly by following Simon Willison) and I'm still solidly in the "get back to me when this stuff can successfully review a pull request or even interpret a traceback" camp...

ukuina 3y ago

Yep, time to start adding source_file.prompt sidecar files next to each generated module so debugging sessions can start at the same initial condition.