12 ms·

Remember the phrase "developers spend 10% of the time writing code and 90% of the time debugging?" Even if it isn't 90%, most developers like writing code more

Remember the phrase "developers spend 10% of the time writing code and 90% of the time debugging?" Even if it isn't 90%, most developers like writing code more than debugging, so most would prefer to automate the latter. AI translating natural language into code probably isn't as important as AI generating bug-free code and/or debugging its code. Even GPT-4 struggles with this: sometimes you point out a bug and it works, but sometimes it just can't find the issue and starts hallucinating even more as it gets confused. Everyone's trying to train GPT models to write code, but maybe we should be training them how to use a debugger. Though its a lot harder to map text generation to debugging... Also, it's a bit ironic how one way to prevent bugs is using stronger type systems and formal methods. But, AI is particularly bad at formal methods. But maybe with a system like MCTS combined with much faster generation...

Hirrolot 3y ago
I'm imagining an AI based on a deductive system rather than sequential text generation. This is roughly how "strong type systems" work, and so it might be simpler to map this model to formal methods. By the way, if you spend 90% of time debugging your code, I think that's really sad. Either the programming language sucks, or the codebase sucks, or both. I probably spend 10-20% max of my time on debugging.
- hnfong 3y ago
  The "strong type systems" work by having you design the types correctly in the first place. If designed correctly they work wonders, and 90+% of the time if the code compiles it's probably correct. That's a big "if" though, and most of your time writing "strong type systems" is coming up with the correct type structure. The one thing worse than debugging code is debugging "types". Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python.
  - raincole 3y ago
    > Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python. Everyone complains how slowly Rust compiles. But at least its error messages are much better.
  - lmm 3y ago
    > Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python. That's definitely in the "programming language sucks" category. Good languages make it easy to debug problems with your types.
  - Hirrolot 3y ago
    > The "strong type systems" work by having you design the types correctly in the first place. I was responding to a comment that was suggesting letting an AI to do that. > The one thing worse than debugging code is debugging "types". It depends on a language. In languages with first-class types, we have the same tools to debug types. Classical C++ templates are more akin to a macro system than a type system.
  - mikebenfield 3y ago
    > Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python. C++ is sort of the canonical example of a language with a terrible type system, so it's not really the right place to look for this comparison.
Turskarama 3y ago
Training an AI to write unit tests (and refactor code to be more testable) would be a real game changer.
- riffraff 3y ago
  Copilot does a decent job with "dumb" unit tests, e.g. "when the condition X is Y then Z". But it obviously doesn't have knowledge of your domain, and I don't think any LLM could do a perfect job without that.
- byby 3y ago
  I don't think you get it. The game has already changed. And nobody trained the AI to write unit tests or refactor code. It could do it as emergent behavior. It happened so fast that so many people are in denial and many people aren't even asking the obvious questions. Simply ASK chatGPT to refactor your code to be more unit testable and IT Can do it. Of course it clearly has your typical LLM problems but it is roughly 50% there. The game has changed. If AI gets to 100%, not only will the game be changed, but humans will no longer be part of the game. See what I asked chatGPT: ME: def (x: int): for i in range(x): print(i) Can you refactor the code so it can be more unit testable? CHATGPT: Certainly! To make the code more unit testable, we can refactor it into a function that returns a list of integers instead of directly printing them. Here's the refactored code: python def generate_number_list(x: int) -> List[int]: number_list = [] for i in range(x): number_list.append(i) return number_list In this refactored code, we've created a function called generate_number_list that takes an integer x as input. Instead of printing each number, we use a number_list variable to store the generated numbers by appending them in the loop. Finally, we return the number_list. By returning the list instead of printing the numbers directly, we make the code more testable because we can now write unit tests that assert the correctness of the generated list. If boggles my mind that I see threads of people saying "what if we train LLMs to do this or to do that?" or whatever. Guys, just ask the LLM to do what you want in plain English. It can do it.
  - mjburgess 3y ago
    That isn't the game. The game is designing software to requirements. It's writing literature for a new era. It's creating X for A audience with N vauge unspecified needs -- where X is a complex product made of many parts, involving many people, with shifting and changing problems/solutions/requirements. The game was never writing the stack overflow answer -- that was already written.
byby 3y ago
>Everyone's trying to train GPT models to write code, but maybe we should be training them how to use a debugger. Though its a lot harder to map text generation to debugging... We can actually try this now. Literally tell the LLM what you want to do and work with it. See how far you can take it. You will of course be asking the LLM for debug line inputs and you will be providing it with outputs as you run the debugger yourself.
- antihipocrat 3y ago
  Or just learn to write it ourselves? If it takes the same amount of time to work with the LLM as coding it from scratch then I'd prefer to improve my coding ability while I do the work.
  - byby 3y ago
    I'm just saying the tech is already here. The core engine can do it. Before you go on and write such a system it's better to test if the LLM can do debugging to an efficacy level that we require. I don't think anyone has tried this yet and we do know LLMs have certain issues. But make no mistake, the possibility that an LLM knows how to debug programs is actually quite high. If it can do this: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ it can likely debug a program, but I can't say definitively because I'm too lazy to try.
    - antihipocrat 3y ago
      Thanks for sharing that link, from that example I can see how LLMs could be used to speed up the learning process. I do wonder though whether the methods that the LLM provides are reflective of best practice or whether they are simply what happens to be most written in SO or blog posts.
      - pixl97 3y ago
        So an interesting behavior of LLMs is something like the following. "Write C++ code that sorts the following inputs" versus "Write (version) C++ code that sorts the following inputs, ensure the code is secure and uses best practices" And you'll likely get a different answer.
raincole 3y ago
> Also, it's a bit ironic how one way to prevent bugs is using stronger type systems and formal methods. But, AI is particularly bad at formal methods. It kinda works tho. In my anecdote Copilot works much better with C# than Python simply because I can write the signature of a function and that it generates the content. (I know Python has type annotations too, but Copilot just isn't as smart as with C#. Perhaps because there isn't enough training data in typed Python?)
- moffkalast 3y ago
  Hmm, so the endgame would be the most strongly defined and strict language so LLMs can immediately see and fix mistakes, but automate the tedium of writing it by having them as an intermediary?
  - raincole 3y ago
    The endgame is either the elimination of jobs or the elimination of human beings. But yes, what you described matches my experience. I'd say Copilot benefits those "tedious but relatively strict" languages the most.
ly3xqhl8g9 3y ago
Train them how to hallucinate the debugger and the world. Forget about running the debugger only for your program, simulate the entire stack, networking and databases included, for a true timeless debugger.
mattlutze 3y ago
Most software devs would prefer a debugger, but most non-software devs would probably prefer a virtual developer. If a technical product manager was able to generate the code needed for a feature using some combination of LLMs and CoPilots—even for proof of concepts, for example—the business would need to hire fewer developers and saves significant operating costs. Extending it to your everyday person. Aunt Jane bought a new washing machine and wants to add it into her smart house. Either she's gotta call the niece to hook it all up, write the integration to Home Assistant, write automations for her iOS devices, etc, or, she asks her favorite LLM / Co-Pilot for help and it step-by-step walks her through it directly, with code, with individual troubleshooting. There's business in both use cases, but the total addressable market is quite a bit bigger for one of them.
- auggierose 3y ago
  How about the washing machine just plugs itself seamlessly into her smart house, without Aunt Jane having to do anything than a quick identity check on her phone? Probably needs to be an Apple washing machine, though.
  - brokenkebaby 3y ago
    ...with expensive quickly deteriorating cable
    - moffkalast 3y ago
      Those are sold extra. Also it only has an Apple Socket™ by default, so you'll need a dongle to have it fit common outlets.
  - crabmusket 3y ago
    You mean some kind of interoperability standard?? We'll invent AGI before that becomes a reality!
- discreteevent 3y ago
  >it step-by-step walks her through it directly, with code This is exactly where it completely breaks down. The AI might as well show her some hieroglyphics.
blibble 3y ago
I have worked on one 30 year old system that was written by people that knew what they were doing it was then maintained by at least 20 different people that had no idea what they were doing very little of the altered logic makes any sense and 95% of the time consists of trying to reverse engineer what they were trying to do then fix it up without breaking other parts of the codebase as many of the logic bugs interact with each other and end up cancelling out with the garbage generated by LLMs... I suspect all development is going to turn into this
- Guthur 3y ago
  My thought exactly, and compounded that we will now have far less practiced coders as they all generate the banal they should be learning with.
- nonethewiser 3y ago
  Or maybe it can help you understand the code
  - blibble 3y ago
    9 times out of 10 there is nothing to understand the logic is simply broken
- pydry 3y ago
  You mean one day rich executives are going to be in desperate need of and highly reliant upon skilled developers who can understand and maintain their mission critical systems? Sounds awful. I was hoping for the future where we would all be out of a job.
  - bryanrasmussen 3y ago
    check the wages for Cobol developers in your area recently? on edit: https://www.zippia.com/cobol-programmer-jobs/salary/# https://www.zippia.com/cobol-programmer-jobs/salary/#
    - 3y ago
the_other 3y ago
> Everyone's trying to train GPT models to write code, but maybe we should be training them how to use a debugger. Though its a lot harder to map text generation to debugging... AI lay-person, web developer here. This hits at something that's been nagging at me for a few weeks. LLMs get labelled "intelligent", but they're "just" spitting out word/symbol patterns. They don't comprehend the meaning of the patterns (as best I have come to understand it from reading news). Deeper than this, "intelligence" isn't rooted in words. It's rooted in experience, logic and their interaction. The words are a side-effect; they're a way to label the experiences. We're not training LLMs with experiences, only a reference to experience had by others, which may or may not map to the experience of the interlocutor. The LLM has no experience and limited/no logic to wrestle those experiences into wisdom. Debugging is a "pursuit of wisdom" activity. Statistics, which underpin LLMs often identify truths (we wouldn't use them in science if they didn't), but they also hide truths. You need to clean data, take unusual slices of it, recombine it, and otherwise process it in order to uncover the true truths. I'm not sure LLMs can do that yet (if at all), and strongly doubt they can without _experience_ (sensory input, intention, failure etc).
- Ygg2 3y ago
  > LLMs get labelled "intelligent", but they're "just" spitting out word/symbol patterns. I think the word you are looking is knowledgeable. They have huge knowledge repository and can remix it in whatever way they please, but I don't think if you gave them a novel problem (outside their realm of knowledge) they could solve it.
  - gaogao 3y ago
    The unicorn example from MSFT's Sparks of AGI paper, https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712, probably counts as a novel problem. As another example, ImageBlind, https://ai.facebook.com/blog/imagebind-six-modalities-binding-ai/ https://ai.facebook.com/blog/imagebind-six-modalities-bindin..., learned modality translations explicitly not in its training set such as text -> sound.
- Version467 3y ago
  > They don't comprehend the meaning of the patterns (as best I have come to understand it from reading news). It is not obvious to me that this is true. The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token. You'll find different opinions as to wether this happens with todays llm's, but it's definitely not a clear cut issue.
bjourne 3y ago
Spot the bug in the following C function for reading files: bool files_read(const char *path, char **data, size_t *size) { FILE *f = fopen(path, "rb"); if (!f) { return false; } fseek(f, 0, SEEK_END); size_t n = (size_t)ftell(f); rewind(f); if (size) { *size = n; } if (!data) { goto ok; } *data = (char*)malloc(sizeof(char)*(n + 1)); if (fread(*data, 1, n, f) != n) { free(*data); return false; } *data[n] = '\0'; ok: fclose(f); return true; } ChatGPT solves this task instantly. Many, even experienced C developers, would have trouble finding the bug. And the lack of error checking on malloc is not a bug.
- golergka 3y ago
  Just for reference, copied this snippet in GPT-4, it generated this answer in 1,5 seconds: > The issue in this code is related to memory access. The line data[n] = '\0'; should be (data)[n] = '\0';. And then went on to generate corrected code and this explanation: > The original expression, data[n], first applies the array subscript operation, which is equivalent to (data + n), and then tries to dereference the resulting pointer, which is not the intention here. This will likely lead to a segmentation fault because data is a pointer to pointer, and data[n] will try to access memory that has not been allocated. > The corrected expression, (*data)[n], first dereferences data to get the allocated char array, and then applies the array subscript operation to set the null terminator at the correct location.
  - ChatGTP 3y ago
    Just wondering if you think this type of bug hasn't been covered at least 50x in many online forums, I'd be very surprised? I'm not trying to say this isn't impressive, but seems fairly obvious and more like something a linter / analyzer would pickup ?
    - almog 3y ago
      Even if this isn't based on similar questions that can be found online, a bug in existing systems usually wouldn't introduce itself by framing an isolated section of code. More often than not it'd be through indirection or a side effect that covers a larger execution path.
pornel 3y ago
Rust is 90% arguing with the compiler and 10% debugging. And yesterday GPT-4 solved a WTF lifetime issue for me pulling a trick I wouldn't have thought about.
- Ygg2 3y ago
  Probably because its corpus of knowledge includes someone solving it like that.
  - pixl97 3y ago
    I'm not really a programmer, but when I write code for stuff I need this is how I operate. The novel idea I want to solve is the end goal, not the intermediate code that I'm gluing together that other people have generally created.
- ChatGTP 3y ago
  What was the WTF lifetime issue?
  - pornel 3y ago
    I wanted to compare function pointers, but that's tricky due to unique function types, disallowed non-primitive casts, plus apparently not allowed for types with higher-ranked lifetimes (for<'a> fn(&'a)). GPT-4 came up with using a type alias (type FnComparable<'a>) that added necessary coercions and gave them equal lifetime.
Ygg2 3y ago
Honestly ChatGPT-4 sucks at writing code (if similar code isn't in its corpus of knowledge). I asked it to write a YAML parser in pure Rust. The first thing it did was write this: pub enum YamlNode { Scalar(String), Sequence(Vec<YamlNode>), Map(HashMap<String, YamlNode>), } This is what I expect a freshman in college to write as their first Yaml parser. Literally unusable as a starting point. I did try to correct it but it tripped up and somehow managed to fuck that up to.
- hderms 3y ago
  Why does it suck, because it's doing a lot of allocations?
  - Ygg2 3y ago
    No. You have a fair point, but not my biggest gripe. It's much more fundamental to what YAML is. YAML allows YamlNodes as keys in a map (aka complex keys). E.g. ? [ a, b] : "complex key, scalar value" You asked it for YAML parser and it gave you a shitty JSON parser. Also this is the most banal thing about YAML, something that should be obvious just browsing the examples. And I gave it test suite and link to YAML spec. Most work in YAML is around super obscure corner cases.
api 3y ago
Stronger type systems really do prevent bugs. In Rust I probably spend 20% of the time at most debugging. Of course you pay for that by the fact that it takes longer up front to write Rust because you have to think about it more, but overall I think it’s a net win. You also get the bonus of nasty bugs like crash bugs and exploitable memory problems become almost entirely a thing of the past. You can ship new software you know is very reliable.
bravura 3y ago
I’ve said it before and I’ll say it again. A time traveling retroactive debugger integrated with an LLM for interrogating what happened and why, would be awesome.
crabmusket 3y ago
This a classic case of understanding programming as an activity that is primarily about text production. This is what Naur was arguing against in Programming As Theory Building [1]. Taking Nair's viewpoint, coding AI will be useful only to the extent that it assists programmers to build and employ their theory of the program. Or to itself develop a useful theory of a specific program, not just programs in general. [1] https://gist.github.com/onlurking/fc5c81d18cfce9ff81bc968a7f342fb1 https://gist.github.com/onlurking/fc5c81d18cfce9ff81bc968a7f...
AnimalMuppet 3y ago
I think that training GPT to use a debugger is going to be hard. For many debug problems, the amount of context you have to keep is going to be really hard for GPT. > But, AI is particularly bad at formal methods. AI? Or GPT? Earlier, reasoning-based attempts at AI seem like they would be a natural fit for formal methods.

Hirrolot 3y ago

I'm imagining an AI based on a deductive system rather than sequential text generation. This is roughly how "strong type systems" work, and so it might be simpler to map this model to formal methods. By the way, if you spend 90% of time debugging your code, I think that's really sad. Either the programming language sucks, or the codebase sucks, or both. I probably spend 10-20% max of my time on debugging.

hnfong 3y ago
The "strong type systems" work by having you design the types correctly in the first place. If designed correctly they work wonders, and 90+% of the time if the code compiles it's probably correct. That's a big "if" though, and most of your time writing "strong type systems" is coming up with the correct type structure. The one thing worse than debugging code is debugging "types". Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python.
- raincole 3y ago
  > Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python. Everyone complains how slowly Rust compiles. But at least its error messages are much better.
- lmm 3y ago
  > Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python. That's definitely in the "programming language sucks" category. Good languages make it easy to debug problems with your types.
- Hirrolot 3y ago
  > The "strong type systems" work by having you design the types correctly in the first place. I was responding to a comment that was suggesting letting an AI to do that. > The one thing worse than debugging code is debugging "types". It depends on a language. In languages with first-class types, we have the same tools to debug types. Classical C++ templates are more akin to a macro system than a type system.
- mikebenfield 3y ago
  > Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python. C++ is sort of the canonical example of a language with a terrible type system, so it's not really the right place to look for this comparison.

Turskarama 3y ago

Training an AI to write unit tests (and refactor code to be more testable) would be a real game changer.

riffraff 3y ago
Copilot does a decent job with "dumb" unit tests, e.g. "when the condition X is Y then Z". But it obviously doesn't have knowledge of your domain, and I don't think any LLM could do a perfect job without that.
byby 3y ago
I don't think you get it. The game has already changed. And nobody trained the AI to write unit tests or refactor code. It could do it as emergent behavior. It happened so fast that so many people are in denial and many people aren't even asking the obvious questions. Simply ASK chatGPT to refactor your code to be more unit testable and IT Can do it. Of course it clearly has your typical LLM problems but it is roughly 50% there. The game has changed. If AI gets to 100%, not only will the game be changed, but humans will no longer be part of the game. See what I asked chatGPT: ME: def (x: int): for i in range(x): print(i) Can you refactor the code so it can be more unit testable? CHATGPT: Certainly! To make the code more unit testable, we can refactor it into a function that returns a list of integers instead of directly printing them. Here's the refactored code: python def generate_number_list(x: int) -> List[int]: number_list = [] for i in range(x): number_list.append(i) return number_list In this refactored code, we've created a function called generate_number_list that takes an integer x as input. Instead of printing each number, we use a number_list variable to store the generated numbers by appending them in the loop. Finally, we return the number_list. By returning the list instead of printing the numbers directly, we make the code more testable because we can now write unit tests that assert the correctness of the generated list. If boggles my mind that I see threads of people saying "what if we train LLMs to do this or to do that?" or whatever. Guys, just ask the LLM to do what you want in plain English. It can do it.
- mjburgess 3y ago
  That isn't the game. The game is designing software to requirements. It's writing literature for a new era. It's creating X for A audience with N vauge unspecified needs -- where X is a complex product made of many parts, involving many people, with shifting and changing problems/solutions/requirements. The game was never writing the stack overflow answer -- that was already written.

byby 3y ago

>Everyone's trying to train GPT models to write code, but maybe we should be training them how to use a debugger. Though its a lot harder to map text generation to debugging... We can actually try this now. Literally tell the LLM what you want to do and work with it. See how far you can take it. You will of course be asking the LLM for debug line inputs and you will be providing it with outputs as you run the debugger yourself.

antihipocrat 3y ago
Or just learn to write it ourselves? If it takes the same amount of time to work with the LLM as coding it from scratch then I'd prefer to improve my coding ability while I do the work.
- byby 3y ago
  I'm just saying the tech is already here. The core engine can do it. Before you go on and write such a system it's better to test if the LLM can do debugging to an efficacy level that we require. I don't think anyone has tried this yet and we do know LLMs have certain issues. But make no mistake, the possibility that an LLM knows how to debug programs is actually quite high. If it can do this: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ it can likely debug a program, but I can't say definitively because I'm too lazy to try.
  - antihipocrat 3y ago
    Thanks for sharing that link, from that example I can see how LLMs could be used to speed up the learning process. I do wonder though whether the methods that the LLM provides are reflective of best practice or whether they are simply what happens to be most written in SO or blog posts.
    - pixl97 3y ago
      So an interesting behavior of LLMs is something like the following. "Write C++ code that sorts the following inputs" versus "Write (version) C++ code that sorts the following inputs, ensure the code is secure and uses best practices" And you'll likely get a different answer.

raincole 3y ago

> Also, it's a bit ironic how one way to prevent bugs is using stronger type systems and formal methods. But, AI is particularly bad at formal methods. It kinda works tho. In my anecdote Copilot works much better with C# than Python simply because I can write the signature of a function and that it generates the content. (I know Python has type annotations too, but Copilot just isn't as smart as with C#. Perhaps because there isn't enough training data in typed Python?)

moffkalast 3y ago
Hmm, so the endgame would be the most strongly defined and strict language so LLMs can immediately see and fix mistakes, but automate the tedium of writing it by having them as an intermediary?
- raincole 3y ago
  The endgame is either the elimination of jobs or the elimination of human beings. But yes, what you described matches my experience. I'd say Copilot benefits those "tedious but relatively strict" languages the most.

ly3xqhl8g9 3y ago

Train them how to hallucinate the debugger and the world. Forget about running the debugger only for your program, simulate the entire stack, networking and databases included, for a true timeless debugger.

mattlutze 3y ago

Most software devs would prefer a debugger, but most non-software devs would probably prefer a virtual developer. If a technical product manager was able to generate the code needed for a feature using some combination of LLMs and CoPilots—even for proof of concepts, for example—the business would need to hire fewer developers and saves significant operating costs. Extending it to your everyday person. Aunt Jane bought a new washing machine and wants to add it into her smart house. Either she's gotta call the niece to hook it all up, write the integration to Home Assistant, write automations for her iOS devices, etc, or, she asks her favorite LLM / Co-Pilot for help and it step-by-step walks her through it directly, with code, with individual troubleshooting. There's business in both use cases, but the total addressable market is quite a bit bigger for one of them.

auggierose 3y ago
How about the washing machine just plugs itself seamlessly into her smart house, without Aunt Jane having to do anything than a quick identity check on her phone? Probably needs to be an Apple washing machine, though.
- brokenkebaby 3y ago
  ...with expensive quickly deteriorating cable
  - moffkalast 3y ago
    Those are sold extra. Also it only has an Apple Socket™ by default, so you'll need a dongle to have it fit common outlets.
- crabmusket 3y ago
  You mean some kind of interoperability standard?? We'll invent AGI before that becomes a reality!
discreteevent 3y ago
>it step-by-step walks her through it directly, with code This is exactly where it completely breaks down. The AI might as well show her some hieroglyphics.

blibble 3y ago

I have worked on one 30 year old system that was written by people that knew what they were doing it was then maintained by at least 20 different people that had no idea what they were doing very little of the altered logic makes any sense and 95% of the time consists of trying to reverse engineer what they were trying to do then fix it up without breaking other parts of the codebase as many of the logic bugs interact with each other and end up cancelling out with the garbage generated by LLMs... I suspect all development is going to turn into this

Guthur 3y ago
My thought exactly, and compounded that we will now have far less practiced coders as they all generate the banal they should be learning with.
nonethewiser 3y ago
Or maybe it can help you understand the code
- blibble 3y ago
  9 times out of 10 there is nothing to understand the logic is simply broken
pydry 3y ago
You mean one day rich executives are going to be in desperate need of and highly reliant upon skilled developers who can understand and maintain their mission critical systems? Sounds awful. I was hoping for the future where we would all be out of a job.
- bryanrasmussen 3y ago
  check the wages for Cobol developers in your area recently? on edit: https://www.zippia.com/cobol-programmer-jobs/salary/# https://www.zippia.com/cobol-programmer-jobs/salary/#
  - 3y ago

the_other 3y ago

> Everyone's trying to train GPT models to write code, but maybe we should be training them how to use a debugger. Though its a lot harder to map text generation to debugging... AI lay-person, web developer here. This hits at something that's been nagging at me for a few weeks. LLMs get labelled "intelligent", but they're "just" spitting out word/symbol patterns. They don't comprehend the meaning of the patterns (as best I have come to understand it from reading news). Deeper than this, "intelligence" isn't rooted in words. It's rooted in experience, logic and their interaction. The words are a side-effect; they're a way to label the experiences. We're not training LLMs with experiences, only a reference to experience had by others, which may or may not map to the experience of the interlocutor. The LLM has no experience and limited/no logic to wrestle those experiences into wisdom. Debugging is a "pursuit of wisdom" activity. Statistics, which underpin LLMs often identify truths (we wouldn't use them in science if they didn't), but they also hide truths. You need to clean data, take unusual slices of it, recombine it, and otherwise process it in order to uncover the true truths. I'm not sure LLMs can do that yet (if at all), and strongly doubt they can without _experience_ (sensory input, intention, failure etc).

Ygg2 3y ago
> LLMs get labelled "intelligent", but they're "just" spitting out word/symbol patterns. I think the word you are looking is knowledgeable. They have huge knowledge repository and can remix it in whatever way they please, but I don't think if you gave them a novel problem (outside their realm of knowledge) they could solve it.
- gaogao 3y ago
  The unicorn example from MSFT's Sparks of AGI paper, https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712, probably counts as a novel problem. As another example, ImageBlind, https://ai.facebook.com/blog/imagebind-six-modalities-binding-ai/ https://ai.facebook.com/blog/imagebind-six-modalities-bindin..., learned modality translations explicitly not in its training set such as text -> sound.
Version467 3y ago
> They don't comprehend the meaning of the patterns (as best I have come to understand it from reading news). It is not obvious to me that this is true. The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token. You'll find different opinions as to wether this happens with todays llm's, but it's definitely not a clear cut issue.

bjourne 3y ago

Spot the bug in the following C function for reading files: bool files_read(const char *path, char **data, size_t *size) { FILE *f = fopen(path, "rb"); if (!f) { return false; } fseek(f, 0, SEEK_END); size_t n = (size_t)ftell(f); rewind(f); if (size) { *size = n; } if (!data) { goto ok; } *data = (char*)malloc(sizeof(char)*(n + 1)); if (fread(*data, 1, n, f) != n) { free(*data); return false; } *data[n] = '\0'; ok: fclose(f); return true; } ChatGPT solves this task instantly. Many, even experienced C developers, would have trouble finding the bug. And the lack of error checking on malloc is not a bug.

golergka 3y ago
Just for reference, copied this snippet in GPT-4, it generated this answer in 1,5 seconds: > The issue in this code is related to memory access. The line data[n] = '\0'; should be (data)[n] = '\0';. And then went on to generate corrected code and this explanation: > The original expression, data[n], first applies the array subscript operation, which is equivalent to (data + n), and then tries to dereference the resulting pointer, which is not the intention here. This will likely lead to a segmentation fault because data is a pointer to pointer, and data[n] will try to access memory that has not been allocated. > The corrected expression, (*data)[n], first dereferences data to get the allocated char array, and then applies the array subscript operation to set the null terminator at the correct location.
- ChatGTP 3y ago
  Just wondering if you think this type of bug hasn't been covered at least 50x in many online forums, I'd be very surprised? I'm not trying to say this isn't impressive, but seems fairly obvious and more like something a linter / analyzer would pickup ?
  - almog 3y ago
    Even if this isn't based on similar questions that can be found online, a bug in existing systems usually wouldn't introduce itself by framing an isolated section of code. More often than not it'd be through indirection or a side effect that covers a larger execution path.

pornel 3y ago

Rust is 90% arguing with the compiler and 10% debugging. And yesterday GPT-4 solved a WTF lifetime issue for me pulling a trick I wouldn't have thought about.

Ygg2 3y ago
Probably because its corpus of knowledge includes someone solving it like that.
- pixl97 3y ago
  I'm not really a programmer, but when I write code for stuff I need this is how I operate. The novel idea I want to solve is the end goal, not the intermediate code that I'm gluing together that other people have generally created.
ChatGTP 3y ago
What was the WTF lifetime issue?
- pornel 3y ago
  I wanted to compare function pointers, but that's tricky due to unique function types, disallowed non-primitive casts, plus apparently not allowed for types with higher-ranked lifetimes (for<'a> fn(&'a)). GPT-4 came up with using a type alias (type FnComparable<'a>) that added necessary coercions and gave them equal lifetime.

Ygg2 3y ago

Honestly ChatGPT-4 sucks at writing code (if similar code isn't in its corpus of knowledge). I asked it to write a YAML parser in pure Rust. The first thing it did was write this: pub enum YamlNode { Scalar(String), Sequence(Vec<YamlNode>), Map(HashMap<String, YamlNode>), } This is what I expect a freshman in college to write as their first Yaml parser. Literally unusable as a starting point. I did try to correct it but it tripped up and somehow managed to fuck that up to.

hderms 3y ago
Why does it suck, because it's doing a lot of allocations?
- Ygg2 3y ago
  No. You have a fair point, but not my biggest gripe. It's much more fundamental to what YAML is. YAML allows YamlNodes as keys in a map (aka complex keys). E.g. ? [ a, b] : "complex key, scalar value" You asked it for YAML parser and it gave you a shitty JSON parser. Also this is the most banal thing about YAML, something that should be obvious just browsing the examples. And I gave it test suite and link to YAML spec. Most work in YAML is around super obscure corner cases.

api 3y ago

Stronger type systems really do prevent bugs. In Rust I probably spend 20% of the time at most debugging. Of course you pay for that by the fact that it takes longer up front to write Rust because you have to think about it more, but overall I think it’s a net win. You also get the bonus of nasty bugs like crash bugs and exploitable memory problems become almost entirely a thing of the past. You can ship new software you know is very reliable.

bravura 3y ago

I’ve said it before and I’ll say it again. A time traveling retroactive debugger integrated with an LLM for interrogating what happened and why, would be awesome.

crabmusket 3y ago

This a classic case of understanding programming as an activity that is primarily about text production. This is what Naur was arguing against in Programming As Theory Building [1]. Taking Nair's viewpoint, coding AI will be useful only to the extent that it assists programmers to build and employ their theory of the program. Or to itself develop a useful theory of a specific program, not just programs in general. [1] https://gist.github.com/onlurking/fc5c81d18cfce9ff81bc968a7f342fb1 https://gist.github.com/onlurking/fc5c81d18cfce9ff81bc968a7f...

AnimalMuppet 3y ago

I think that training GPT to use a debugger is going to be hard. For many debug problems, the amount of context you have to keep is going to be really hard for GPT. > But, AI is particularly bad at formal methods. AI? Or GPT? Earlier, reasoning-based attempts at AI seem like they would be a natural fit for formal methods.