7 ms·

Also if there are fewer humans involved in the code production there is a lot of room for producing code that "works", but is not cohesive or maintainable. Inva

Also if there are fewer humans involved in the code production there is a lot of room for producing code that "works", but is not cohesive or maintainable. Invariably there will be a point at which something is broken and someone will need to wade through the mess to find why it's broken and try to fix it.

bluefirebrand 3y ago
Nah, you just throw it out and have the AI generate an all new one with different problems!
- Jensson 3y ago
  I really look forward to all programs now having new strange bugs every release. They already do, but I expect AI to do that more at first.
  - falcor84 3y ago
    I don't know if AIs will ever get really good at QA in general, but I do think that AIs can get quite good quickly at regression testing.
  - ipaddr 3y ago
    The hacking opportunities will be endless. Feeding AI the exploit will be new.
- CuriouslyC 3y ago
  That's how bads use GPT to code. The right way is to ask GPT to break the problem down into a bunch of small strongly typed helper functions with unit tests, then ask it to compose the solution from those helper functions, also with integration tests. If tests fail at any point you can just feed the failure output along with the test and helper function code back in and it will almost always get it right for reasonably non-trivial things by the second try. It can also be good to provide some example helper functions/tests to give it style guidelines.
  - bluefirebrand 3y ago
    If you're already doing all of this work then it's trivial to actually type all the stuff in yourself Is GPT actually saving you any time if it can't actually do the hard part?
    - CuriouslyC 3y ago
      It's not really "all this work," once you have good prompts you can use them to crank out a lot of code very quickly. You can use it to crank out thousands of lines of code a day that are somewhat formulaic, but not so formulaic that a simple rules based system could do it. For example, I took text document with headers for table names and unordered lists for table columns, and had it produce a database schema which only required minor tuning, which I then used to generate sqlmodel classes and typescript types. Then I created an example component for one entity and it created similar components for the others in the schema. LLMS are exceptionally good at this sort of domain transformation, a decent engineer could easily crank out 2-5k lines/day if they were mostly doing this sort of work.
      - c-cube 3y ago
        Now your description of "good prompts" to reuse has created an abomination in my mind. I blame you. The abomination: prompts being reused by way of yaml templating, a Helm chart of sorts but for LLM prompts. The delicious combination of yaml programming and prompt engineering. I hope it never exists.
    - croo 3y ago
      You know with GPT you can do these steps in a language you are not familiar with and it will still work. If you don't know some aspect of the language or it's environmental specifics you can just chat until you find out enough to continue.
  - thetacitman 3y ago
    How do I know if a problem needs to be broken down by GPT, and how do I know if it broke the problem down correctly? What if GPT is broken or has a billing error, how do I break down the problem then?
AlwaysRock 3y ago
This is the future imagined by A Fire Upon the Deep and its sequel. While less focused on the code being generated by ai, it features seemingly endless amounts of code and programs that can do almost anything but the difficulty is finding the program that works for you and is safe to use. To some extent... This is already the world we live in. A lot of code is unreadable without a lot of effort or expertise. If all code was open sourced there would almost certainly be code written to do just about anything you'd like it to. The difficulty would be finding that code and customizing it for your use.
- Normalcy294812 3y ago
  Thanks for the Book Title. It looks like an interesting read.
  - scubbo 3y ago
    Caution - lots of people like to talk about this "code archeology" idea as if it's a central driving point of the book, whereas in fact it's mentioned once in passing in the prologue and is never again relevant to the story. Don't get me wrong, it's still a decent book on its own merits - but don't go into it expecting that to be the main point of the book (I did, and disappointed as a result).
    - JackMorgan 3y ago
      As a counterpoint, the main nemesis of the book comes from software that is found in archaeological expedition. While software archeology doesn't show up after the first chapter, the ramifications of what happens in that world due to so much software is pretty central.
      - scubbo 3y ago
        This is certainly true, and doesn't detract from my disappointment not to have actually seen the software archeology in practice.
    - 3y ago
broast 3y ago
Certainly many of us here already have a good amount of experience debugging giant legacy spaghetti code-bases written by people you can't talk to, or people who can't debug their own code. That job may not change much.
- billforsternz 3y ago
  I remember one such occasion back in a previous tech boom (late 90s) and it turned out the reason I couldn't talk to the guy who wrote this particular pile of Italian nutrition was that the Feds had shown up one day and taken him to jail (something to do with pump and dump market manipulation via a faked analyst report [edit: actually a faked press release I now remember. "SmallCapCorp (NASDAQ: SCC$) announces they have received a record breaking order for their next gen product / aquisition offer / something like that from RandomIsraeliCompanyThatMightNotEvenHaveExisted"]). A lot of software engineers would spend a portion of their day tracking their volatile stock / options etc. in those years.
mbil 3y ago
Maybe I'm being overly optimistic but in a future where a model can digest hundreds of thousands of lines of code, write unit tests, and do refactors, will this even be a problem?

bluefirebrand 3y ago

Nah, you just throw it out and have the AI generate an all new one with different problems!

Jensson 3y ago
I really look forward to all programs now having new strange bugs every release. They already do, but I expect AI to do that more at first.
- falcor84 3y ago
  I don't know if AIs will ever get really good at QA in general, but I do think that AIs can get quite good quickly at regression testing.
- ipaddr 3y ago
  The hacking opportunities will be endless. Feeding AI the exploit will be new.
CuriouslyC 3y ago
That's how bads use GPT to code. The right way is to ask GPT to break the problem down into a bunch of small strongly typed helper functions with unit tests, then ask it to compose the solution from those helper functions, also with integration tests. If tests fail at any point you can just feed the failure output along with the test and helper function code back in and it will almost always get it right for reasonably non-trivial things by the second try. It can also be good to provide some example helper functions/tests to give it style guidelines.
- bluefirebrand 3y ago
  If you're already doing all of this work then it's trivial to actually type all the stuff in yourself Is GPT actually saving you any time if it can't actually do the hard part?
  - CuriouslyC 3y ago
    It's not really "all this work," once you have good prompts you can use them to crank out a lot of code very quickly. You can use it to crank out thousands of lines of code a day that are somewhat formulaic, but not so formulaic that a simple rules based system could do it. For example, I took text document with headers for table names and unordered lists for table columns, and had it produce a database schema which only required minor tuning, which I then used to generate sqlmodel classes and typescript types. Then I created an example component for one entity and it created similar components for the others in the schema. LLMS are exceptionally good at this sort of domain transformation, a decent engineer could easily crank out 2-5k lines/day if they were mostly doing this sort of work.
    - c-cube 3y ago
      Now your description of "good prompts" to reuse has created an abomination in my mind. I blame you. The abomination: prompts being reused by way of yaml templating, a Helm chart of sorts but for LLM prompts. The delicious combination of yaml programming and prompt engineering. I hope it never exists.
  - croo 3y ago
    You know with GPT you can do these steps in a language you are not familiar with and it will still work. If you don't know some aspect of the language or it's environmental specifics you can just chat until you find out enough to continue.
- thetacitman 3y ago
  How do I know if a problem needs to be broken down by GPT, and how do I know if it broke the problem down correctly? What if GPT is broken or has a billing error, how do I break down the problem then?

AlwaysRock 3y ago

This is the future imagined by A Fire Upon the Deep and its sequel. While less focused on the code being generated by ai, it features seemingly endless amounts of code and programs that can do almost anything but the difficulty is finding the program that works for you and is safe to use. To some extent... This is already the world we live in. A lot of code is unreadable without a lot of effort or expertise. If all code was open sourced there would almost certainly be code written to do just about anything you'd like it to. The difficulty would be finding that code and customizing it for your use.

Normalcy294812 3y ago
Thanks for the Book Title. It looks like an interesting read.
- scubbo 3y ago
  Caution - lots of people like to talk about this "code archeology" idea as if it's a central driving point of the book, whereas in fact it's mentioned once in passing in the prologue and is never again relevant to the story. Don't get me wrong, it's still a decent book on its own merits - but don't go into it expecting that to be the main point of the book (I did, and disappointed as a result).
  - JackMorgan 3y ago
    As a counterpoint, the main nemesis of the book comes from software that is found in archaeological expedition. While software archeology doesn't show up after the first chapter, the ramifications of what happens in that world due to so much software is pretty central.
    - scubbo 3y ago
      This is certainly true, and doesn't detract from my disappointment not to have actually seen the software archeology in practice.
  - 3y ago

broast 3y ago

Certainly many of us here already have a good amount of experience debugging giant legacy spaghetti code-bases written by people you can't talk to, or people who can't debug their own code. That job may not change much.

billforsternz 3y ago
I remember one such occasion back in a previous tech boom (late 90s) and it turned out the reason I couldn't talk to the guy who wrote this particular pile of Italian nutrition was that the Feds had shown up one day and taken him to jail (something to do with pump and dump market manipulation via a faked analyst report [edit: actually a faked press release I now remember. "SmallCapCorp (NASDAQ: SCC$) announces they have received a record breaking order for their next gen product / aquisition offer / something like that from RandomIsraeliCompanyThatMightNotEvenHaveExisted"]). A lot of software engineers would spend a portion of their day tracking their volatile stock / options etc. in those years.

mbil 3y ago

Maybe I'm being overly optimistic but in a future where a model can digest hundreds of thousands of lines of code, write unit tests, and do refactors, will this even be a problem?