4 ms·

... What. It hallucinates and it doesn't compile, fine. It hallucinates and flips a 1 with a -1; oops that's a lot of lost revenue. But it compiled, right? It

... What. It hallucinates and it doesn't compile, fine. It hallucinates and flips a 1 with a -1; oops that's a lot of lost revenue. But it compiled, right? It hallucinates, and in 4% of cases rejects a home loan when it shouldn't because of a convoluted set of nested conditions, only there is no one on staff that can explain the logic of why something is laid out the way it is and I mean, it works 96% of the time so don't rock the boat. Oops, we just oppressed a minority group or everyone named Dave because you were lazy.

mrtksn 3y ago
As I said, you are still responsible for the quality control. You are supposed to notice that everyone is named Dave and tell ChatGPT to fix it. Write tests, read code, run & observe for odd behaviours. It's not an autonomous agent just yet.
- somewhereoutth 3y ago
  As we all know, it is much easier to read and verify code you've written yourself - perhaps it is only code you've written yourself that can be properly read and verified. As ever, tests can be of only limited utility (separate discussion).
  - mrtksn 3y ago
    It's easier to read the code you recently wrote, sure. But in real life people use and debug other people's code all the time, LLM generated code is just like that. Also, if you make it generate the code in small enough blocks you also end up knowing the codebase is if you wrote it.
- gumballindie 3y ago
  But why should i waste time using a broken product when i can do it properly myself? To me a lot of this debate sounds like people obsessively promoting a product for some odd reason, as if they were the happy owners of a hammer in search of a nail.
  - mrtksn 3y ago
    If you are faster and more productive that way, do it that way. Most people are not geniuses and polymaths, it's much easier and cheaper for me to design the architecture and ask ChatGPT to generate the code in many different languages(Swift/HTML/JS/CSS on the client side and Py, JS, PHP on the server side). It's easier because although I'm proficient an all these it's very hard for me to switch solving client specific JS problems to server specific JS problems or between graphics and animation related problems and data processing problems with Swift. It's also cheaper because I don't have to pay someone to do it for me. In my case, I know all that well enough to spot a problem and debug, I just don't want to go through the trouble of actually writing it.
    - gumballindie 3y ago
      The debate here is wether openai's product, chatgpt, can indeed deliver what it claims - coding, saving dogs' lives, mental health counceling, and so on. It would appear that it doesn't but it does mislead people without experience in whatever field they use it. For instance if i ask it about law i am impressed, but when I ask it about coding of software engineering it blatantly fails. The conclusion being that as a procedural text generator it is impressive - it nails language - but the value of the output is far from settled. This debate is important because as technical people it is our reposnbility to inform non technical people about the use of this technology and to bring awareness about potential misleading claims its seller makes - as it was the case with crypto currencies, and many other technologies that promised the world delivered nothing of real benefit (but made people rich in the process by exploting the uniformed).
      - mrtksn 3y ago
        That's not the debate, it's the first time I'm hearing about that in this thread.
  - johnfn 3y ago
    It's funny you say that. Reading over a lot of the comments here sound like a lot of people obsessively dismissing a swiss army knife because it doesn't have their random favorite tool of choice.
- throwaway346434 3y ago
  You've missed the subtle point here. Imagine you walk in 4 years down the track and try to examine AI generated logic committed under a dev's credentials. It's written in an odd, but certain way. There is no documentation. The original dev is MIA. You know there is something you defective from the helpdesk tickets coming through, but it's also a complex area. You want to go through a process of writing tests, refactoring, understanding, but to redeploy this is hard work. You talk to your manager. It's not everyone. Neither of you realize its only people named Dave/minority attribute X affected, because why would that matter? You need 40 hours of budget to begin to maybe fix this. Institutionally, this is not supportable because it's "only affecting 4% of users and that's not many". Close ticket, move on. Only it's everyone named Dave. 100% of the people born to this earth with parents who named them Dave are, for absolutely no discernable reason, denied and oppressed.
wokwokwok 3y ago
The output of an LLM is a distribution, and yes, if you’re just taking the first answer, that’s problematic. However, it is a distribution, and than means the majority of solutions are not weird edge cases, they’re valid solutions. Your job as a user is to generate multiple solutions and then review them and pick the one you like the most, and maybe modify it to work correctly if it has weird edge cases. How do you do that? Well, you can start by following a structured process where you define success criteria as a validator (eg. Tests, compiler, parser, linters) and fitness criteria as a scorer (code metrics like complexity, runtime, memory use, etc)… then: 1) define goal 2) generate multiple solution candidates 3) filter candidates by validator (does it compile? Pass tests? Etc) 4) score the solutions (is it pure? Is it efficient? Etc) 5) pick the best solution 6) manually review and tweak the solution This structured and disciplined approach to software engineering works. Many of the steps (eg. 3, 4, 5) can be automated. It generates meaningful quality code results. You can use it with or without AI… You don’t have to follow this approach, but my point is that you can; there is nothing fundamentally intractable able using a language model to generate code. The problem that you’re critiquing is the trivial and naive approach of just hitting “generate” and blindly copying that into your code base. …that’s stupid and dangerous, but it’s also a straw man. Seriously; people writing code with these models aren’t doing that; when you read blogs and posts from people, eg. Building seriously using copilot you’ll see this pattern emerge repeatedly: Generate multiple solutions. Tweak your prompt. Ask for small pure dependency free code blocks. Review the and test output. It’s not a dystopian AI future, it’s just another tool.
- emporas 3y ago
  In general, one should not instruct GPT to solve a problem. The instructions should be about generating code, after a human thought process took place, and then generate even more code, then even more, and after merging all the code together the problem is solved. The particulars are roughly what you describe, in how to achieve that.

mrtksn 3y ago

As I said, you are still responsible for the quality control. You are supposed to notice that everyone is named Dave and tell ChatGPT to fix it. Write tests, read code, run & observe for odd behaviours. It's not an autonomous agent just yet.

somewhereoutth 3y ago
As we all know, it is much easier to read and verify code you've written yourself - perhaps it is only code you've written yourself that can be properly read and verified. As ever, tests can be of only limited utility (separate discussion).
- mrtksn 3y ago
  It's easier to read the code you recently wrote, sure. But in real life people use and debug other people's code all the time, LLM generated code is just like that. Also, if you make it generate the code in small enough blocks you also end up knowing the codebase is if you wrote it.
gumballindie 3y ago
But why should i waste time using a broken product when i can do it properly myself? To me a lot of this debate sounds like people obsessively promoting a product for some odd reason, as if they were the happy owners of a hammer in search of a nail.
- mrtksn 3y ago
  If you are faster and more productive that way, do it that way. Most people are not geniuses and polymaths, it's much easier and cheaper for me to design the architecture and ask ChatGPT to generate the code in many different languages(Swift/HTML/JS/CSS on the client side and Py, JS, PHP on the server side). It's easier because although I'm proficient an all these it's very hard for me to switch solving client specific JS problems to server specific JS problems or between graphics and animation related problems and data processing problems with Swift. It's also cheaper because I don't have to pay someone to do it for me. In my case, I know all that well enough to spot a problem and debug, I just don't want to go through the trouble of actually writing it.
  - gumballindie 3y ago
    The debate here is wether openai's product, chatgpt, can indeed deliver what it claims - coding, saving dogs' lives, mental health counceling, and so on. It would appear that it doesn't but it does mislead people without experience in whatever field they use it. For instance if i ask it about law i am impressed, but when I ask it about coding of software engineering it blatantly fails. The conclusion being that as a procedural text generator it is impressive - it nails language - but the value of the output is far from settled. This debate is important because as technical people it is our reposnbility to inform non technical people about the use of this technology and to bring awareness about potential misleading claims its seller makes - as it was the case with crypto currencies, and many other technologies that promised the world delivered nothing of real benefit (but made people rich in the process by exploting the uniformed).
    - mrtksn 3y ago
      That's not the debate, it's the first time I'm hearing about that in this thread.
- johnfn 3y ago
  It's funny you say that. Reading over a lot of the comments here sound like a lot of people obsessively dismissing a swiss army knife because it doesn't have their random favorite tool of choice.
throwaway346434 3y ago
You've missed the subtle point here. Imagine you walk in 4 years down the track and try to examine AI generated logic committed under a dev's credentials. It's written in an odd, but certain way. There is no documentation. The original dev is MIA. You know there is something you defective from the helpdesk tickets coming through, but it's also a complex area. You want to go through a process of writing tests, refactoring, understanding, but to redeploy this is hard work. You talk to your manager. It's not everyone. Neither of you realize its only people named Dave/minority attribute X affected, because why would that matter? You need 40 hours of budget to begin to maybe fix this. Institutionally, this is not supportable because it's "only affecting 4% of users and that's not many". Close ticket, move on. Only it's everyone named Dave. 100% of the people born to this earth with parents who named them Dave are, for absolutely no discernable reason, denied and oppressed.

wokwokwok 3y ago

The output of an LLM is a distribution, and yes, if you’re just taking the first answer, that’s problematic. However, it is a distribution, and than means the majority of solutions are not weird edge cases, they’re valid solutions. Your job as a user is to generate multiple solutions and then review them and pick the one you like the most, and maybe modify it to work correctly if it has weird edge cases. How do you do that? Well, you can start by following a structured process where you define success criteria as a validator (eg. Tests, compiler, parser, linters) and fitness criteria as a scorer (code metrics like complexity, runtime, memory use, etc)… then: 1) define goal 2) generate multiple solution candidates 3) filter candidates by validator (does it compile? Pass tests? Etc) 4) score the solutions (is it pure? Is it efficient? Etc) 5) pick the best solution 6) manually review and tweak the solution This structured and disciplined approach to software engineering works. Many of the steps (eg. 3, 4, 5) can be automated. It generates meaningful quality code results. You can use it with or without AI… You don’t have to follow this approach, but my point is that you can; there is nothing fundamentally intractable able using a language model to generate code. The problem that you’re critiquing is the trivial and naive approach of just hitting “generate” and blindly copying that into your code base. …that’s stupid and dangerous, but it’s also a straw man. Seriously; people writing code with these models aren’t doing that; when you read blogs and posts from people, eg. Building seriously using copilot you’ll see this pattern emerge repeatedly: Generate multiple solutions. Tweak your prompt. Ask for small pure dependency free code blocks. Review the and test output. It’s not a dystopian AI future, it’s just another tool.

emporas 3y ago
In general, one should not instruct GPT to solve a problem. The instructions should be about generating code, after a human thought process took place, and then generate even more code, then even more, and after merging all the code together the problem is solved. The particulars are roughly what you describe, in how to achieve that.