3 ms·

> Software engineers should be critical of the outputs of large language models, as they tend to hallucinate and produce inaccurate or incorrect code. This see

> Software engineers should be critical of the outputs of large language models, as they tend to hallucinate and produce inaccurate or incorrect code. This seems to be the case more than not for certain tasks, anything assembler or C I have ever asked it has turned out to be at least somewhat wrong. Mixing styles and syntax all over the place. I am not afraid that some generative AI will take my job anytime soon.

varispeed 3y ago
These things are a matter of writing a correct prompt. Like "use this and that naming convention for variables and keep this and that style". You can also ask it to write tests for the code so you can verify it is working or add your own additional tests. Even if the produced code is wrong, it usually takes a few steps to correct it and still saves time. It is an amazing tool and time saver if you know what you are doing, but helps with research as well. For instance if you want to code something in the domain you know little about, it can give you ideas where to look and then improve your prompts based on that. In the context of taking anyone's job is like saying that a spreadsheet is going to replace accountants. It's just a tool.
- weevil 3y ago
  My first step when using an LLM is asking it to produce a test suite for a function, with a load of example inputs and outputs. 9 times out of 10, I've been presented with something incorrect which I need to correct first. I'm reasonably good at being specific and clear in my directions, but I quickly arrived at the conclusion that LLMs are simply not good at producing accurate code in a way that saves me time.
- DonaldPShimoda 3y ago
  > These things are a matter of writing a correct prompt. No, they aren't. ChatGPT doesn't know things. It's just a very fancy predictive text engine. For any given prompt, it will provide a response that is engineered to sound authoritative, regardless of whether any information is correct. It will summon case law out of the aether when prompted by a lawyer; it will conjure paper titles and author names from thin air when prompted by a researcher; it will certainly generate semantically meaningless code very often. It's absolutely ludicrous to assert that you just need a "better prompt" to counteract these kinds of responses because this is not a bug — it's literally just how it works.
  - Kiro 3y ago
    Read the next sentence after your quote. The point is that you should include code and examples in your prompt (Copilot is so good since it includes the surrounding code and open files in the prompt to understand your specific context), not that you should craft an exceptional "act as rockstar engineer" prompt.
    - DonaldPShimoda 3y ago
      I did read it, but the whole premise is flawed due to an apparently incomplete understanding of how LLMs work. Including code samples in your prompt won't have the effect you think it will. LLMs are trained to produce results that are statistically likely to be syntactically well-formed according to assumptions made about how "language" works. So when you provide code samples, the model incorporates those into the response. But it doesn't have any actually comprehension of what's going on in those code samples, or what any code "means"; it's all just pushing syntax around. So what happens is you end up with responses that are more likely to look like what you want, but there's no guarantee or even necessarily a correlation that the tuned responses will actually produce meaningfully good code. This increases the odds of a bug slipping by because, at a glance, it looked correct. Until LLMs can generate code with proofs of semantic meaning, I don't think it's a good idea to trust them. You're welcome to do as you please, of course, but I would never use them for anything I work on.
      - Kiro 3y ago
        If it works it works and it definitely works for me. I've been using Copilot for about a year and I can't imagine coding without it again. I cannot recall any bugs slipping by because of it. If anything it makes me write less bugs, since it has no problem taking tedious edge cases into account.
        strus 3y ago
        > I've been using Copilot for about a year and I can't imagine coding without it again I for example used Copilot for 2 months at work and wouldn't pay for it. Most suggestions where either useless or buggy. But I work in a huge C++ codebase, maybe that's hard for it as C++ is also hard for ChatGPT.
- omgmajk 3y ago
  The question is, do you actually save time by coaxing the language model into an answer or would you just save time by writing it yourself? I have this friend who gets obsessed with things very easily and ChatGPT got to him quite a bit. He spent about two months perfecting his AI persona and starts every chat with several hundred words of directions before asking any questions. I find that this also produces the wrong answers many times.
  - throwuwu 3y ago
    For me it saves time by keeping my momentum up. I don’t lean on it when I’m in a flow state and cruising through the code I’m writing but as soon as I hit a wall I jump over to chat and start working on a solution with it. This saves a huge amount of time that would otherwise be spent banging my head against the keyboard or googling or reading random SO posts and dev blogs and documentation or even just the time wasted when I get frustrated enough to stop working on the problem and wind up browsing hn.
intelVISA 3y ago
Thankfully, as a human I never produce inaccurate or incorrect code.
- tiffanyg 3y ago
  Brings to mind the excellent team behind REAPER - "cockos": John Schwartz, in the role of "Italics" is reportedly "so highly trained that he can type code that compiles correctly almost 3% of the time"! Remarkable! https://www.cockos.com/team.php https://www.cockos.com/team.php (I think just about anyone 'serious' learns pretty quickly that compiler errors are in the 'our Lord and Saviour' category. Unusually distributed generally quite rare but easily catastrophic runtime 'Heisenbugs' are the fruit of the devil!)
sigstoat 3y ago
> anything assembler or C I have ever asked it has turned out to be at least somewhat wrong i've found that its quality is proportional to the amount of questions about the subject on the internet. if i ask it for help with popular javascript frameworks, it vastly improves my productivity (i'm not a frontend person). it still coughs up wrong stuff half the time, but even then it can cut through the constant churn of the frameworks' terminology and give me enough hint to find what i need in the docs quickly. if i ask it about, say, specific details of the STM32 HAL, it knows just enough to come up with something that i'll waste my time reading.
- omgmajk 3y ago
  Funny you should say that because it was STM32 that I was working with when ChatGPT blew up.

varispeed 3y ago

These things are a matter of writing a correct prompt. Like "use this and that naming convention for variables and keep this and that style". You can also ask it to write tests for the code so you can verify it is working or add your own additional tests. Even if the produced code is wrong, it usually takes a few steps to correct it and still saves time. It is an amazing tool and time saver if you know what you are doing, but helps with research as well. For instance if you want to code something in the domain you know little about, it can give you ideas where to look and then improve your prompts based on that. In the context of taking anyone's job is like saying that a spreadsheet is going to replace accountants. It's just a tool.

weevil 3y ago
My first step when using an LLM is asking it to produce a test suite for a function, with a load of example inputs and outputs. 9 times out of 10, I've been presented with something incorrect which I need to correct first. I'm reasonably good at being specific and clear in my directions, but I quickly arrived at the conclusion that LLMs are simply not good at producing accurate code in a way that saves me time.
DonaldPShimoda 3y ago
> These things are a matter of writing a correct prompt. No, they aren't. ChatGPT doesn't know things. It's just a very fancy predictive text engine. For any given prompt, it will provide a response that is engineered to sound authoritative, regardless of whether any information is correct. It will summon case law out of the aether when prompted by a lawyer; it will conjure paper titles and author names from thin air when prompted by a researcher; it will certainly generate semantically meaningless code very often. It's absolutely ludicrous to assert that you just need a "better prompt" to counteract these kinds of responses because this is not a bug — it's literally just how it works.
- Kiro 3y ago
  Read the next sentence after your quote. The point is that you should include code and examples in your prompt (Copilot is so good since it includes the surrounding code and open files in the prompt to understand your specific context), not that you should craft an exceptional "act as rockstar engineer" prompt.
  - DonaldPShimoda 3y ago
    I did read it, but the whole premise is flawed due to an apparently incomplete understanding of how LLMs work. Including code samples in your prompt won't have the effect you think it will. LLMs are trained to produce results that are statistically likely to be syntactically well-formed according to assumptions made about how "language" works. So when you provide code samples, the model incorporates those into the response. But it doesn't have any actually comprehension of what's going on in those code samples, or what any code "means"; it's all just pushing syntax around. So what happens is you end up with responses that are more likely to look like what you want, but there's no guarantee or even necessarily a correlation that the tuned responses will actually produce meaningfully good code. This increases the odds of a bug slipping by because, at a glance, it looked correct. Until LLMs can generate code with proofs of semantic meaning, I don't think it's a good idea to trust them. You're welcome to do as you please, of course, but I would never use them for anything I work on.
    - Kiro 3y ago
      If it works it works and it definitely works for me. I've been using Copilot for about a year and I can't imagine coding without it again. I cannot recall any bugs slipping by because of it. If anything it makes me write less bugs, since it has no problem taking tedious edge cases into account.
      - strus 3y ago
        > I've been using Copilot for about a year and I can't imagine coding without it again I for example used Copilot for 2 months at work and wouldn't pay for it. Most suggestions where either useless or buggy. But I work in a huge C++ codebase, maybe that's hard for it as C++ is also hard for ChatGPT.
omgmajk 3y ago
The question is, do you actually save time by coaxing the language model into an answer or would you just save time by writing it yourself? I have this friend who gets obsessed with things very easily and ChatGPT got to him quite a bit. He spent about two months perfecting his AI persona and starts every chat with several hundred words of directions before asking any questions. I find that this also produces the wrong answers many times.
- throwuwu 3y ago
  For me it saves time by keeping my momentum up. I don’t lean on it when I’m in a flow state and cruising through the code I’m writing but as soon as I hit a wall I jump over to chat and start working on a solution with it. This saves a huge amount of time that would otherwise be spent banging my head against the keyboard or googling or reading random SO posts and dev blogs and documentation or even just the time wasted when I get frustrated enough to stop working on the problem and wind up browsing hn.

intelVISA 3y ago

Thankfully, as a human I never produce inaccurate or incorrect code.

tiffanyg 3y ago
Brings to mind the excellent team behind REAPER - "cockos": John Schwartz, in the role of "Italics" is reportedly "so highly trained that he can type code that compiles correctly almost 3% of the time"! Remarkable! https://www.cockos.com/team.php https://www.cockos.com/team.php (I think just about anyone 'serious' learns pretty quickly that compiler errors are in the 'our Lord and Saviour' category. Unusually distributed generally quite rare but easily catastrophic runtime 'Heisenbugs' are the fruit of the devil!)

sigstoat 3y ago

> anything assembler or C I have ever asked it has turned out to be at least somewhat wrong i've found that its quality is proportional to the amount of questions about the subject on the internet. if i ask it for help with popular javascript frameworks, it vastly improves my productivity (i'm not a frontend person). it still coughs up wrong stuff half the time, but even then it can cut through the constant churn of the frameworks' terminology and give me enough hint to find what i need in the docs quickly. if i ask it about, say, specific details of the STM32 HAL, it knows just enough to come up with something that i'll waste my time reading.

omgmajk 3y ago
Funny you should say that because it was STM32 that I was working with when ChatGPT blew up.