3 ms·

It returns factually incorrect data, and it returns code with subtle but important errors if you ask it anything that's not regurgitated a thousand times in the

by zekica 4y ago

It returns factually incorrect data, and it returns code with subtle but important errors if you ask it anything that's not regurgitated a thousand times in the training dataset. Don't get me wrong, it has emergent properties (more than you would expect from a fancy autocomplete), but factual output was never GPT-4's nor any other LLM's design goal.

progrus 4y ago
I’ve started using it as a sort of “shotgun approach” - characterize what you want without trying too hard, let it toss out a jumble of interesting snippets that are often very good leads, and then take over refactoring from there. I do agree that others who don’t understand what it’s doing may not even review the code output, and I predict that this will cause at least one death in the next ten years.
- musha68k 4y ago
  *weeks/*months
  - progrus 4y ago
    Facts, sadly.
- bodge5000 4y ago
  I suppose it depends what specifically you're getting it to do, but I've found in my testing the shotgun approach to be more work than just writing it out myself. Honestly I've found its biggest value in learning a new language, it outputs with so many errors that it forces me to get in deep with debugging it. I haven't tried gpt-4 yet though, maybe its a vast improvement, I'm just not using it enough to justify the costs
  - progrus 4y ago
    It really depends on how comfortable you are with the language you’re using, and how recently you switched contexts. For anything unfamiliar, it’s pretty impressive how much it can help. In fact, I bet if you did an experiment comparing people reading a textbook about a somewhat-familiar language, vs. just asking GPT to do things and noticing the bugs, the latter group would reach proficiency faster.
- squeaky-clean 4y ago
  It's just way easier for me to edit some existing code than it is for me to write it from scratch. Especially if it's incorrect (a la XKCD 386). For stuff where I'm not sure where to start it's been very helpful. Even though I often keep very little of the GPT generated code by the time I'm done.
thorax 4y ago
> It returns factually incorrect data, and it returns code with subtle but important errors if you ask it anything that's not regurgitated a thousand times in the training dataset. To be fair, that's what pretty much every person does. The bar does seem pretty high if we need more than that (especially if not specifically trained on a topic). It's not a universally perfect expert servant, but I've been exploring the code generation of GPT4 in detail (i.e. via the 'cataclysm' module I just posted about). In 1 minute it can write functions as good as the average developer intern most of the time. We're keeping score in a weird way if we're responding quickly with it needing to "code without subtle but important errors". Because that's the majority of human developers, too. I've been writing code for 30 years, and if you put a gun to my head, I would still have subtle but important flaws in every first typing of any complex generated code. I'm not saying you're bashing it, by the way, I get your point, but I do worry a bit when the first response is citing that the SOTA models get things wrong in 0-shot situations without full context. That's describing all of us.
- corbulo 4y ago
  The standard should be higher for machines. Historically, they're inorganic tools. If a hammer was only as dense as bone, would it be a good hammer?
  - stale2002 4y ago
    > would it be a good hammer? Yes, it is possible that such a hammer would still be very useful, if that was the only available option. Imperfect hammers can still be very very useful, when compared to having no hammers.
- taberiand 4y ago
  I always say, if some nontrivial code compiles and runs on the first attempt, then you just haven't found the bugs yet. GPT-4 is a fantastic collaboration tool for senior developers, who know what they want in detail and can review, verify and apply the output it generates. Just yesterday I needed to write some detailed bash scripts. I'm no Linux guru but I know what I want and that was enough - in minutes I had a solid script that did everything I needed and wanted, something that would have taken much longer to hunt down through Google. And then I asked it about SQL, C#, AWS, Terraform, Rust and on and on and everything was high quality. The only way I could have gained similar results without ChatGPT would have been to post all my questions to the dev slack channel and engage in hours long discussions with my colleagues.
yodsanklai 4y ago
> factual output was never GPT-4's nor any other LLM's design goal. Tell that to 50% of HN who think it's going to replace Google. I think they should be a red disclaimer that answers are often incorrect and shouldn't be trusted in any way. That being said, I think it's a useful tool for all queries that don't expect a factually correct answer, or can be easily checked. For instance, summary, rewriting, simple code synthesis, and possibly many other niche applications.

by zekica 4y ago

progrus 4y ago

I’ve started using it as a sort of “shotgun approach” - characterize what you want without trying too hard, let it toss out a jumble of interesting snippets that are often very good leads, and then take over refactoring from there. I do agree that others who don’t understand what it’s doing may not even review the code output, and I predict that this will cause at least one death in the next ten years.

musha68k 4y ago
*weeks/*months
- progrus 4y ago
  Facts, sadly.
bodge5000 4y ago
I suppose it depends what specifically you're getting it to do, but I've found in my testing the shotgun approach to be more work than just writing it out myself. Honestly I've found its biggest value in learning a new language, it outputs with so many errors that it forces me to get in deep with debugging it. I haven't tried gpt-4 yet though, maybe its a vast improvement, I'm just not using it enough to justify the costs
- progrus 4y ago
  It really depends on how comfortable you are with the language you’re using, and how recently you switched contexts. For anything unfamiliar, it’s pretty impressive how much it can help. In fact, I bet if you did an experiment comparing people reading a textbook about a somewhat-familiar language, vs. just asking GPT to do things and noticing the bugs, the latter group would reach proficiency faster.
squeaky-clean 4y ago
It's just way easier for me to edit some existing code than it is for me to write it from scratch. Especially if it's incorrect (a la XKCD 386). For stuff where I'm not sure where to start it's been very helpful. Even though I often keep very little of the GPT generated code by the time I'm done.

thorax 4y ago

> It returns factually incorrect data, and it returns code with subtle but important errors if you ask it anything that's not regurgitated a thousand times in the training dataset. To be fair, that's what pretty much every person does. The bar does seem pretty high if we need more than that (especially if not specifically trained on a topic). It's not a universally perfect expert servant, but I've been exploring the code generation of GPT4 in detail (i.e. via the 'cataclysm' module I just posted about). In 1 minute it can write functions as good as the average developer intern most of the time. We're keeping score in a weird way if we're responding quickly with it needing to "code without subtle but important errors". Because that's the majority of human developers, too. I've been writing code for 30 years, and if you put a gun to my head, I would still have subtle but important flaws in every first typing of any complex generated code. I'm not saying you're bashing it, by the way, I get your point, but I do worry a bit when the first response is citing that the SOTA models get things wrong in 0-shot situations without full context. That's describing all of us.

corbulo 4y ago
The standard should be higher for machines. Historically, they're inorganic tools. If a hammer was only as dense as bone, would it be a good hammer?
- stale2002 4y ago
  > would it be a good hammer? Yes, it is possible that such a hammer would still be very useful, if that was the only available option. Imperfect hammers can still be very very useful, when compared to having no hammers.
taberiand 4y ago
I always say, if some nontrivial code compiles and runs on the first attempt, then you just haven't found the bugs yet. GPT-4 is a fantastic collaboration tool for senior developers, who know what they want in detail and can review, verify and apply the output it generates. Just yesterday I needed to write some detailed bash scripts. I'm no Linux guru but I know what I want and that was enough - in minutes I had a solid script that did everything I needed and wanted, something that would have taken much longer to hunt down through Google. And then I asked it about SQL, C#, AWS, Terraform, Rust and on and on and everything was high quality. The only way I could have gained similar results without ChatGPT would have been to post all my questions to the dev slack channel and engage in hours long discussions with my colleagues.

yodsanklai 4y ago

> factual output was never GPT-4's nor any other LLM's design goal. Tell that to 50% of HN who think it's going to replace Google. I think they should be a red disclaimer that answers are often incorrect and shouldn't be trusted in any way. That being said, I think it's a useful tool for all queries that don't expect a factually correct answer, or can be easily checked. For instance, summary, rewriting, simple code synthesis, and possibly many other niche applications.