5 ms·

ChatGPT is not capable of writing production quality code. Many (most) companies have internal policies against deploying any code written by an LLM. The point

ChatGPT is not capable of writing production quality code. Many (most) companies have internal policies against deploying any code written by an LLM. The point isn’t to slow devs down, but to mitigate risk. This is especially important in the customer/payments stack. This is not the right place to “save a couple hours”. Maybe if this was for some one-off offline analysis, sure. The fact that it works is insufficient proof that it was the right thing to do. Building a habit of relying on LLM generated code is an inherently risky practice, and ChatGPT will literally warn you against trusting its outputs. Sure, it lets you growth hack your way through sort term problems, but in the long term I’m not convinced this is responsible decision making at the current levels of LLM technology. Or maybe I’m just a Luddite, stuck in my old ways.

mrdatawolf 3y ago
My current suggestion is to consider it the work of a just on-boarded intern. It will save you some time but you still need to walk thru the code to make sure it will work as intended.
- b112 3y ago
  My experience has been it's faster to write code yourself, than via a just on boarded intern + review + fixes.
  - deleted 3y ago
    [deleted]
  - i_am_jl 3y ago
    The time savings isn't down to quality, the difference is that an LLM does in seconds what an intern does in hours or days.
  - climb_stealth 3y ago
    Yes, but part of that time is an investment into the intern's professional development. Everyone started there at some point. It can be hard to remember though when there are unrealistic deadlines and helping someone inexperienced to do the work is twice the effort.
- TechBro8615 3y ago
  First, it's worth noting the code in the blog post is not "production code," but rather one-off or periodically used scripts for accelerating manual business processes, with results that are easy to manually check. But in regards to production code, I agree. When code is committed to a codebase, a human should review it. Assuming you trust your review process, it shouldn't matter whether the code submitted for review was written by a human or a language model. If it does make a difference, then your review process is already broken. It should catch bad code regardless of whether it was created by human or machine. It's still worth knowing the source of commits, but only for context in understanding how it was generated. You know humans are likely to make certain classes of error, and you can learn to watch out for the blind spots of your teammates, just like you can learn the idiosyncrasies and weak points of GPT generated code. Personally, I don't think we're quite at "ask GPT to commit directly to the repo," but we're getting close. The constant refrain of "try GPT-4" has become a trope, but the difference is immediately noticeable. Whereas GPT-3.5 will make a mistake or two in every 50 line file, GPT-4 is capable of producing fully correct code that you can immediately run successfully. At the moment it works best for isolated prompts like "create a component to do X," or "write a script to do Y," but if you can provide it with the interface to call an external function, then suddenly that isolated code is just another part of an existing system. As tooling improves for working collaboratively with large language models and providing them with realtime contextual feedback of code correctness (especially for statically analyzeble or type-checked languages), they will become increasingly indispensable to the workflow of productive developers. If you haven't used co-pilot yet, I encourage you to try it for at least a month. You'll develop an intuition for what it's capable of and will eventually wonder how you ever coded without it. Also make sure to try prompting GPT-4 to create functions, components or scripts. The results are truly surprising and exciting.
linuxftw 3y ago
I use ChatGPT to write code for work constantly. The quality is quite high, it saves me lots of time, on the order of hours typically. If a company prevents me from using ChatGPT, I will use it clandestinely unless they offer an equivalent. There's no going back.
- fxleach 3y ago
  This is outright false. I have used ChatGPT many times over the last couple months and I have caught it give me un-working code, unfinished code, and terribly buggy code. When you point this out it will say Oh sorry about that here is an updated version, and I've caught it give another bug, and another after that. If you are telling me the quality of code that ChatGPT gives you is high then it pains me to say but you must not provide high quality code yourself.
  - linuxftw 3y ago
    Sure, it doesn't nail it 100% on the first prompt 100% of the time. Sometimes it takes a few prompts. It's no big deal. If you can't get it to write effective code, either you're working in a very niche area, or you haven't figured out how to use it properly.
    - runnerup 3y ago
      Another reason someone can’t get it to write effective code is if they don’t know how to code or aren’t a very good programmer. I use it a ton. Most of the time it’s very helpful, sometimes I can’t get it to write effective code. If the code it outputs doesn’t meet my standards, I just don’t use it. But I know what I’m looking for, and when ChatGPT generates it, if not only saves me a shitload of time, but more importantly it saves me a ton of mental energy that I can spend elsewhere. The biggest thing for me is that using ChatGPT helps my brain do fewer “context switches” between focus on high level business logic and low level implementation logic. By staying “high level” I’m able to accomplish more each day because I don’t get lost in the sauce as often. I often have to “upgrade” the code myself with tests, better comments, modify the data structures a bit. Sometimes I tell ChatGPT to do this, sometimes I do it myself. But it’s been very helpful overall. The big takeaway is that your output will only be as good as your own programming skill, regardless if you use ChatGPT or write it yourself.
pengaru 3y ago
It's terrible, but not far removed from what's been already happening with "developers" copying and pasting StackOverflow comments into a text editor and making uninformed compiler-error-guided-edits until it runs then done! The root of the problem here is people making production stuff who don't know wtf they're doing. If they turn to SO posts, LLMs, or "developers" on fiverr/upwork doing the same thing, is there really much of a difference? LLMs seem to mostly be tightening the loop of horror that's already been happening. Same downward trajectory, increased velocity.
- libraryatnight 3y ago
  Just seems like programming will be joining the ranks of most tools. There will always be craftsmen, there will always be professionals, and then there will be the guy bolting together ice chests and garbage disposals to make margarita mixers on his patio or the kid with a duct tape exhaust rig on his Honda Civic. I guess, to your point, it's only trouble if the margarita mixer guy is put in charge of something that matters? :D (might be a bad example, I've known some fine engineers and mechanics that are absolutely margarita mixer guy, but hopefully my point is taken lol)
  - pixl97 3y ago
    >Just seems like programming will be joining I'd love to know about the nirvana you've been in up till now, because working around code from numerous large companies the vast majority of it is the crappest ass crap straight from the crapper with no redeeming qualities, and it has been this way forever. I'm not saying their isn't good parts, there are general core routines that sheer need for them to be performant and non-data corrupting forced some Sr engineer to fix them.
kredd 3y ago
Frankly speaking, probably the latter. I've been using Copilot for over a year now, and obviously it makes stupid mistakes, but it sped up my general coding speed. Now, I don't have much experience (maybe around 10ish years of programming professionally) in comparison to greybeards, but it works. Haven't used ChatGPT much, but as long as the user understands its shortcomings and reviews/refines its outputs, it's fine. People who write code also make mistakes, yet we don't consider it "inherently risky practice". We just review others' code, tweak it, make it more appropriate for prod and voila. Same thing applies here.
- dylan604 3y ago
  >but as long as the user understands its shortcomings and reviews/refines its outputs, it's fine. nice caveat doing a heckuvallot of heavy lifting. i understand that we're talking about coders and sort have this inferred impression that coders will have this understanding, but...that's an awfully broad brush you've used to paint over the simple fact that most people using LLMs (in general) are not understanding this.
  - MetaWhirledPeas 3y ago
    > the simple fact that most people using LLMs (in general) are not understanding this How do you know most people using LLMs are not understanding this?
    - dylan604 3y ago
      Because ChatGPT has been opened to the public
      - inopinatus 3y ago
        PHP also lowered the bar to programming, and we got over the consequences of that. Eventually.
  - tmpX7dMeXU 3y ago
    Can we please get back to the actual topic is hand. The author clearly says that they took measures that’d seem to imply that they have an understanding of the shortcomings of ChatGPT. You’re using this as a soapbox to cast holier than thou elitist aspersions on an imagined Everyone Else that isn’t as enlightened as you. This exact “LLMs bad and here’s why!” thread of conversation is getting so old. The fact that it always has the same few talking points is evidence enough that those indulging themselves in it have been party to these conversations before. They know how it goes. And now it’s their turn to say the same old tired and in this case largely irrelevant things to sound smart and to pat themselves on the back.
Pxtl 3y ago
Realistically chatgpt isn't writing the financial code. Stripe did that already. Chatgpt is just reading snippets of Stripe's API examples for you and applying the code for a common use-case.
inopinatus 3y ago
The latter, I'd have to suggest. GPT-4 generates code that is slightly better than the average junior programmer, which is to say, it is often confidently incorrect and needs review before committing, but either option remains a net productivity gain than no assistant at all. "Your job will not be taken by an AI. Your job will be taken by someone assisted by an AI." The process touched on in the article, with thorough review before commit by a human with in-depth experience of the language and APIs and the domain in question, is exactly how AI-generated code should be incorporated into a workflow. The earlier slander against the author's technical ability seems misguided and unsupportable.
AussieWog93 3y ago
I've used ChatGPT (GPT-4) to write production code. As long as you keep the scope small ("Write some example code that calls $API in Python", "Make it asynchronous; so I can queue up n calls and execute them in parallel"), it generates perfectly good code that is easy to understand too.
imroot 3y ago
The main reason that the infosec folks have paused the use of LLM's to generate code is copyright concerns: who "owns" the code that's generated, the LLM, or, the company?
2c2c2c 3y ago
read the code before you use it? what are you even typing man
1letterunixname 3y ago
That's slow corporate thinking. I'm familiar with it but disagree because the efficiency gains are undeniable and it puts a business on a noncompetitive stance if they fail to change with the times. ChatGPT and Copilot are like intern software devs who can produce code in seconds. They generate code that's usually close, but not always correct, at the savings of a great deal of your time of typing the whole thing vs. verifying correctness. For critical and complex algorithms, it's not worth using ML coding assistants right now, but they will be in the future. It's obvious that that's where it's headed: massive efficiency gains for non-technical and barely technical people, and the decline in demand for software engineers, and with it, also a decline in software engineering salaries.

mrdatawolf 3y ago

My current suggestion is to consider it the work of a just on-boarded intern. It will save you some time but you still need to walk thru the code to make sure it will work as intended.

b112 3y ago
My experience has been it's faster to write code yourself, than via a just on boarded intern + review + fixes.
- deleted 3y ago
  [deleted]
- i_am_jl 3y ago
  The time savings isn't down to quality, the difference is that an LLM does in seconds what an intern does in hours or days.
- climb_stealth 3y ago
  Yes, but part of that time is an investment into the intern's professional development. Everyone started there at some point. It can be hard to remember though when there are unrealistic deadlines and helping someone inexperienced to do the work is twice the effort.
TechBro8615 3y ago
First, it's worth noting the code in the blog post is not "production code," but rather one-off or periodically used scripts for accelerating manual business processes, with results that are easy to manually check. But in regards to production code, I agree. When code is committed to a codebase, a human should review it. Assuming you trust your review process, it shouldn't matter whether the code submitted for review was written by a human or a language model. If it does make a difference, then your review process is already broken. It should catch bad code regardless of whether it was created by human or machine. It's still worth knowing the source of commits, but only for context in understanding how it was generated. You know humans are likely to make certain classes of error, and you can learn to watch out for the blind spots of your teammates, just like you can learn the idiosyncrasies and weak points of GPT generated code. Personally, I don't think we're quite at "ask GPT to commit directly to the repo," but we're getting close. The constant refrain of "try GPT-4" has become a trope, but the difference is immediately noticeable. Whereas GPT-3.5 will make a mistake or two in every 50 line file, GPT-4 is capable of producing fully correct code that you can immediately run successfully. At the moment it works best for isolated prompts like "create a component to do X," or "write a script to do Y," but if you can provide it with the interface to call an external function, then suddenly that isolated code is just another part of an existing system. As tooling improves for working collaboratively with large language models and providing them with realtime contextual feedback of code correctness (especially for statically analyzeble or type-checked languages), they will become increasingly indispensable to the workflow of productive developers. If you haven't used co-pilot yet, I encourage you to try it for at least a month. You'll develop an intuition for what it's capable of and will eventually wonder how you ever coded without it. Also make sure to try prompting GPT-4 to create functions, components or scripts. The results are truly surprising and exciting.

linuxftw 3y ago

I use ChatGPT to write code for work constantly. The quality is quite high, it saves me lots of time, on the order of hours typically. If a company prevents me from using ChatGPT, I will use it clandestinely unless they offer an equivalent. There's no going back.

fxleach 3y ago
This is outright false. I have used ChatGPT many times over the last couple months and I have caught it give me un-working code, unfinished code, and terribly buggy code. When you point this out it will say Oh sorry about that here is an updated version, and I've caught it give another bug, and another after that. If you are telling me the quality of code that ChatGPT gives you is high then it pains me to say but you must not provide high quality code yourself.
- linuxftw 3y ago
  Sure, it doesn't nail it 100% on the first prompt 100% of the time. Sometimes it takes a few prompts. It's no big deal. If you can't get it to write effective code, either you're working in a very niche area, or you haven't figured out how to use it properly.
  - runnerup 3y ago
    Another reason someone can’t get it to write effective code is if they don’t know how to code or aren’t a very good programmer. I use it a ton. Most of the time it’s very helpful, sometimes I can’t get it to write effective code. If the code it outputs doesn’t meet my standards, I just don’t use it. But I know what I’m looking for, and when ChatGPT generates it, if not only saves me a shitload of time, but more importantly it saves me a ton of mental energy that I can spend elsewhere. The biggest thing for me is that using ChatGPT helps my brain do fewer “context switches” between focus on high level business logic and low level implementation logic. By staying “high level” I’m able to accomplish more each day because I don’t get lost in the sauce as often. I often have to “upgrade” the code myself with tests, better comments, modify the data structures a bit. Sometimes I tell ChatGPT to do this, sometimes I do it myself. But it’s been very helpful overall. The big takeaway is that your output will only be as good as your own programming skill, regardless if you use ChatGPT or write it yourself.

pengaru 3y ago

It's terrible, but not far removed from what's been already happening with "developers" copying and pasting StackOverflow comments into a text editor and making uninformed compiler-error-guided-edits until it runs then done! The root of the problem here is people making production stuff who don't know wtf they're doing. If they turn to SO posts, LLMs, or "developers" on fiverr/upwork doing the same thing, is there really much of a difference? LLMs seem to mostly be tightening the loop of horror that's already been happening. Same downward trajectory, increased velocity.

libraryatnight 3y ago
Just seems like programming will be joining the ranks of most tools. There will always be craftsmen, there will always be professionals, and then there will be the guy bolting together ice chests and garbage disposals to make margarita mixers on his patio or the kid with a duct tape exhaust rig on his Honda Civic. I guess, to your point, it's only trouble if the margarita mixer guy is put in charge of something that matters? :D (might be a bad example, I've known some fine engineers and mechanics that are absolutely margarita mixer guy, but hopefully my point is taken lol)
- pixl97 3y ago
  >Just seems like programming will be joining I'd love to know about the nirvana you've been in up till now, because working around code from numerous large companies the vast majority of it is the crappest ass crap straight from the crapper with no redeeming qualities, and it has been this way forever. I'm not saying their isn't good parts, there are general core routines that sheer need for them to be performant and non-data corrupting forced some Sr engineer to fix them.

kredd 3y ago

Frankly speaking, probably the latter. I've been using Copilot for over a year now, and obviously it makes stupid mistakes, but it sped up my general coding speed. Now, I don't have much experience (maybe around 10ish years of programming professionally) in comparison to greybeards, but it works. Haven't used ChatGPT much, but as long as the user understands its shortcomings and reviews/refines its outputs, it's fine. People who write code also make mistakes, yet we don't consider it "inherently risky practice". We just review others' code, tweak it, make it more appropriate for prod and voila. Same thing applies here.

dylan604 3y ago
>but as long as the user understands its shortcomings and reviews/refines its outputs, it's fine. nice caveat doing a heckuvallot of heavy lifting. i understand that we're talking about coders and sort have this inferred impression that coders will have this understanding, but...that's an awfully broad brush you've used to paint over the simple fact that most people using LLMs (in general) are not understanding this.
- MetaWhirledPeas 3y ago
  > the simple fact that most people using LLMs (in general) are not understanding this How do you know most people using LLMs are not understanding this?
  - dylan604 3y ago
    Because ChatGPT has been opened to the public
    - inopinatus 3y ago
      PHP also lowered the bar to programming, and we got over the consequences of that. Eventually.
- tmpX7dMeXU 3y ago
  Can we please get back to the actual topic is hand. The author clearly says that they took measures that’d seem to imply that they have an understanding of the shortcomings of ChatGPT. You’re using this as a soapbox to cast holier than thou elitist aspersions on an imagined Everyone Else that isn’t as enlightened as you. This exact “LLMs bad and here’s why!” thread of conversation is getting so old. The fact that it always has the same few talking points is evidence enough that those indulging themselves in it have been party to these conversations before. They know how it goes. And now it’s their turn to say the same old tired and in this case largely irrelevant things to sound smart and to pat themselves on the back.

Pxtl 3y ago

Realistically chatgpt isn't writing the financial code. Stripe did that already. Chatgpt is just reading snippets of Stripe's API examples for you and applying the code for a common use-case.

inopinatus 3y ago

The latter, I'd have to suggest. GPT-4 generates code that is slightly better than the average junior programmer, which is to say, it is often confidently incorrect and needs review before committing, but either option remains a net productivity gain than no assistant at all. "Your job will not be taken by an AI. Your job will be taken by someone assisted by an AI." The process touched on in the article, with thorough review before commit by a human with in-depth experience of the language and APIs and the domain in question, is exactly how AI-generated code should be incorporated into a workflow. The earlier slander against the author's technical ability seems misguided and unsupportable.

AussieWog93 3y ago

I've used ChatGPT (GPT-4) to write production code. As long as you keep the scope small ("Write some example code that calls $API in Python", "Make it asynchronous; so I can queue up n calls and execute them in parallel"), it generates perfectly good code that is easy to understand too.

imroot 3y ago

The main reason that the infosec folks have paused the use of LLM's to generate code is copyright concerns: who "owns" the code that's generated, the LLM, or, the company?

2c2c2c 3y ago

read the code before you use it? what are you even typing man

1letterunixname 3y ago

That's slow corporate thinking. I'm familiar with it but disagree because the efficiency gains are undeniable and it puts a business on a noncompetitive stance if they fail to change with the times. ChatGPT and Copilot are like intern software devs who can produce code in seconds. They generate code that's usually close, but not always correct, at the savings of a great deal of your time of typing the whole thing vs. verifying correctness. For critical and complex algorithms, it's not worth using ML coding assistants right now, but they will be in the future. It's obvious that that's where it's headed: massive efficiency gains for non-technical and barely technical people, and the decline in demand for software engineers, and with it, also a decline in software engineering salaries.