4 ms·
My experience using GPT-4 for coding is that it's got the knowledge and skill of a senior engineer, with the high maintenance of a junior engineer. You can get
by chatmasta 3y ago
My experience using GPT-4 for coding is that it's got the knowledge and skill of a senior engineer, with the high maintenance of a junior engineer. You can get it to output small sections of quality code, but the amount of prodding it takes to piece it all together means you may as well have spent the time writing it yourself. But the future of GPT as a coding assistant is definitely bright. It just needs more chaining, so I feel less like its assistant, asking it to come up with prompts and then pasting them back to it after iterating on the code.
- Our_Benefactors 3y agoIt’s still way faster than I could hope to be. It’s a true refactoring machine with 0 mental effort. “Here is this code block, make it do this other thing, add this feature and argument and error condition” etc. It augments the amount of output that’s possible to achieve at any seniority level.
- moffkalast 3y ago> so I feel less like its assistant Ah so I'm not the only one who feels this way lol. It's as brilliant as it is dumb.
- naiv 3y agoFor me it is not even the code per se but how it helps me with naming database tables and properties when I describe the broader scope. The future is more than bright, especially with a new model that is more current. Offtopic but just today I was aksing it for the most current versions it knows of right now: Python: 3.9 JavaScript: ECMAScript 2021 (ES12) Java: 16 C++: C++20 C#: 9.0 (.NET 5.0) Ruby: 3.0.0 Swift: 5.4 Go: 1.16 Rust: 1.51.0 TypeScript: 4.3 PHP: 8.0 Kotlin: 1.5.0 Scala: 3.0 R: 4.0.5 Perl: 5.32 And frameworks: Python: Django: Version 3.2 Flask: Version 2.0.1 Pyramid: Version 2.0 TensorFlow: Version 2.6.0 PyTorch: Version 1.9.0 JavaScript: Node.js: Version 16.9.1 Express.js: Version 4.17.1 React.js: Version 17.0.2 Angular: Version 12.2.4 Vue.js: Version 3.2.6 Java: Spring Framework: Version 5.3.9 Hibernate: Version 5.4.32.Final Struts: Version 2.5.26 PHP: Laravel: Version 8.54.0 Symfony: Version 5.3.6 CodeIgniter: Version 4.1.4 Ruby: Ruby on Rails: Version 6.1.4 Sinatra: Version 2.1.0 C#: .NET Core: Version 5.0 ASP.NET: Version 5.0 Entity Framework Core: Version 5.0
- dontupvoteme 3y agoOh that's interesting, I didn't think about asking it directly about what version it thinks is current. I wonder if it can hallucinate here? It's really irksome when it tries to use functions which were renamed or removed. This could be detected automatically (and possibly remapped in some cases) I got mostly the same versions as you, both on chatgpt3/4, using english. If you ask it in other languages the minor version also seems to change often. Finnish gives you dates, and code-davinici-edit-001 is 1 major release behind on almost everything. Tässä on luettelo ohjelmistojen viimeisimmistä vakaiden versioista: Python: 3.9.5 (30.4.2021) JavaScript: ECMAScript 2021 (23.3.2021) ECMAScript: ECMAScript 2021 (23.3.2021) Java: JDK 17 (28.9.2021) C++: C++20 (20.2.2020) C#: .NET 6 (8.11.2022) Ruby: 3.0.2 (24.8.2021) Swift: Swift 5.5 (20.9.2021) Go: 1.17 (16.8.2021) Rust: 1.54.0 (27.5.2021) TypeScript: 4.4 (28.7.2021) PHP: 8.0.9 (29.7.2021) Kotlin: 1.5.31 (26.8.2021) Scala: 2.13.6 (17.2.2021) R: 4.1.0 (18.5.2021) Perl: 5.34.0 (30.5.2021) ... Sinatra: 2.1.0 (10.4.2021) .NET Core: 6.0 (8.11.2022) ASP.NET: 5.0.10 (19.8.2021)
- BeefWellington 3y agoAll it is doing is responding with things that look like accurate languages and version numbers. They may be accurate but are absolutely not representative of what it has been trained against.
- RadiozRadioz 3y agoAsking it directly is a good way to get a response containing version numbers that look correct, but not a good way to actually indicate which versions it's been trained on. A better method would be to look at the language features it's using and infer from there. Or, better still, look at which versions were out when the training data was collected (which I believe is September 2021, don't quote me on that).
- grenoire 3y agoIs there a risk of it hallucinating what new features could do instead of basing information on having seen them before?
- dontupvoteme 3y agoThe main thing it's lacking is a client-side merge/error-detection engine which combines all of the previous outputs. Anything over ~100LoC and it gets sloppy including it all in the code block. Which is fine, because I don't need it in your context window if that bit is working and you're not relying on it. Though I have gotten outputs up to nearly 250 lines from it.. Automatically feeding back code details + Traceback messages also fixes errors decently often enough.
- alfalfasprout 3y agoThose aren't very good senior engineers then lol. GPT-4 has the coding skills of an overconfident junior engineer and little else. I agree the future of GPT-X models as coding tools is bright. But for actually doing engineering work outside coding (or even delicate changes in an existing code base) much less so.
- _ea1k 3y agoI'd love it if junior engineers organized code as well as ChatGPT. I'd also love it if all engineers stopped mixing spaces and tabs.
- deleted 3y ago[deleted]
- alfalfasprout 3y agoAdd a linting step to your CI to catch this stuff?
- _ea1k 3y agoMaybe I could get chatgpt to write those pipeline steps for me. :)
- oarsinsync 3y agoYou can and you should. It’s this kind of busywork I farm out to GPT these days and it’s great at it. I suck at it, so it saves me an hour to focus on what I’m good at!
- kristopolous 3y agoThat can mostly be fixed with conventional prettify tools. No modern ai required
- fragmede 3y agoDo you have a style guide? ChatGPT, on spaces vs tabs: https://chat.openai.com/share/b2b0be49-54f9-4f73-a753-3edbd1ead4c0 https://chat.openai.com/share/b2b0be49-54f9-4f73-a753-3edbd1...
- jorblumesea 3y agoGenerative AI will not be able to do that in any true sense. Not the way you're thinking about it.
- Mizoguchi 3y agoAgree. However I noticed an unintended benefit of using GPT for coding is that it helps you think carefully about the problem you are trying to solve when writing the prompt.
- behnamoh 3y ago"Language as a tool of thought" is a technology we often neglect we are equipped with.
- dkersten 3y agoThis is essentially what rubber ducking is.
- AtNightWeCode 3y agoThat is why most developer jobs are safe. Tasks from stakeholders often contains only a title.
- chasd00 3y agoi don't use it that much for code but when i do it's for a specific function or method. I describe the inputs, the logic i want, and the outputs i need. So basically i have it worked out in my head I just let chatgpt type it out for me. Any mistakes it makes are pretty easy to catch when using it this way.
- flir 3y agoRubber ducking in the age of WFH is definitely a use I've found for it.
- furyofantares 3y agoGPT-4 has incredible breadth and no depth. It's an excellent complement to a strong programmer, who will have incredible depth -- and may have breadth relative to other coders, but will be very narrow in the whole space of programming. Just don't use it for things you're already an expert on, except perhaps for starting out / bypassing boilerplate.
- nbardy 3y agoIt has depth as well. You just need to copy in the relevant docs. It’s writing fault tolerant data scraping scripts for me I don’t know how to write myself.
- rngname22 3y agoIt's Einstein with dementia/alzheimers.
- throwuwu 3y agoThey just put out a blog post about how they’re doing exactly that by rewarding chain of thought style reasoning.