7 ms·

Is programming even the hard part about programming? In all seriousness, what we would really need from an AI to start really saving me time would be for it to

Is programming even the hard part about programming? In all seriousness, what we would really need from an AI to start really saving me time would be for it to interview all the customers/partners involved on the project, determine the scope of function needed, boil all that down to a set of sensible domain models that make sense to everyone, identify where/when messages need to be passed, determine which things can happen piecemeal as opposed to requiring a bulk op, deciding on a persistence technology based on the nature of the requirements... And that's just an abbreviated form of what I go through when designing a back-end for a tiny boutique application. 99% of programming is making decisions. When you finally have everything planned, the code can almost write itself, but getting to that point requires so much background knowledge that I'm not sure GPT4 will be hunting for my job anytime soon. Or even really augmenting it. I'd be happier if they could just get auto-complete in VS Code to not suck complete balls.

snickerbockers 3y ago
i'm honestly not worried at all about LLMs because most of my jobs have consisted of fixing problems in other peoples' code, and i haven't seen any evidence that an LLM will be capable of doing that on a non-trivial program any time in the near future. I have, however, seen evidence that chatgpt will create many more problems and make my job harder.
- SrslyJosh 3y ago
  > i haven't seen any evidence that an LLM will be capable of doing that on a non-trivial program any time in the near future Or ever, given that the level of abstraction LLMs work at is completely wrong. They can approximate the syntax of things in their training corpus, but logic? The lights are off and nobody's home.
  - noobermin 3y ago
    The bad news is that everyone (including the CEO interviewed) who lords over you don't know that and won't hesitate dumping all that work on you.
  - eru 3y ago
    Figuring out the logic in code doesn't seem that different from figuring out the logic in other human produced text. At least it doesn't seem harder, if anything it's probably easier for a machine. Yes, at the moment GPT4 and the like aren't all that good yet, but they have shown that they have started understanding semantics.
  - two_in_one 3y ago
    It all started this year, just a few months ago, remember? It will get only better from here, don't worry. Or do, not sure. Anyway, you can't avoid it.
    - JohnFen 3y ago
      > Anyway, you can't avoid it. That seems certain. It's why I'm putting serious thought into leaving the industry.
      - two_in_one 3y ago
        Carpeting and nursing looks safe for now. Most other jobs are not. Some are very competitive already, like painting, writing, all sorts of design. Without AI it will be hard to find a job there. Driving, piloting will be mostly automated soon. Rather then leaving it's better to adapt. AI is just one of technologies in the list. They come and go, that's the specific of IT. Except AI will stay, it will change with the time, but will never go away. Besides, it's the coolest thing right now. And will create new jobs around itself.
  - XenophileJKO 3y ago
    I've already had the GPT3.5-Turbo model walk through and step-by-step isolate and diagnose errors. They 100% can troubleshoot and correct issues in the code. Literally you give it the code and the error and it can walk you through finding the solution. When I say walk you through, I generally mean when you provide it a function but the error is caused by some input that doesn't conform to expectations. If the error were just a defect in the code it can generally point that out instantly. Obviously GPT4 is even better.
    - Panzer04 3y ago
      Most bugs I've worked on relate to some weirdness that requires tracking down a specific nonobvious offending function. How would GPT help with that at all? Maybe if you know a particular function is wrong, and ask it to find a bug, but by then most of the work has already been done.
    - snickerbockers 3y ago
      >If the error were just a defect in the code it can generally point that out instantly. thats not fixing bugs, that's static analysis. Finding the solution to the specific problem that needs to be solved is a lot more difficult than identifying any problem and then solving it.
- kytazo 3y ago
  That's wishful thinking if you ask me
- XenophileJKO 3y ago
  I wouldn't be so sure. Combine current capability levels with the larger context windows and they can probably already point out most of the problems with code. I recently fed a very large file into GPT4 and it handed me a few serious bugs that I hadn't noticed after a few self-reviews.
- Gigachad 3y ago
  I remember >10 years ago thinking that learning another language was probably pointless because google translate is 90% there already and soon will obsolete that knowledge. Or all the people saying trucking is on the verge of being automated.
  - JohnFen 3y ago
    > google translate is 90% there 90% there? I disagree. 90% would mean that most of the time it gets the translation right. That hasn't been my experience.
padolsey 3y ago
> the code can almost write itself My 2c: I've been an eng for around 15 yrs. I semi-recently had a brain injury so haven't been able to dedicate anywhere near as much mental cognition to programming recently. That's why I've been unable to maintain full-time work. I started using chatgpt around 3 months ago. Initially skeptical, I started giving it fun and weird logical/semantic puzzles to satisfyingly "prove" my intuition that it was insufficient to solve any true problems (and that we humans are still needed!). However, I soon became humbled to its capabilities. There are many many things it cannot do, but I've been amazed at the things it can do if given nuanced and detailed enough prompts. I've realised that, if I prompt it well enough, and use my existing knowledge from those accrued 15yrs, I can get awesome results. I'm working on a small project right now and it's written about ~70% of the code. I've had to make various corrections along the way, but I've found that I can focus on content and larger 'macro' domain logic than annoying (tho intriguing) procedural coding. It's been so incredibly empowering and freeing to be able to dedicate more brain to _what_ I want to build instead of _how_ I want to build it. My normal build process is now something like this: - state problem and what you desire, with good specificity - [optional] give it your current working code, specifying frameworks, rough file structure - confirm it understands and clarify/correct it if necessary - [important] ask it to ask _you_ questions for clarification - ask it for an overview of how it would solve the problem - (for big tasks, expect a macro rundown) - (for small tasks, expect or ask for the actual code) - [important] make specific requests about lang/module/algorithms - [important] ask it to write test suites to prove its code works and paste in any errors or assertion failures you encounter. It'll surprise you with its advice. It doesn't replace my need to code, but OMG it makes it so much less burdensome and I'm learning a tonne as well : )
- dwaltrip 3y ago
  That’s super interesting. I’m recovering from burnout and other health issues and I’ve found it to be occasional helpful in the way you are describing. For me, it can smooth out the process and “lower the intensity” of accomplishing any particular task, especially if it is something where I don’t know how to do it off the top of my head (what libs / functions to use, how to call them, etc). I can then pretty easily correct any mistakes, and I didn’t have to spend 20 mins googling, reading docs, and so on in order to solve the problem. If you don’t mind, do you have any good examples of how you prompt it? Your process looks pretty nice / robust, it would be cool to see it in action. Also, have you used gpt-4 much or can you get away with using 3.5 sometimes?
eru 3y ago
Whether this approach works depends a lot on what you are trying to write. GPT4 is not very good at understanding new algorithms and data structures for example. (I recently tried very hard, but it failed miserably. I can talk about the details, if someone is interested.) But it might be good enough at helping you organise a sprawling project.
- rubyskills 3y ago
  Yes I'd like the details on this. My experience has been the opposite of you prompt it correctly, or it has the algorithm or data structure trained in its model already.
bruce511 3y ago
I don't need it to write code. I don't need it to interview customers. I need it yourself attend endless, pointless, weekly zoom meetings where a manager with zero understanding of the task being done, why its being done, and with no idea how to do it, nevertheless is happy to review open tickets and discuss them. This task will take 2 months. I'm busy working on it. If you want a 5 minute email every week on how its going, then fine. If you want me to throw my toys when i hit a road-block then I'm for with that. But no, we need weekly status update meetings with all the other developers, testers, product owners, all wasting their and my time, just because a manager is "managing". Forget code, that's not the hard part. When the AI can just be my doppelganger in the meeting on my behalf THEN I'll worry about AI taking my job.
- pkdpic 3y ago
  In all seriousness I haven't seen the idea of LLMs replacing managerial functions in a while, it would be an interesting inversion of a quasi-post-labor utopia.
  - bruce511 3y ago
    AI would likely make great managers. Given competent staff. Alas while I rag on about incompetent managers, it's not uncommon to find staff who, charitably, need a lot of "management".
    - jiggawatts 3y ago
      The best part of AI management would be that for the first time, the manager would understand what their direct reports are saying. If I say something like “There may be a compatibility risk due to DNS apex records”, I’ll have to spend hours explaining this to a disinterested non-technical manager. The AI understands the concept and doesn’t need me to explain.
- helsinkiandrew 3y ago
  > I need it yourself attend endless, pointless, weekly zoom meetings where a manager with zero understanding of the task being done. You don't need AI for that, you need to brush up your resume and find another job - The biggest regret of my career is not having left places early when the organisation/management style sucked the enjoyment/productivity out of what you do, particularly if everyone else there agrees with you.
ryoshu 3y ago
Programming is easy. Asking the right question is hard.
devjab 3y ago
Some code writes itself, and I hope generative AI will help with it, but a lot of code doesn’t. Which is likely why generative AI is so terrible at writing good code. Of course that doesn’t mean that generative AI won’t see widespread adoption by non-programmers in digitalisation in the coming decade. We already have a lot of “process people” making things with GPT, and those things work. Or at least, they sort of work, but they are also build so terrible that they won’t scale and won’t be maintainable. Which is fine for a while, and it’s probably even fine for the lifetime of some programs. Because let’s be honest, often the quality of the programs that are implemented in non-tech enterprise aren’t important. In fact, often excel “programmers” can frankly do wonders in terms creating business value on short lived automation that won’t need to scale or be maintained in the long run because it’s simply going to be replaced by the time it stops being useful because you’ll have grown to a size where you’re buying SAP or similar (regardless of whether it’s a good idea or not). I do think that a lot of us are going to spend a lot of time “cleaning up” after non-programmers doing GPT programming. Which will be lucrative and boring. But writing good code for complicated problems? I’m not sure when/if generative AI will be able to handle that. I had hopes until GPT. We use it quite a lot mind you, it writes a lot of our documentation. We have high hopes it’ll eventually get good enough to write a lot of our unit tests as well, and obviously we’re already in a world where a lot of the “trivial” code can be auto-generated, but we were frankly able to do that before generative AI, but actual programming? Heh.
SergeAx 3y ago
You understand that your generated code is from September 2021 at best, right? Maybe it's okay for some niches, but I see lots of evolution in almost all segments of software engineering, especially frontend and ML.
firecall 3y ago
> Is programming even the hard part about programming? Exactly! Figuring out Programming challenges isnt really ever part of the work I do. Which is mostly business process stuff. Comprehending APIs is often a pain. A few times CoPilot has helped by auto-completing the incantation I needed when my brain wasnt working and I couldn't get an understanding from the docs. So as you say, good autocomplete is all I really need. That and decent documentation! There is never a moment where I think I'll break my flow and have a chart conversation to write some code for me. Never. One thing I did think would be useful: If AI could abstract my already written and duplicated code into testable and robust reusable Classes/Methods for me!
- mdp2021 3y ago
  > Comprehending APIs ... good autocomplete ... decent documentation The ability to ask questions about contents of the documentation as opposed to inefficient RTFM is one of the possibilities for which I recognize LLMs as potentially especially useful. (They should also be able to point to the source like actual "search" though.)
- JohnFen 3y ago
  > So as you say, good autocomplete is all I really need. Maybe this is why I haven't found the great value in ChatGPT. I don't find value in autocomplete, good or otherwise.
JSavageOne 3y ago
Interesting take, but I think you're drastically underestimating how much work the programming part is. I think currently at least 90%+ of the work is actually programming / implementing the thing, and that's what AI is going to replace.
- anonu 3y ago
  I'm not sure about the 90% figure you quote. I'd say it's a lot less from my experience. But even in that "programming part" I'd say the time to implement core functionality follows the Pareto principle. You can probably code up 80% of what you need in 20% of the time. The other 80% ends up being QA and bug fix iteration.
gonzo41 3y ago
I am currently uplifting a 30 year old code to a somewhat newer codebase to try and give it another 20-30 years. I spend a lot of time talking with humans to rediscover the context of the system. AI is not going to each my lunch. But it sure is handy being able to make sense of some old logic and syntax. It literally saves me hours in having to figure out strange code snippets and such. GPT is very impressive. But we'll be fine. There's plenty more complexity to come, so be smart, be a part of that complexity.
- ralphc 3y ago
  How does this work & which GPT are you using? So you upload a bunch of code somehow and ask questions about it?
  - gonzo41 3y ago
    I just read it carefully and put the clangers into chatgpt for some suggestions on different approaches. There's a mix of Perl, old PHP, Aspx and mssql plsql all interacting. So it's less about getting it to do all the work, and just keeping me from getting bogged down in the trivial stuff.
pjacotg 3y ago
Agreed, my experience programming has been similar. Simply put the hardest part of most big software projects is just building the right thing.

snickerbockers 3y ago

i'm honestly not worried at all about LLMs because most of my jobs have consisted of fixing problems in other peoples' code, and i haven't seen any evidence that an LLM will be capable of doing that on a non-trivial program any time in the near future. I have, however, seen evidence that chatgpt will create many more problems and make my job harder.

SrslyJosh 3y ago
> i haven't seen any evidence that an LLM will be capable of doing that on a non-trivial program any time in the near future Or ever, given that the level of abstraction LLMs work at is completely wrong. They can approximate the syntax of things in their training corpus, but logic? The lights are off and nobody's home.
- noobermin 3y ago
  The bad news is that everyone (including the CEO interviewed) who lords over you don't know that and won't hesitate dumping all that work on you.
- eru 3y ago
  Figuring out the logic in code doesn't seem that different from figuring out the logic in other human produced text. At least it doesn't seem harder, if anything it's probably easier for a machine. Yes, at the moment GPT4 and the like aren't all that good yet, but they have shown that they have started understanding semantics.
- two_in_one 3y ago
  It all started this year, just a few months ago, remember? It will get only better from here, don't worry. Or do, not sure. Anyway, you can't avoid it.
  - JohnFen 3y ago
    > Anyway, you can't avoid it. That seems certain. It's why I'm putting serious thought into leaving the industry.
    - two_in_one 3y ago
      Carpeting and nursing looks safe for now. Most other jobs are not. Some are very competitive already, like painting, writing, all sorts of design. Without AI it will be hard to find a job there. Driving, piloting will be mostly automated soon. Rather then leaving it's better to adapt. AI is just one of technologies in the list. They come and go, that's the specific of IT. Except AI will stay, it will change with the time, but will never go away. Besides, it's the coolest thing right now. And will create new jobs around itself.
- XenophileJKO 3y ago
  I've already had the GPT3.5-Turbo model walk through and step-by-step isolate and diagnose errors. They 100% can troubleshoot and correct issues in the code. Literally you give it the code and the error and it can walk you through finding the solution. When I say walk you through, I generally mean when you provide it a function but the error is caused by some input that doesn't conform to expectations. If the error were just a defect in the code it can generally point that out instantly. Obviously GPT4 is even better.
  - Panzer04 3y ago
    Most bugs I've worked on relate to some weirdness that requires tracking down a specific nonobvious offending function. How would GPT help with that at all? Maybe if you know a particular function is wrong, and ask it to find a bug, but by then most of the work has already been done.
  - snickerbockers 3y ago
    >If the error were just a defect in the code it can generally point that out instantly. thats not fixing bugs, that's static analysis. Finding the solution to the specific problem that needs to be solved is a lot more difficult than identifying any problem and then solving it.
kytazo 3y ago
That's wishful thinking if you ask me
XenophileJKO 3y ago
I wouldn't be so sure. Combine current capability levels with the larger context windows and they can probably already point out most of the problems with code. I recently fed a very large file into GPT4 and it handed me a few serious bugs that I hadn't noticed after a few self-reviews.
Gigachad 3y ago
I remember >10 years ago thinking that learning another language was probably pointless because google translate is 90% there already and soon will obsolete that knowledge. Or all the people saying trucking is on the verge of being automated.
- JohnFen 3y ago
  > google translate is 90% there 90% there? I disagree. 90% would mean that most of the time it gets the translation right. That hasn't been my experience.

padolsey 3y ago

> the code can almost write itself My 2c: I've been an eng for around 15 yrs. I semi-recently had a brain injury so haven't been able to dedicate anywhere near as much mental cognition to programming recently. That's why I've been unable to maintain full-time work. I started using chatgpt around 3 months ago. Initially skeptical, I started giving it fun and weird logical/semantic puzzles to satisfyingly "prove" my intuition that it was insufficient to solve any true problems (and that we humans are still needed!). However, I soon became humbled to its capabilities. There are many many things it cannot do, but I've been amazed at the things it can do if given nuanced and detailed enough prompts. I've realised that, if I prompt it well enough, and use my existing knowledge from those accrued 15yrs, I can get awesome results. I'm working on a small project right now and it's written about ~70% of the code. I've had to make various corrections along the way, but I've found that I can focus on content and larger 'macro' domain logic than annoying (tho intriguing) procedural coding. It's been so incredibly empowering and freeing to be able to dedicate more brain to _what_ I want to build instead of _how_ I want to build it. My normal build process is now something like this: - state problem and what you desire, with good specificity - [optional] give it your current working code, specifying frameworks, rough file structure - confirm it understands and clarify/correct it if necessary - [important] ask it to ask _you_ questions for clarification - ask it for an overview of how it would solve the problem - (for big tasks, expect a macro rundown) - (for small tasks, expect or ask for the actual code) - [important] make specific requests about lang/module/algorithms - [important] ask it to write test suites to prove its code works and paste in any errors or assertion failures you encounter. It'll surprise you with its advice. It doesn't replace my need to code, but OMG it makes it so much less burdensome and I'm learning a tonne as well : )

dwaltrip 3y ago
That’s super interesting. I’m recovering from burnout and other health issues and I’ve found it to be occasional helpful in the way you are describing. For me, it can smooth out the process and “lower the intensity” of accomplishing any particular task, especially if it is something where I don’t know how to do it off the top of my head (what libs / functions to use, how to call them, etc). I can then pretty easily correct any mistakes, and I didn’t have to spend 20 mins googling, reading docs, and so on in order to solve the problem. If you don’t mind, do you have any good examples of how you prompt it? Your process looks pretty nice / robust, it would be cool to see it in action. Also, have you used gpt-4 much or can you get away with using 3.5 sometimes?

eru 3y ago

Whether this approach works depends a lot on what you are trying to write. GPT4 is not very good at understanding new algorithms and data structures for example. (I recently tried very hard, but it failed miserably. I can talk about the details, if someone is interested.) But it might be good enough at helping you organise a sprawling project.

rubyskills 3y ago
Yes I'd like the details on this. My experience has been the opposite of you prompt it correctly, or it has the algorithm or data structure trained in its model already.

bruce511 3y ago

I don't need it to write code. I don't need it to interview customers. I need it yourself attend endless, pointless, weekly zoom meetings where a manager with zero understanding of the task being done, why its being done, and with no idea how to do it, nevertheless is happy to review open tickets and discuss them. This task will take 2 months. I'm busy working on it. If you want a 5 minute email every week on how its going, then fine. If you want me to throw my toys when i hit a road-block then I'm for with that. But no, we need weekly status update meetings with all the other developers, testers, product owners, all wasting their and my time, just because a manager is "managing". Forget code, that's not the hard part. When the AI can just be my doppelganger in the meeting on my behalf THEN I'll worry about AI taking my job.

pkdpic 3y ago
In all seriousness I haven't seen the idea of LLMs replacing managerial functions in a while, it would be an interesting inversion of a quasi-post-labor utopia.
- bruce511 3y ago
  AI would likely make great managers. Given competent staff. Alas while I rag on about incompetent managers, it's not uncommon to find staff who, charitably, need a lot of "management".
  - jiggawatts 3y ago
    The best part of AI management would be that for the first time, the manager would understand what their direct reports are saying. If I say something like “There may be a compatibility risk due to DNS apex records”, I’ll have to spend hours explaining this to a disinterested non-technical manager. The AI understands the concept and doesn’t need me to explain.
helsinkiandrew 3y ago
> I need it yourself attend endless, pointless, weekly zoom meetings where a manager with zero understanding of the task being done. You don't need AI for that, you need to brush up your resume and find another job - The biggest regret of my career is not having left places early when the organisation/management style sucked the enjoyment/productivity out of what you do, particularly if everyone else there agrees with you.

ryoshu 3y ago

Programming is easy. Asking the right question is hard.

devjab 3y ago

Some code writes itself, and I hope generative AI will help with it, but a lot of code doesn’t. Which is likely why generative AI is so terrible at writing good code. Of course that doesn’t mean that generative AI won’t see widespread adoption by non-programmers in digitalisation in the coming decade. We already have a lot of “process people” making things with GPT, and those things work. Or at least, they sort of work, but they are also build so terrible that they won’t scale and won’t be maintainable. Which is fine for a while, and it’s probably even fine for the lifetime of some programs. Because let’s be honest, often the quality of the programs that are implemented in non-tech enterprise aren’t important. In fact, often excel “programmers” can frankly do wonders in terms creating business value on short lived automation that won’t need to scale or be maintained in the long run because it’s simply going to be replaced by the time it stops being useful because you’ll have grown to a size where you’re buying SAP or similar (regardless of whether it’s a good idea or not). I do think that a lot of us are going to spend a lot of time “cleaning up” after non-programmers doing GPT programming. Which will be lucrative and boring. But writing good code for complicated problems? I’m not sure when/if generative AI will be able to handle that. I had hopes until GPT. We use it quite a lot mind you, it writes a lot of our documentation. We have high hopes it’ll eventually get good enough to write a lot of our unit tests as well, and obviously we’re already in a world where a lot of the “trivial” code can be auto-generated, but we were frankly able to do that before generative AI, but actual programming? Heh.

SergeAx 3y ago

You understand that your generated code is from September 2021 at best, right? Maybe it's okay for some niches, but I see lots of evolution in almost all segments of software engineering, especially frontend and ML.

firecall 3y ago

> Is programming even the hard part about programming? Exactly! Figuring out Programming challenges isnt really ever part of the work I do. Which is mostly business process stuff. Comprehending APIs is often a pain. A few times CoPilot has helped by auto-completing the incantation I needed when my brain wasnt working and I couldn't get an understanding from the docs. So as you say, good autocomplete is all I really need. That and decent documentation! There is never a moment where I think I'll break my flow and have a chart conversation to write some code for me. Never. One thing I did think would be useful: If AI could abstract my already written and duplicated code into testable and robust reusable Classes/Methods for me!

mdp2021 3y ago
> Comprehending APIs ... good autocomplete ... decent documentation The ability to ask questions about contents of the documentation as opposed to inefficient RTFM is one of the possibilities for which I recognize LLMs as potentially especially useful. (They should also be able to point to the source like actual "search" though.)
JohnFen 3y ago
> So as you say, good autocomplete is all I really need. Maybe this is why I haven't found the great value in ChatGPT. I don't find value in autocomplete, good or otherwise.

JSavageOne 3y ago

Interesting take, but I think you're drastically underestimating how much work the programming part is. I think currently at least 90%+ of the work is actually programming / implementing the thing, and that's what AI is going to replace.

anonu 3y ago
I'm not sure about the 90% figure you quote. I'd say it's a lot less from my experience. But even in that "programming part" I'd say the time to implement core functionality follows the Pareto principle. You can probably code up 80% of what you need in 20% of the time. The other 80% ends up being QA and bug fix iteration.

gonzo41 3y ago

I am currently uplifting a 30 year old code to a somewhat newer codebase to try and give it another 20-30 years. I spend a lot of time talking with humans to rediscover the context of the system. AI is not going to each my lunch. But it sure is handy being able to make sense of some old logic and syntax. It literally saves me hours in having to figure out strange code snippets and such. GPT is very impressive. But we'll be fine. There's plenty more complexity to come, so be smart, be a part of that complexity.

ralphc 3y ago
How does this work & which GPT are you using? So you upload a bunch of code somehow and ask questions about it?
- gonzo41 3y ago
  I just read it carefully and put the clangers into chatgpt for some suggestions on different approaches. There's a mix of Perl, old PHP, Aspx and mssql plsql all interacting. So it's less about getting it to do all the work, and just keeping me from getting bogged down in the trivial stuff.

pjacotg 3y ago

Agreed, my experience programming has been similar. Simply put the hardest part of most big software projects is just building the right thing.