9 ms·

Have you ever done DIY? Things like replacing a tap could be done very quickly using YouTube. Why do people still call plumbers? Because reality is never as cle

Have you ever done DIY? Things like replacing a tap could be done very quickly using YouTube. Why do people still call plumbers? Because reality is never as clean-cut as YouTube videos. Sometimes, you find the connection parts have completely corroded, and you can't undo them with your standard DIY tools; other times, you discover you have odd-sized connections (metrics versus non-metrics and vice versa). The real skill of a plumber is to be able to handle real-life problems. It is the same for coding. Yes, you can ask it to generate basic code, and it will work. But my job as a software engineer isn't to write code. It is to deal with messy humans. Sure, ChatGPT can write an app that does three things. Does it work reliably on all different mobile phone devices? And do you only need three things, or do you want 16? Most clients know precisely what they want; they can't express it accurately in words in a single iteration. When you have delivered those three things, they will often ask for more. They will often ask for a hexagon-shaped room after you have built them a perfect circular-based one and then complain heavily if you quote it will take several weeks to refactor.

mnky9800n 2y ago
you also have no way of verifying that the youtube video is telling you to do the right thing. even if it works, it still could not be up to code so suddenly you have expensive changes when you have to sell your house. same with chatgpt. when i ask it to do things i know a lot about, i am often time shocked how bad the code is that it generates and often does different stuff. makes me wonder about the code that i ask it to generate to do things i dont know about. haha.
- FinnLobsien 2y ago
  typical Gell-Mann amnesia. I write a lot so I see this when I've used AI to generate writing. If I ask it to generate something that I understand: "Wow this is the most surface level garbage anyone could create by reading one SEO-optimized article" When it's about something I don't know anything about: "Wow it's impressive how far AI has come" What I do find AI super useful for is finding stuff like metaphors. "I want to find a metaphor between product-led vs. sales-led growth strategies for startups. Please give me a list of 10 i could use" Those prompts are incredibly useful. I reckon there's some type of analogy with coding here.
- atrettel 2y ago
  I've done the same thing for the subject that I did my PhD in, and oh boy is the result weird. It is somewhat right at a high level but most of the low-level details are wrong at worst or confused at best. The explanation is incredibly shallow and misses so many important aspects that matter when implementing the idea in practice. It repeatedly refers to Mark Morkovin as "Nikolai Morkovin". And it even makes up a citation as a reference, though knowing this subject deeply I recognize that it seems to have combined two different references from 1962 and 1977 into a single nonexistent reference.
FinnLobsien 2y ago
Interesting. It sounds like there are two parts to this: a) Product sense becoming more important i.e. talking to users and figuring out what they want and how to build it rather than the pure ability of using code to make the computer do something b) Having the machine but no idea how to repair/augment it only gets you so far. Like you might be able to run a lawn care business but if your lawnmower breaks and you can't repair it your business is dead. Is that about right?
- oxfordmale 2y ago
  Yes, that is about right. It is also about knowing when to mow the lawn. Generally, you shouldn't mow the lawn when it is too wet, as it is too heavy on your mower. You also want to avoid mowing it when it has been too sunny, as the grass will turn brown. That requires experience beyond buying a mower.
- solardev 2y ago
  (Not the original poster) My experience with ChatGPT for coding has actually been the opposite: It was better at understanding and guessing underlying needs than the typical programmer I've worked with, who usually needed everything spelled out in technical bullet point requirements. Clients don't think like that (because they don't know how software works), but programmers need that (or need to come up with it on their own) to write the code. ChatGPT on the other hand seems pretty good at using its vast training data to guess at your intent and needs, as long as it's a relatively common use case. I would say that's one of its great strengths (its ability to combine your prompt and its vast natural language data to give you an answer that effectively summarizes what your good options are). It's a better product owner than most programmers, in other words. On the other hand, its coding ability isn't very impressive. It has the ability to generate code and it will run it and debug it if you ask it to, but it doesn't always do that by itself, and it's not great at doing "sanity tests" along the way to make sure each unit of code it's producing is actually doing what it should. For the most part it's doing the same sort of statistical modeling of language for coding as it does for natural languages, but code is a lot more precise and nuanced and can't just be a popularity contest of most statistical likely outcomes. As a programmer, even a beginner one, you can usually spot these mistakes or at least ask it to explain or change some part of the generated code. However, for someone with zero coding experience, it might be harder to even spot these mistakes to begin with. You might be able to deploy the code but it might not actually do the thing you think it's doing, or maybe only do it under some circumstances but not others. To be fair, though, the same things can happen when you work with human programmers. There's a lot of back and forth there too and trial and error. And many of us aren't great communicators, so ChatGPT has an advantage there. If you're patient with it, it might be a nice long term partner. But it's not quite at the "make me a _____ app and deploy it" one-prompt stage yet. Probably soon! I have no doubt one day soon it will replace developers like me, but for the time being, it works more effectively when paired with a human (and vice versa).

mnky9800n 2y ago

you also have no way of verifying that the youtube video is telling you to do the right thing. even if it works, it still could not be up to code so suddenly you have expensive changes when you have to sell your house. same with chatgpt. when i ask it to do things i know a lot about, i am often time shocked how bad the code is that it generates and often does different stuff. makes me wonder about the code that i ask it to generate to do things i dont know about. haha.

FinnLobsien 2y ago
typical Gell-Mann amnesia. I write a lot so I see this when I've used AI to generate writing. If I ask it to generate something that I understand: "Wow this is the most surface level garbage anyone could create by reading one SEO-optimized article" When it's about something I don't know anything about: "Wow it's impressive how far AI has come" What I do find AI super useful for is finding stuff like metaphors. "I want to find a metaphor between product-led vs. sales-led growth strategies for startups. Please give me a list of 10 i could use" Those prompts are incredibly useful. I reckon there's some type of analogy with coding here.
atrettel 2y ago
I've done the same thing for the subject that I did my PhD in, and oh boy is the result weird. It is somewhat right at a high level but most of the low-level details are wrong at worst or confused at best. The explanation is incredibly shallow and misses so many important aspects that matter when implementing the idea in practice. It repeatedly refers to Mark Morkovin as "Nikolai Morkovin". And it even makes up a citation as a reference, though knowing this subject deeply I recognize that it seems to have combined two different references from 1962 and 1977 into a single nonexistent reference.

FinnLobsien 2y ago

Interesting. It sounds like there are two parts to this: a) Product sense becoming more important i.e. talking to users and figuring out what they want and how to build it rather than the pure ability of using code to make the computer do something b) Having the machine but no idea how to repair/augment it only gets you so far. Like you might be able to run a lawn care business but if your lawnmower breaks and you can't repair it your business is dead. Is that about right?

oxfordmale 2y ago
Yes, that is about right. It is also about knowing when to mow the lawn. Generally, you shouldn't mow the lawn when it is too wet, as it is too heavy on your mower. You also want to avoid mowing it when it has been too sunny, as the grass will turn brown. That requires experience beyond buying a mower.
solardev 2y ago
(Not the original poster) My experience with ChatGPT for coding has actually been the opposite: It was better at understanding and guessing underlying needs than the typical programmer I've worked with, who usually needed everything spelled out in technical bullet point requirements. Clients don't think like that (because they don't know how software works), but programmers need that (or need to come up with it on their own) to write the code. ChatGPT on the other hand seems pretty good at using its vast training data to guess at your intent and needs, as long as it's a relatively common use case. I would say that's one of its great strengths (its ability to combine your prompt and its vast natural language data to give you an answer that effectively summarizes what your good options are). It's a better product owner than most programmers, in other words. On the other hand, its coding ability isn't very impressive. It has the ability to generate code and it will run it and debug it if you ask it to, but it doesn't always do that by itself, and it's not great at doing "sanity tests" along the way to make sure each unit of code it's producing is actually doing what it should. For the most part it's doing the same sort of statistical modeling of language for coding as it does for natural languages, but code is a lot more precise and nuanced and can't just be a popularity contest of most statistical likely outcomes. As a programmer, even a beginner one, you can usually spot these mistakes or at least ask it to explain or change some part of the generated code. However, for someone with zero coding experience, it might be harder to even spot these mistakes to begin with. You might be able to deploy the code but it might not actually do the thing you think it's doing, or maybe only do it under some circumstances but not others. To be fair, though, the same things can happen when you work with human programmers. There's a lot of back and forth there too and trial and error. And many of us aren't great communicators, so ChatGPT has an advantage there. If you're patient with it, it might be a nice long term partner. But it's not quite at the "make me a _____ app and deploy it" one-prompt stage yet. Probably soon! I have no doubt one day soon it will replace developers like me, but for the time being, it works more effectively when paired with a human (and vice versa).