5 ms·

I'm a PM who is relatively technical, but I haven't written more than a few lines of code (stuff like minor modifications to my Shopify site) in a decade. I sa

I'm a PM who is relatively technical, but I haven't written more than a few lines of code (stuff like minor modifications to my Shopify site) in a decade. I saw a bunch of people talking about how GPT helped them code stuff on Twitter, so I thought I'd give it a try. Right now I'm building a sort of simple, mock version of the type of software that integrates with my company's APIs. I've successfully managed to create a simple web application that creates a new object, hits my company's API endpoint to create a corresponding object on our software, allows me to upload a document locally and then allows me to upload that document to our software via API as well. It's all a little messy and clearly not production-ready, but it works. It would've taken me probably a few months on nights and weekends do this (mostly refreshing myself on JS and Python). Instead I've done it in <24h (would've been shorter except for GPT-4's message limit). I'm sometimes able to spot and fix GPT's bugs, but even when I'm not, it walks me through adding more logging and successfully debugs issues. Sometimes it takes a few tries and a little direction as to what I suspect the issue is, but so far it's fixed everything that's come up. I don't think this would be doable for a totally non-technical person, but I do think it'll get there pretty soon. I'm just absolutely blown away.

devjab 4y ago
One of the reasons I'm not blown away is that everything we've tasked it to do has resulted in rather terrible answers. A lot of them outright didn't work. When they did, the way GPT created our solutions were often in a way that wouldn't work well over time. Unfortunately you wouldn't necessarily know that unless you really know your tools. In many ways, this isn't too different from people performing brute-force programming through their favorite search engine, but at least most people know that "Google programming" is sort of bad. I think we're going to be cleaning up the GPT messes for decades to come because it's very confidently incorrect and much more accessible as you point out. I think we're going to see a lot of programmers who are going to trust GPT a little too much, and I think that's sort of scary. For the most part that is going to work out just fine. Often the quality of your programming isn't actually going to matter that much, because as long as it solves the business needs okish, then it's frankly great. That's not always the case, however, imagine someone using GPT to get your healthcare software wrong. I'm still impressed with it in other areas. I think it'll do wonders in the world of office automation because it seems to have the ability to succeed at this much better than any previous "no-code" attempt where the logic would almost always end up requiring people who are basically programmers for it to work. I think GPT will help here, requiring less "superusers" for a department to move their data flows into automation. Especially in areas, where efficiency and stability aren't necessarily that important if the automation-tools mean you don't need three full time employees moving data from one system to another.
- dwaltrip 4y ago
  Have you tried gpt-4? It's not perfect but it is a clear improvement for code.
- sharemywin 4y ago
  Are people actually allowing it to perform actual software development? State your assumptions. Read and summarize the pervious documents generate a data flow diagram. generate a data model. Get it to inquire about use cases and requirements generate tests for these uses cases and requirements.
  - Cyphase 4y ago
    I've played with getting it to ask me about my requirements. I think that was on ChatGPT3.5? It wasn't great, but it wasn't horrible.
- tablespoon 4y ago
  > ...but at least most people know that "Google programming" is sort of bad. I think we're going to be cleaning up the GPT messes for decades to come because it's very confidently incorrect and much more accessible as you point out. Speaking of which: https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned https://meta.stackoverflow.com/questions/421831/temporary-po... > Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers. > The primary problem is that while the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce. There are also many people trying out ChatGPT to create answers, without the expertise or willingness to verify that the answer is correct prior to posting. Because such answers are so easy to produce, a large number of people are posting a lot of answers. The volume of these answers (thousands) and the fact that the answers often require a detailed read by someone with at least some subject matter expertise in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure. If we're lucky ChatGPT will poison itself by pissing in its well, but it will take a lot of good things with it.
majormajor 4y ago
It'll definitely help people learn faster - the "what to try next" problem is HUGE in programming discovery. Modern languages (and tools like autocomplete) have already helped that a lot compared to assembly code or binary, this looks like the biggest jump in a long time. The path of programming so far has been moving from "describe how to do something" to "describe what to do" which this is certainly in line with.
helf 4y ago
Honestly, it seems the people most blown away by things like this are people who haven’t much experience in the various fields. It’s easy to get amazed by something that can halfway do something you can’t do at all automatically. But as others have pointed out, it’s not that great at it and not knowing enough to do it yourself means you don’t know enough to catch and fix bugs. So this move to using chatgpt and similar in production by people who otherwise wouldn’t be able to do things in production is worrisome, imo.

devjab 4y ago

One of the reasons I'm not blown away is that everything we've tasked it to do has resulted in rather terrible answers. A lot of them outright didn't work. When they did, the way GPT created our solutions were often in a way that wouldn't work well over time. Unfortunately you wouldn't necessarily know that unless you really know your tools. In many ways, this isn't too different from people performing brute-force programming through their favorite search engine, but at least most people know that "Google programming" is sort of bad. I think we're going to be cleaning up the GPT messes for decades to come because it's very confidently incorrect and much more accessible as you point out. I think we're going to see a lot of programmers who are going to trust GPT a little too much, and I think that's sort of scary. For the most part that is going to work out just fine. Often the quality of your programming isn't actually going to matter that much, because as long as it solves the business needs okish, then it's frankly great. That's not always the case, however, imagine someone using GPT to get your healthcare software wrong. I'm still impressed with it in other areas. I think it'll do wonders in the world of office automation because it seems to have the ability to succeed at this much better than any previous "no-code" attempt where the logic would almost always end up requiring people who are basically programmers for it to work. I think GPT will help here, requiring less "superusers" for a department to move their data flows into automation. Especially in areas, where efficiency and stability aren't necessarily that important if the automation-tools mean you don't need three full time employees moving data from one system to another.

dwaltrip 4y ago
Have you tried gpt-4? It's not perfect but it is a clear improvement for code.
sharemywin 4y ago
Are people actually allowing it to perform actual software development? State your assumptions. Read and summarize the pervious documents generate a data flow diagram. generate a data model. Get it to inquire about use cases and requirements generate tests for these uses cases and requirements.
- Cyphase 4y ago
  I've played with getting it to ask me about my requirements. I think that was on ChatGPT3.5? It wasn't great, but it wasn't horrible.
tablespoon 4y ago
> ...but at least most people know that "Google programming" is sort of bad. I think we're going to be cleaning up the GPT messes for decades to come because it's very confidently incorrect and much more accessible as you point out. Speaking of which: https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned https://meta.stackoverflow.com/questions/421831/temporary-po... > Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers. > The primary problem is that while the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce. There are also many people trying out ChatGPT to create answers, without the expertise or willingness to verify that the answer is correct prior to posting. Because such answers are so easy to produce, a large number of people are posting a lot of answers. The volume of these answers (thousands) and the fact that the answers often require a detailed read by someone with at least some subject matter expertise in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure. If we're lucky ChatGPT will poison itself by pissing in its well, but it will take a lot of good things with it.

majormajor 4y ago

It'll definitely help people learn faster - the "what to try next" problem is HUGE in programming discovery. Modern languages (and tools like autocomplete) have already helped that a lot compared to assembly code or binary, this looks like the biggest jump in a long time. The path of programming so far has been moving from "describe how to do something" to "describe what to do" which this is certainly in line with.

helf 4y ago

Honestly, it seems the people most blown away by things like this are people who haven’t much experience in the various fields. It’s easy to get amazed by something that can halfway do something you can’t do at all automatically. But as others have pointed out, it’s not that great at it and not knowing enough to do it yourself means you don’t know enough to catch and fix bugs. So this move to using chatgpt and similar in production by people who otherwise wouldn’t be able to do things in production is worrisome, imo.