4 ms·

So err, this whole IT thing has been nice for the last decades. Maybe we have another decade until it can handle its own fingers...

notart666 4y ago
It really can't and it doesn't really work the way you think it does which is ironic that you'd think people on this site would be informed but I guess ycombinator has a financial conflict of interest in doing so. I wouldn't be surprised if they make posts on this topic with a very heavy bias like they do for a lot of their own investments. I remember being shown this software almost a decade ago in dealing with chat in online video gaming and it ended becoming a serious conversation at the ACM SIG CHI about it's use and abuse for spreading certain things. At the end of the day, we realized it was an arms race and the only way to win, was to simply not play. But of course these ideas were rejected for it went against the financial interests of the parties involved.
- lmarcos 4y ago
  The point is not what we (techies) think about the usefulness of ChatGPT. The point is that managers will definitely think ChatGPT is worth doing... Just like they think Jira/Scrum/Agile/etc. is worth our time. I definitely see management paying for ChatGPT.
  - urthor 4y ago
    It's also not about how good ChatGPT is now. How good will ChatGPT be in 5 years? It's scary to imagine.
    - ben_w 4y ago
      It’s also impossible to imagine correctly. Back in 2009, I was completely convinced that we’d be able to go into a car dealership and buy a brand new vehicle which had no steering wheel because the self-driving AI was just that good, within 10 years. Seemed reasonable at the time on the basis of the DARPA Grand Challenge results, but even 13 years later, it didn’t happen.
      - urthor 4y ago
        Yes, exactly. There's also mathematically excellent reasons why that happened. Self-driving cars are an impossibly complex problem. Statistics are statistics. Predicting the minority class correctly 99% of the time isn't good enough for autonomous driving. A car has to break for little Suzie 100% of the time. However, generating 1,000 lines of code for a CRUD app? That's 99% bug free? That's a helluva lot better than I can do. As with all things. The solution is watch what the domain experts do.
        sarchertech 4y ago
        The equivalent is closer to a CRUD app that serves 99% of requests correctly. Which is nowhere near good enough to use. But even if we do go with 99% bug free for the sake of argument, the usefulness depends on the type of bug. How harmful is it? How easy is it to detect? I had my wife (a physician) ask ChatGPT medical questions and it was almost always subtly but dangerously and confidently wrong. It looked fine to me but it took an expert to spot the flaws. And frequently it required specialist knowledge that a physician outside of my wife’s specialty wouldn’t even know to find the problems. If you need a senior engineer to read and understand every line of code this thing spits out I don’t see it as providing more than advanced autocomplete in real world use (which to be fair could be quite helpful). It frequently takes more time to read and really comprehend a junior engineers PR than it would have to just do it myself. The only reason I’m not is mentoring.
        urthor 4y ago
        While I am in total agreement with you for many domains. ...knowing my first employer out of college, they'd be VERY happy with 99%. "The end user will test the software" is very much the adage of the age. Cut priced, low quality.
      - FiberBundle 4y ago
        I think this is the crucial point, just as in all AI applications the way it deals with corner cases will decide its impact on the job market. And corner cases are usually where AI has consistently been performing badly. Just as a reminder, after we had spectacular results on ImageNet, highly respected AI researchers were predicting the end of the radiologist occupation. Turns out that even when some state-of-the-art CV classification algorithm is used on any kind of scan, you still need a radiologist to look at the image basically in the same way as before. If you write large-scale applications with the help of a system like ChatGPT you will still need to create accurate test coverage and an understanding of the problem that is essentially equivalent to that of the people writing the code themselves. Whether all of this would in the end actually lead to large enough productivity increases depends on how error-prone the AI generated code will be and given that it takes a lot more time to dig into unfamiliar codebases than those you've written yourself, I think it's anything but obvious that this will have a huge overall impact on the industry. But obviously I might be biased here, since I have a stake in the game.
    - nnoitra 4y ago
- urthor 4y ago
  While it's never going to replace the man behind the machine. It still seems highly likely that "stitching libraries together" development workflows in 10-15 years will involve large amounts of copy-editing the output of large language models. The trajectory of improvements, from GitHub Copilot to ChatGPT, is too steep.
  - bigDinosaur 4y ago
    Web development workflows honestly are often already at the stage of stitching together the output of large language models (Stack Overflow being the most well known such language model). I'm still surprised it pulls the salaries it does.
  - alchemist1e9 4y ago
    10-15 years? I think you mean next year!
  - ilaksh 4y ago
    In my opinion, the only way it takes 10 years to get there is if all progress stops within the next 30 days. Because it can literally almost do that stitching libraries together task now, if you give it a compiler and runtime environment and have it iterate on errors. Open AI has said they will release a big update before Christmas. This could include an API. And if we assume a text-only environment. But we already have the first text-to-video models, so we should assume that ChatGPT like systems will be built with multimodal models such that they would include information about UI interactions etc. in the near future. No reason to suppose that those advances would take ten years. We are seeing major improvements every 6-12 months.
- ben_w 4y ago
  Whatever you saw a decade ago, it definitely wasn’t this. I do recommend you play with it. But if you don’t feel like signing up with their free account, here’s a screen recording of me asking it some random general knowledge questions and instructing it to use a different language in the response each time: https://youtu.be/XX2rbcrXblk https://youtu.be/XX2rbcrXblk
  - astrange 4y ago
    > Whatever you saw a decade ago, it definitely wasn’t this. To be fair, the post might be from Schmidhuber, in which case it’s true he saw it 20 years ago. I saw it 10 years ago and it was called MegaHAL, wasn’t this good though.
    - ben_w 4y ago
      I had to look up MegaHal, apparently that was based at least in part on a hidden Markov Model. Using "it" in this way to refer to both that and GPT-family LLMs, or similarly saying "I remember being shown this software almost a decade ago" like the other commenter, is like saying "I remember seeing Kitty Hawk fly, and there's no way that can get someone across the Atlantic" when being presented with a chance to a free seat on the first flight of the Concord. (Actual human level intelligence in this analogy is a spaceship we still have not built, that's not where I'm going with this).
      - astrange 4y ago
        It’s not clear to me MegaHAL language models are less powerful than transformer models. The difference is the “largeness”, but that’s a hardware/dataset/budget detail.
- jhoelzel 4y ago
  Thank you for your rant but you are simply wrong. I do have a bachelor in psychology and am coding since 20 years. This is not a turn based game ran in dos. You value yourself way to highly and maybe its you who should update their knowledge base ;)
risyachka 4y ago
It has value only if you know what it is talking about, thus it can only assist. If you ask it about AWS - you need to already know AWS, same with business etc. And if you can't make your own elevator pitch - the odds that you will be able to run a successful company are little to none. Website builders and ready-to-use templates have been around for decades and they only increased the demand for devs. Even if GPT will be able to produce complete system from start to end - those systems will need support, fixes, etc. Sure you will need less engineers per system but the amount of systems will increase.
- skor 4y ago
  just to verify the code for an enterprise sized system is likely going to take longer than building it. Also consider how long it takes you to bake a feature into code you have never seen before to code that you wrote.
- brushfoot 4y ago
  To add to this - GPT is good at synthesizing existing information, but it's bad at combining it in truly novel ways. OP's startup idea is in a well-understood domain. There are lots of existing competitors offering basically that exact service. There are clear, obvious paths to building it. GPT can draw from blog posts and tutorials. On the other hand, it does poorly with significantly novel ideas. I tried it with one I've been working on that combines 3D printing and crafting in new ways. Its answers just regurgitated blog posts about existing combinations. It even ignored the grammar of my questions, which switched the usual direct object with the object of the preposition. Which makes sense, because it isn't doing critical thinking - it's pattern matching.
  - spaniard89277 4y ago
    This is my experience too. I've tried to do simple task like building small JS scripts and while it's doable, is like having a kid who needs oversight and can't clearly connect ideas. And coding is supposedly an area where it shines. GPT is waaay better than us humans at finding and summarizing information. Way way faster. But I've tried to write a blog post about some technical stuff and I finally gave up after many tries. It's good. It's frightening already. But it's still not "there". It's like having a extremely fast but bit dumb virtual assistant. Way better than anything we had before.

notart666 4y ago

It really can't and it doesn't really work the way you think it does which is ironic that you'd think people on this site would be informed but I guess ycombinator has a financial conflict of interest in doing so. I wouldn't be surprised if they make posts on this topic with a very heavy bias like they do for a lot of their own investments. I remember being shown this software almost a decade ago in dealing with chat in online video gaming and it ended becoming a serious conversation at the ACM SIG CHI about it's use and abuse for spreading certain things. At the end of the day, we realized it was an arms race and the only way to win, was to simply not play. But of course these ideas were rejected for it went against the financial interests of the parties involved.

lmarcos 4y ago
The point is not what we (techies) think about the usefulness of ChatGPT. The point is that managers will definitely think ChatGPT is worth doing... Just like they think Jira/Scrum/Agile/etc. is worth our time. I definitely see management paying for ChatGPT.
- urthor 4y ago
  It's also not about how good ChatGPT is now. How good will ChatGPT be in 5 years? It's scary to imagine.
  - ben_w 4y ago
    It’s also impossible to imagine correctly. Back in 2009, I was completely convinced that we’d be able to go into a car dealership and buy a brand new vehicle which had no steering wheel because the self-driving AI was just that good, within 10 years. Seemed reasonable at the time on the basis of the DARPA Grand Challenge results, but even 13 years later, it didn’t happen.
    - urthor 4y ago
      Yes, exactly. There's also mathematically excellent reasons why that happened. Self-driving cars are an impossibly complex problem. Statistics are statistics. Predicting the minority class correctly 99% of the time isn't good enough for autonomous driving. A car has to break for little Suzie 100% of the time. However, generating 1,000 lines of code for a CRUD app? That's 99% bug free? That's a helluva lot better than I can do. As with all things. The solution is watch what the domain experts do.
      - sarchertech 4y ago
        The equivalent is closer to a CRUD app that serves 99% of requests correctly. Which is nowhere near good enough to use. But even if we do go with 99% bug free for the sake of argument, the usefulness depends on the type of bug. How harmful is it? How easy is it to detect? I had my wife (a physician) ask ChatGPT medical questions and it was almost always subtly but dangerously and confidently wrong. It looked fine to me but it took an expert to spot the flaws. And frequently it required specialist knowledge that a physician outside of my wife’s specialty wouldn’t even know to find the problems. If you need a senior engineer to read and understand every line of code this thing spits out I don’t see it as providing more than advanced autocomplete in real world use (which to be fair could be quite helpful). It frequently takes more time to read and really comprehend a junior engineers PR than it would have to just do it myself. The only reason I’m not is mentoring.
        urthor 4y ago
        While I am in total agreement with you for many domains. ...knowing my first employer out of college, they'd be VERY happy with 99%. "The end user will test the software" is very much the adage of the age. Cut priced, low quality.
    - FiberBundle 4y ago
      I think this is the crucial point, just as in all AI applications the way it deals with corner cases will decide its impact on the job market. And corner cases are usually where AI has consistently been performing badly. Just as a reminder, after we had spectacular results on ImageNet, highly respected AI researchers were predicting the end of the radiologist occupation. Turns out that even when some state-of-the-art CV classification algorithm is used on any kind of scan, you still need a radiologist to look at the image basically in the same way as before. If you write large-scale applications with the help of a system like ChatGPT you will still need to create accurate test coverage and an understanding of the problem that is essentially equivalent to that of the people writing the code themselves. Whether all of this would in the end actually lead to large enough productivity increases depends on how error-prone the AI generated code will be and given that it takes a lot more time to dig into unfamiliar codebases than those you've written yourself, I think it's anything but obvious that this will have a huge overall impact on the industry. But obviously I might be biased here, since I have a stake in the game.
  - nnoitra 4y ago
urthor 4y ago
While it's never going to replace the man behind the machine. It still seems highly likely that "stitching libraries together" development workflows in 10-15 years will involve large amounts of copy-editing the output of large language models. The trajectory of improvements, from GitHub Copilot to ChatGPT, is too steep.
- bigDinosaur 4y ago
  Web development workflows honestly are often already at the stage of stitching together the output of large language models (Stack Overflow being the most well known such language model). I'm still surprised it pulls the salaries it does.
- alchemist1e9 4y ago
  10-15 years? I think you mean next year!
- ilaksh 4y ago
  In my opinion, the only way it takes 10 years to get there is if all progress stops within the next 30 days. Because it can literally almost do that stitching libraries together task now, if you give it a compiler and runtime environment and have it iterate on errors. Open AI has said they will release a big update before Christmas. This could include an API. And if we assume a text-only environment. But we already have the first text-to-video models, so we should assume that ChatGPT like systems will be built with multimodal models such that they would include information about UI interactions etc. in the near future. No reason to suppose that those advances would take ten years. We are seeing major improvements every 6-12 months.
ben_w 4y ago
Whatever you saw a decade ago, it definitely wasn’t this. I do recommend you play with it. But if you don’t feel like signing up with their free account, here’s a screen recording of me asking it some random general knowledge questions and instructing it to use a different language in the response each time: https://youtu.be/XX2rbcrXblk https://youtu.be/XX2rbcrXblk
- astrange 4y ago
  > Whatever you saw a decade ago, it definitely wasn’t this. To be fair, the post might be from Schmidhuber, in which case it’s true he saw it 20 years ago. I saw it 10 years ago and it was called MegaHAL, wasn’t this good though.
  - ben_w 4y ago
    I had to look up MegaHal, apparently that was based at least in part on a hidden Markov Model. Using "it" in this way to refer to both that and GPT-family LLMs, or similarly saying "I remember being shown this software almost a decade ago" like the other commenter, is like saying "I remember seeing Kitty Hawk fly, and there's no way that can get someone across the Atlantic" when being presented with a chance to a free seat on the first flight of the Concord. (Actual human level intelligence in this analogy is a spaceship we still have not built, that's not where I'm going with this).
    - astrange 4y ago
      It’s not clear to me MegaHAL language models are less powerful than transformer models. The difference is the “largeness”, but that’s a hardware/dataset/budget detail.
jhoelzel 4y ago
Thank you for your rant but you are simply wrong. I do have a bachelor in psychology and am coding since 20 years. This is not a turn based game ran in dos. You value yourself way to highly and maybe its you who should update their knowledge base ;)

risyachka 4y ago

It has value only if you know what it is talking about, thus it can only assist. If you ask it about AWS - you need to already know AWS, same with business etc. And if you can't make your own elevator pitch - the odds that you will be able to run a successful company are little to none. Website builders and ready-to-use templates have been around for decades and they only increased the demand for devs. Even if GPT will be able to produce complete system from start to end - those systems will need support, fixes, etc. Sure you will need less engineers per system but the amount of systems will increase.

skor 4y ago
just to verify the code for an enterprise sized system is likely going to take longer than building it. Also consider how long it takes you to bake a feature into code you have never seen before to code that you wrote.
brushfoot 4y ago
To add to this - GPT is good at synthesizing existing information, but it's bad at combining it in truly novel ways. OP's startup idea is in a well-understood domain. There are lots of existing competitors offering basically that exact service. There are clear, obvious paths to building it. GPT can draw from blog posts and tutorials. On the other hand, it does poorly with significantly novel ideas. I tried it with one I've been working on that combines 3D printing and crafting in new ways. Its answers just regurgitated blog posts about existing combinations. It even ignored the grammar of my questions, which switched the usual direct object with the object of the preposition. Which makes sense, because it isn't doing critical thinking - it's pattern matching.
- spaniard89277 4y ago
  This is my experience too. I've tried to do simple task like building small JS scripts and while it's doable, is like having a kid who needs oversight and can't clearly connect ideas. And coding is supposedly an area where it shines. GPT is waaay better than us humans at finding and summarizing information. Way way faster. But I've tried to write a blog post about some technical stuff and I finally gave up after many tries. It's good. It's frightening already. But it's still not "there". It's like having a extremely fast but bit dumb virtual assistant. Way better than anything we had before.