4 ms·

Ask HN: Have people started to see cracks in the ChatGPT answers with time

3 points by sandeepkd 1y ago

Context: I have a co-worker who is used to execute the suggested scripts from Chat-gpt with high confidence. Lately one of the scripts to use Github API was not working and he asked me to debug. Turns out, Github had changed/updated the attribute names in the output and they no longer matched with the Chat-Gpt provided script. Hard part, this individual is really good engineer, however he/she was low on confidence about how to debug the Chat-gpt provided script

speedgoose 1y ago
Skill issue. First, switch to Codex. Then, consider using the context7 mcp server to let agents automatically fetch the latest documentation. Debugging is not so different than debugging code written by someone else. It’s a skill you can practice. By the way, agents are also good at debugging.
- techblueberry 1y ago
  This is a great advertisement: Agents: harder, slower and more expensive than just doing it yourself.
  - speedgoose 1y ago
    It depends on the task.
  - pavelai 1y ago
    It's a very overgeneralized statement. Agents could do a lot of work, which most of people wouldn't want to do. and agents can do this on a pretty good level for now. Probably we would enhance their ability to do this in near future. People should stop write code and start to do real life things.
    - techblueberry 1y ago
      What you are failing to understand, is that this is literally the worst these models will ever be. Uber wasn’t profitable for like the first 20 years of their existence, and now look at them. People said the same things about the internet.
      - raw_anon_1111 1y ago
        And thousands of other companies weren’t profitable and disappeared into obscurity and failed, using Uber as proof of anything is the ultimate in survivorship bias.
        pavelai 1y ago
        There a lot of examples of things been critisized in the beginning and strived, there are a lot of things which have gone. It only means that there could be only reasonable critics. Can you say if your criticism is reasonable? I saw a lot of things being thrashed in the beginning. And these things people use everyday. You can choose from thousands of examples First plane didn't have cockpit, and windows, could fly only hundreds of meters, and looked more like a clothes dryer than a plane. Someone could call it inefficient (and really it wasn't efficient). It was easy to call it dangerous. But as we can see, planes evolved since then. When the first radio was invented there was no radio station to listen. When the steam train was invented it was crazy expensive and there was no government which has enough money to build all the railroads we have today All of these technologies were modified by talented people, some of them were inventors, others later adopters. But they changed how this technologies work, and what goods they brought to us AI is too young technology to tell if it fail or not. You can critisize it or you can change the direction of the progress. It's up to you
        raw_anon_1111 1y ago
        And if you can point to thousands, then I can literally 100x for all the things that failed. You only have to go back 25 years in tech to see all of the failures - or YC companies that IPO’d more recently that collectively have a -49% return while the broader market has seen record gains https://medium.com/@Arakunrin/the-post-ipo-performance-of-y-combinator-companies-a-mixed-bag-of-success-and-struggle-58e9a4006f01 https://medium.com/@Arakunrin/the-post-ipo-performance-of-y-... It ends up for instance that Steve Jobs was right about Dropbox - it is a feature not a product.
        pavelai 11mo ago
almosthere 1y ago
Don't use chatgpt for that kind of work, use agents. It will read the error message and adapt, perhaps even go as far as open the release notes for the tools you're using and fix the problems it encounters.
bjourne 1y ago
IMnsHO, nope! People trust the bot way more than their own experience.

3 points by sandeepkd 1y ago

speedgoose 1y ago

Skill issue. First, switch to Codex. Then, consider using the context7 mcp server to let agents automatically fetch the latest documentation. Debugging is not so different than debugging code written by someone else. It’s a skill you can practice. By the way, agents are also good at debugging.

techblueberry 1y ago
This is a great advertisement: Agents: harder, slower and more expensive than just doing it yourself.
- speedgoose 1y ago
  It depends on the task.
- pavelai 1y ago
  It's a very overgeneralized statement. Agents could do a lot of work, which most of people wouldn't want to do. and agents can do this on a pretty good level for now. Probably we would enhance their ability to do this in near future. People should stop write code and start to do real life things.
  - techblueberry 1y ago
    What you are failing to understand, is that this is literally the worst these models will ever be. Uber wasn’t profitable for like the first 20 years of their existence, and now look at them. People said the same things about the internet.
    - raw_anon_1111 1y ago
      And thousands of other companies weren’t profitable and disappeared into obscurity and failed, using Uber as proof of anything is the ultimate in survivorship bias.
      - pavelai 1y ago
        There a lot of examples of things been critisized in the beginning and strived, there are a lot of things which have gone. It only means that there could be only reasonable critics. Can you say if your criticism is reasonable? I saw a lot of things being thrashed in the beginning. And these things people use everyday. You can choose from thousands of examples First plane didn't have cockpit, and windows, could fly only hundreds of meters, and looked more like a clothes dryer than a plane. Someone could call it inefficient (and really it wasn't efficient). It was easy to call it dangerous. But as we can see, planes evolved since then. When the first radio was invented there was no radio station to listen. When the steam train was invented it was crazy expensive and there was no government which has enough money to build all the railroads we have today All of these technologies were modified by talented people, some of them were inventors, others later adopters. But they changed how this technologies work, and what goods they brought to us AI is too young technology to tell if it fail or not. You can critisize it or you can change the direction of the progress. It's up to you
        raw_anon_1111 1y ago
        And if you can point to thousands, then I can literally 100x for all the things that failed. You only have to go back 25 years in tech to see all of the failures - or YC companies that IPO’d more recently that collectively have a -49% return while the broader market has seen record gains https://medium.com/@Arakunrin/the-post-ipo-performance-of-y-combinator-companies-a-mixed-bag-of-success-and-struggle-58e9a4006f01 https://medium.com/@Arakunrin/the-post-ipo-performance-of-y-... It ends up for instance that Steve Jobs was right about Dropbox - it is a feature not a product.
        pavelai 11mo ago

almosthere 1y ago

Don't use chatgpt for that kind of work, use agents. It will read the error message and adapt, perhaps even go as far as open the release notes for the tools you're using and fix the problems it encounters.

bjourne 1y ago

IMnsHO, nope! People trust the bot way more than their own experience.