4 ms·

Honestly I'm not sure why ChatGPT has anything to do with this problem. I remember making the exact same mistake (accidentally using a single function call in

Honestly I'm not sure why ChatGPT has anything to do with this problem. I remember making the exact same mistake (accidentally using a single function call in a schema) back in 2010. No LLMs required. The bigger culprit is probably a lack of testing / debugging. This error would immediately get caught if you simply registered twice on a test instance.

fzeroracer 2y ago
In a world where this entire codebase wasn't generated by ChatGPT, you'd have engineers familiar with the various parts of the system to quickly identify and fix the problem. Testing and debugging isn't just a matter of stepping through code, it's an exercise of seeing where your mental model of the codebase is faulty versus the current reality of it. I've encountered similar problems and they'd be fixed in a matter of hours, not days.
- moody__ 2y ago
  This is spot on, the issue is not the mistake per se but in creating a code base that the team themselves are not familiar with. With some intern or team member generated code you can sit down with them and have them walk you through the code and introduce you to their reasoning, but you can't do that with an LLM. The author even admits to just mimicking the existing structure that the LLM started with when they had to expand, which sounds like a first commit for a team member first getting familiar with some new code. Part of the benefit of being able to write your own code is that you can do it in a way that clicks for you. Hopefully this lets you debug and extend it efficiently. I don't know why someone would squander this opportunity.
- audiodude 2y ago
  Yes I don't understand the perceived benefits of re-writing the code in Python/FastAPI if none of them know Python/FastAPI.
  - chrisjj 2y ago
    "Requirements"?
- chrisjj 2y ago
  > In a world where this entire codebase wasn't generated by ChatGPT, you'd have engineers familiar with the various parts of the system to quickly identify and fix the problem. I wonder why this victim didn't ask ChatGPT to identify and fix the problem...
dragonwriter 2y ago
> Honestly I'm not sure why ChatGPT has anything to do with this problem. Because if humans had written and reviewed the code, multiple team members would have had to have learned Python, and SQLAlchemy specifically, which, even if the mistake was initially made as many times as ChatGPT did, there would have been multiple independent opportunities for it to be caught and questioned and the relevant knowledge shared during development. ChatGPT may be able to crank out immense volumes of superficially functional code, but if its your only “teammate” that understands the libraries used and touches the code, its a huge single point of failure.
yen223 2y ago
> This error would immediately get caught if you simply registered twice on a test instance Friendly reminder: check if your codebase is actually testing this! One of the interesting consequence of running unit tests with a fresh database everytime is that problems related to unique constraints seldom get caught by unit tests.
- chrisjj 2y ago
  But why should we assume twice is enough...
chrisjj 2y ago
> Honestly I'm not sure why ChatGPT has anything to do with this problem. I remember making the exact same mistake (accidentally using a single function call in a schema) back in 2010. No LLMs required. So /that's/ where ChatGPT learned it! :)

fzeroracer 2y ago

In a world where this entire codebase wasn't generated by ChatGPT, you'd have engineers familiar with the various parts of the system to quickly identify and fix the problem. Testing and debugging isn't just a matter of stepping through code, it's an exercise of seeing where your mental model of the codebase is faulty versus the current reality of it. I've encountered similar problems and they'd be fixed in a matter of hours, not days.

moody__ 2y ago
This is spot on, the issue is not the mistake per se but in creating a code base that the team themselves are not familiar with. With some intern or team member generated code you can sit down with them and have them walk you through the code and introduce you to their reasoning, but you can't do that with an LLM. The author even admits to just mimicking the existing structure that the LLM started with when they had to expand, which sounds like a first commit for a team member first getting familiar with some new code. Part of the benefit of being able to write your own code is that you can do it in a way that clicks for you. Hopefully this lets you debug and extend it efficiently. I don't know why someone would squander this opportunity.
audiodude 2y ago
Yes I don't understand the perceived benefits of re-writing the code in Python/FastAPI if none of them know Python/FastAPI.
- chrisjj 2y ago
  "Requirements"?
chrisjj 2y ago
> In a world where this entire codebase wasn't generated by ChatGPT, you'd have engineers familiar with the various parts of the system to quickly identify and fix the problem. I wonder why this victim didn't ask ChatGPT to identify and fix the problem...

dragonwriter 2y ago

> Honestly I'm not sure why ChatGPT has anything to do with this problem. Because if humans had written and reviewed the code, multiple team members would have had to have learned Python, and SQLAlchemy specifically, which, even if the mistake was initially made as many times as ChatGPT did, there would have been multiple independent opportunities for it to be caught and questioned and the relevant knowledge shared during development. ChatGPT may be able to crank out immense volumes of superficially functional code, but if its your only “teammate” that understands the libraries used and touches the code, its a huge single point of failure.

yen223 2y ago

> This error would immediately get caught if you simply registered twice on a test instance Friendly reminder: check if your codebase is actually testing this! One of the interesting consequence of running unit tests with a fresh database everytime is that problems related to unique constraints seldom get caught by unit tests.

chrisjj 2y ago
But why should we assume twice is enough...

chrisjj 2y ago

> Honestly I'm not sure why ChatGPT has anything to do with this problem. I remember making the exact same mistake (accidentally using a single function call in a schema) back in 2010. No LLMs required. So /that's/ where ChatGPT learned it! :)