4 ms·

It is strange that this took 5 days to find. Simply because of logs. Go to logs. Filter by errors. Oh, errors in insert subscription. Seems relevant. I could

It is strange that this took 5 days to find. Simply because of logs. Go to logs. Filter by errors. Oh, errors in insert subscription. Seems relevant. I could understand if the errors were somewhere else. Even if logs didn't exist. Problematic endpoint generating 50 emails per day? I would have immediately thrown a try catch and rendered the error to the user if logging was impossible. Then your very next bug report solves it. Assuming that they had the error (guid collision) - it's not as easy to spot as some commentators are making out. But surely after reading the code s few times. Ironically they should have asked ChatGPT for help debugging

wmedrano 2y ago
ChatGPT gives the following: The code snippet has a subtle but significant issue in the default value of the id column. Here is the problematic part: ... In this line, default=str(uuid.uuid4()) is evaluated only once at the time of the class definition, not each time a new StripeCustomer instance is created....
- sitkack 2y ago
  They should have also asked the LLM for some integration tests. Ensure that the revenue generating codepaths have proper logging. This failure had very little to do with having an LLM write it.
- SamBam 2y ago
  Presumably that's easy enough given just the snippet in question, but they hadn't narrowed the issue down to that snippet because they had no idea it was happening on the user creation, because they didn't have the logs. I expect ChatGPT wouldn't have been able to solve the issue given the entire codebase.
latexr 2y ago
> Ironically they should have asked ChatGPT for help debugging Why are you assuming they didn’t? This is an AI company using AI to build their product and trusting it without proper code review, testing, or guardrails. Clearly they’re all in on AI hype. This took them days to solve, so not only would I bet they asked ChatGPT, I’d wager multiple people tried it multiple times.

wmedrano 2y ago

ChatGPT gives the following: The code snippet has a subtle but significant issue in the default value of the id column. Here is the problematic part: ... In this line, default=str(uuid.uuid4()) is evaluated only once at the time of the class definition, not each time a new StripeCustomer instance is created....

sitkack 2y ago
They should have also asked the LLM for some integration tests. Ensure that the revenue generating codepaths have proper logging. This failure had very little to do with having an LLM write it.
SamBam 2y ago
Presumably that's easy enough given just the snippet in question, but they hadn't narrowed the issue down to that snippet because they had no idea it was happening on the user creation, because they didn't have the logs. I expect ChatGPT wouldn't have been able to solve the issue given the entire codebase.

latexr 2y ago

> Ironically they should have asked ChatGPT for help debugging Why are you assuming they didn’t? This is an AI company using AI to build their product and trusting it without proper code review, testing, or guardrails. Clearly they’re all in on AI hype. This took them days to solve, so not only would I bet they asked ChatGPT, I’d wager multiple people tried it multiple times.