8 ms·
On one hand, thanks for being honest about a story of how this bug came to be. On the other hand, I don’t think advertising the fact that the company introduce
by Aurornis 2y ago
On one hand, thanks for being honest about a story of how this bug came to be.
On the other hand, I don’t think advertising the fact that the company introduced a major bug from copy and pasting ChatGPT code around and that they spent a week being unable to even debug why it was failing.
I don’t know much about this startup, but this blog post had the opposite effect of all of the other high quality post-mortem posts I’ve read lately. Normally the goal of these posts is to demonstrate your team’s rigor and engineering skills, not reveal that ChatGPT is writing your code and your engineers can’t/won’t debug it for a very long time despite knowing that it’s costing them signups.
- userbinator 2y agoIt read like no one really knew what they were doing. "We just let it generate the code and everything seemed to work" is certainly not a good way to market your company.
- Terretta 2y ago> It read like no one really knew what they were doing. "We just let [devs] generate the code and everything seemed to work" is certainly not a good way to [whatever]. Except, have you met startup devs? This is by and large the "move fast then unbreak things" approach.
- masijo 2y agoThis is why working for startups gives me PTSD. I wouldn't recommend it to anyone.
- deleted 2y ago[deleted]
- almost_usual 2y agoThe idea of inheriting a ChatGPT code base no one understands now makes it worse.
- hehdhdjehehegwv 2y agoJust give it to GPT-5 for a refactor, easy!
- pclmulqdq 2y agoIt terrifies me that I have heard people say this unironically.
- hehdhdjehehegwv 2y agoIt’s the natural outcome of SV types denigrating the value of education. Forget knowing anything, just come up with a nice pitch deck and let the LLM write the stack. Not wholly surprised these people are YC backed. I’ve got the impression YC don’t place much weight on technical competence, assuming you can just hire the talent if you know how to sell. Well, now replace “hire some talent” with “get a GPT subscription and YOLO”, and you get the foundation these companies of tomorrow are going to be built on. Which hey, maybe that’s right and they know something I don’t.
- SilasX 2y agoFor that matter, has OP even met the HN accepted wisdom? "No one knows what they're doing, everyone's faking it, it's fine if you are too" -- so don't take it as a red flag when your fumbling around keeps blowing up, because it surely must work that way everywhere else. My early rant against this mentality: https://news.ycombinator.com/item?id=19214749 https://news.ycombinator.com/item?id=19214749
- tmpz22 2y agoIts very humbling coming out of startup-land and working with big tech engineers and realizing their tooling runs circles around everybody else and enables them to be much more precise with their work and scale, though it isn't without trade-offs.
- sanderjd 2y agoYeah but a lot of that is just the accrual of improvements that is possible with a lot of resources over a long period of time. People working in "big tech" aren't fundamentally better at building reliable tools and systems; the time and resource constraints are entirely different.
- crabmusket 2y agoAnd the stakes! This outage might have cost the OP $10k. A similar snafu at a larger company might have cost tens of millions or more.
- fragmede 2y agoThe big tech tooling probably cost tens of millions of dollars to create, and probably had a couple $10k mistakes on the way to getting it written and running.
- deleted 2y ago[deleted]
- ruszki 2y agoIt’s possible to move fast the same way, but break less things than this. For example, in this case, they said that they introduced tests to mitigate this. I can assure you that introducing tests takes more time than Google searches to check in like 2 minutes what each lines really does.
- Kesty 2y agoThe idea of moving fast is to have extensive logs and alerts so you fix all error fasts while they appear without "wasting time" with long expensive tests in a phase where things change every day. 5 days to find out you have "duplicate key" errors in the db is the opposite of fast
- mort96 2y agoEh I imagine they looked over the code as well, doing code review -- and at first glance, the code looks reasonable. I certainly wasn't able to catch the bug even though I tried to find it (and I was given a tiny collection of lines and the knowledge that there's a bug there!). If anything, I think this says something about how dangerous ChatGPT and similar tools are: reading code is harder than writing code, and when you use ChatGPT, your role stops being that of a programmer and becomes that of a code reviewer. Worse, LLMs are excellent at producing plausible output (I mean that's literally all they do), which means the bugs will look like plausibly correct code as well. I don't think this is indicative of people who don't know what they're doing. I think this is indicative of people using "AI" tools to help with programming at all.
- throw46365 2y ago> I don't think this is indicative of people who don't know what they're doing. I think this is indicative of people using "AI" tools to help with programming at all. I think using AI tools to write production code is probably indicative of people who don't really know what they are doing. The best way not to have subtle bugs is to think deeply about your code, not subcontract it out -- whether that is to people far away who both cannot afford to think as deeply about your code and aren't as invested in it, or to an AI that is often right and doesn't know the difference between correct and incorrect. It's just a profound abrogation of good development principles to behave this way. And where is the benefit in doing this repeatedly? You're just going to end up with a codebase nobody really owns on a cognitive level. At least when you look at a StackOverflow answer you see the discussion around it from other real people offering critiques! ETA in advance: and yes, I understand all the comparison points about using third party libraries, and all the left-pad stuff (don't get me started on NPM). But the point stands: the best way not to have bugs is to own your code. To my mind, anyone who is using ChatGPT in this way -- to write whole pieces of business logic, not just to get inspiration -- is failing at their one job. If it's to be yours, it has to come from the brain of someone who is yours too. This is an embarrassing and damaging admission and there is no way around it. ETA (2): code review, as a practice, only works when you and the people who wrote the code have a shared understanding of the context and the goal of the code and are roughly equally invested in getting code through review. Because all the niche cases are illuminated by those discussions and avoided in advance. The less time you've spent on this preamble, the less effective the code review will be. It's a matter of trust and culture as much as it's a matter of comparing requirements with finished code.
- PhasmaFelis 2y ago> It read like no one really knew what they were doing. In my experience, hardly anyone in software does know what they're doing, for sufficiently rigorous values of "know what you're doing." We all read about other people's stupid mistakes, and think "haha, I would never have done that, because I know about XYZ!" And then we go off and happily make some equally stupid mistake because we don't know about ABC.
- zeroonetwothree 2y agoThere’s a difference between not knowing what you are doing and making a mistake.
- PhasmaFelis 2y agoAn awful lot of mistakes are made because one didn't know something that would have enabled one to avoid it. Not knowing what you don't know is difficult to work around.
- deleted 2y ago[deleted]
- throw46365 2y agoThe "zen" of LLMs is that they do not see a real distinction between these two things, or either of these two things and success ;-)
- throw46365 2y agoI dunno. I tend to annoy people when taking on jobs by telling people what I am concerned about and do not understand, and then sharing with them the extent to which I have managed to allay my own concerns through research. I turn down a lot of jobs I don't feel confident with; maybe more than I should. An LLM never will.
- bogota 2y agoThis is getting more common. I have already had people try to tell me how something works from a chat gpt summary. This would have led to us taking a completely different direction… 5 minutes of reading the actual docs and I found out they were wrong. Now at a new company i have caught several people copy pasting gpt code that is just horrendous. It seems like this is where the industry is headed. The only thing i have found gpt to be good at is solving interview questions although it still uses phantom functions about 50% of the time. The future is bumming me out.
- droopyEyelids 2y agoIt’sa Tower of Babel like effect
- gravescale 2y agoLike people who post "here's what ChatGPTx said" instead of their own answer. Quite literally, what is the point? However, I don't think it's really bad for the technical industries long term. It probably does mean that some companies with loose internal quality control and enough shiftless employees pasting enough GPT spew without oversight will go to the wall because their software became unmaintainable and not useful, but this already happens. It's probably not hugely worse than the flood of bootcamp victims who wrote fizzbuzz in Python, get recruited by a credulous, cheap or desperate company and proceed to trash the place if not adequately supervised. If you can't detect bad work being committed, that's mostly on the company, not ChatGPT. Yes, it may make it harder, a bit, but it was oversight you should already have been prepared to have, doubly so if you can't trust employee output. It also probably implies strong QA, which is already a prerequisite of a solid product company. Normal interest rates coming back will cut away companies that waste everyone's time and money by overloading themselves on endlessly compounding technical debt.
- foota 2y agoIs the idea here that normal (read: low?) interest rates will let companies spend more time getting things right?
- krick 2y agoI don't really care about them marketing their company, but, Jesus, seriously, that's how software is going to be written now. TBH, I'm not really sure if it's that much different from how it was, but it sounds just... fabulous.
- gravescale 2y ago> We just let it generate the code and everything seemed to work Everyone using C++20 compilers: side-glancing monkey.
- pclmulqdq 2y agoThe difference is that you can read the code that ChatGPT generates.
- jasonlotito 2y ago> It read like no one really knew what they were doing. Then no one knows what they are doing. I really don't know any company that doesn't make what could be considered rookie mistakes by some armchair "developer" here on HN.
- viraptor 2y agoThey spent 5 days. The bug type is pretty common and could easily be done by a developer. (It's a similar class to the singleton default argument issue that many people complain about) Meh, I don't mind the cautionary tale and don't think chatgpt was even relevant. It's actually a tricky bug, because usual tests wouldn't catch it (db wiped for good isolation) and many ways of manual testing would restart the service (and reset the value) on any change and prevent you from seeing it. Ideally there would be a lint that catches this situation for you.
- crooked-v 2y ago> and could easily be done by a developer ...who didn't know how the ORM they were using worked. That's what makes them look so bad here: nobody knew how it worked, not even at the surface level of knowing what the SQL actually generated by the tool looks like.
- TheRoque 2y agoIn their defense, I find SQLAlchemy syntax quite horrible, and I always have to look up everything. It also got a 2.0 release recently which changes some syntax (good luck guessing which version ChatGPT will use), and makes the process even more annoying.
- zo1 2y agoSQLAlchemy syntax is ridiculously obvious and straightforward as long as you're not doing anything weird. The takeaway here is that they weren't mature enough to realize they were, in fact, doing something "weird". I.e. Using UUIDs for PKs, because hey "Netflix does it so we have to too! Oh and we need an engineering blog to advertise it". Edit. More clarity about why the UUID is my point of blame: If they had used a surrogate and sequential Integer PK for their tables, they would never have to tell SQLAlchemy what the default would be, it's implied because it's the standard and non-weird behavior that doesn't include a footgun.
- sgarland 2y ago
- nxobject 2y agoIn a way, it does give you an opportunity to think about what you appreciate in a detailed postmortem - not just a single cause, but human and organizational factors too, and an attempt to figure out explicit mitigations. I’ll admit the informality and the breezy tone here made me go “woah, they’re a bit cavalier…”
- koito17 2y agoMore importantly, what was the motivation behind a rewrite from TypeScript to Python? From the article Our project was originally full stack NextJS but we wanted to first migrate everything to Python/FastAPI. Seems like this entire mess could've been avoided if they had stuck with their existing codebase, which seemed to have been satisfying their business requirements.
- dgfitz 2y agoI’m sure they are influenced by the likes of Reddit and Twitter rewriting their stack. I mean, that’s what has to be done, right? /s
- __jonas 2y agoIs there a trend there of moving from Next to FastAPI? I would be surprised. Perhaps they are doing some AI thing and want to have python everywhere.
- nicce 2y ago> NextJS There is well-hidden vendor-lock when using NextJS, at least.
- brigadier132 2y agoThere is no vendor lockin with nextjs
- nicce 2y agoSimply put - if you want get the best out of the framework, you need to host it in Vercel. Otherwise, there are better options for frameworks. No need to ”fight it”. You will find many issues from GitHub which are not considered because they would make the framework ”better” or easier to use on other clouds.
- mattigames 2y ago
- jorams 2y agoYeah this is not a good thing to advertise. - They were under large time constraints, but decided a full rewrite to a completely different stack was a good idea. - They copy-pasted a whole bunch of code, tested it manually once locally, once in production, and called it a day. - The debugging procedure for this issue so significant it made them dread waking up involved... testing it once and moving on. Every day. The bug is pretty easy to miss, but should also be trivial to diagnose the moment you look at the error message, and trivial to reproduce if you just try more than once.
- hehdhdjehehegwv 2y agoIronically one of my frequent GPT questions is “X is supposed to do Y, but is doing Z. What logs should I look at and what error messages to keep an eye out for?”
- atrettel 2y agoI'd rather they admit a mistake and learn a lesson from it even if it isn't a good thing to advertise. That said, I agree that you are identifying a more important issue here but I also think you are being a bit too subtle about even if I agree with what you are saying. The real lesson that they should have learned from this ordeal is to never push code directly into production --- period. The article never mentions using a testbed or sandbox beforehand, and I kinda feel like they learned a good lesson but it may in fact be the wrong lesson to learn here.
- theamk 2y agoI don't see how testbed/sandbox would have helped, unless they'd also have a dedicated QA person _and_ configured their sandbox so have dramatically fewer instances. Because I can see "create a new subscription" in the manual test plan, but not "create 5x new subscription".
- owl57 2y ago> configured their sandbox so have dramatically fewer instances May be a wise thing to do anyway. I would even advocate for the extreme, one instance in sandbox. I've seen quite a variety of bugs that would be easier to detect and debug with one instance — from values accidentally persisted between requests (OP is one example of this very broad class of bugs), to a thundering herd bug catched in staging due to less instances and more worker threads per instance. But can't remember even one "distributed" bug caught in staging. > they'd also have a dedicated QA person This is a new feature, and it's user-facing, business-critical and inherently fragile (external service integration, though it broke for another reason). I hope multiple people manually run such things end to end before deploying to prod, even if none of them are dedicated QA?
- AdamJacobMuller 2y agoThe fact that they couldn't find it by looking at error logs is weird to me. This is an entirely forgivable error but should have been found the first time they got an email about it: "Oh, look, the error logs have a duplicate key exception for the primary key, how do we generate primary keys.... (facepalm)" Funnily enough, I saw the error in their snippet as soon as I read it but dismissed it thinking there was some new-fangled python feature which allowed that to work like the function def defines that default= accepts only functions so the function gets passed? -- I haven't kept up with modern python and that sounds cool and I figured the bug couldn't be THAT simple.
- dalemhurley 2y agoI was wondering that too. Why wouldn’t the error be in the logs?
- throwaway346434 2y agoGuess: the logs were on an ec2 instance that was thrown away regularly, and the overnight reports didn't give reproduce steps or timestamps; so when they checked it "works fine". There's value in having your backtrace surfaced to end users rather than swallowing an exception and displaying "didn't work".
- 8n4vidtmkvmk 2y agoI don't think showing stack traces to users is good practice? Every time one of my users gets a didn't work message I log the stack trace instead.
- dalemhurley 2y agoWhy would you show them a stack trace? This should be logged.
- erhaetherth 2y ago
- SmellTheGlove 2y agoThis is a pretty common mistake with sqlalchemy whether you’re using ChatGPT or not. I learned the same lesson years ago, although I caught it while testing. I write plenty of python and I just don’t often pass functions in as parameters. In this case you need to! For something like this where you’re generating a unique id and probably need it in every model, it’s better to write a new Base model that includes things like your unique id, created/changed at timestamps, etc. and then subclass it for every model. Basically, write your model boilerplate once and inherit it. This way you can only fuck this up once and you’re bound to catch it early before you make this mistake in a later addition like subscription management.
- wateroutflow 2y agoYea, more of an issue of how this python library can cause misunderstandings, and ChatGPT failing in the same misunderstanding that would have been made by an engineer who lacks experience in that library. This mistake would have happened even if they did not use ChatGPT.
- hamasho 2y agoI wonder are there linters to detect those types of mistakes for SQLAlchemy. Even though I'm aware of such pitfalls, it's nice if linters can catch them cause I'm not confident to cache them all the time during code review. Some linters like pyright can identify dangerous defaults in function call, like `def hello(x=[]): pass` (mutable values shouldn't be a default). Linter plugins for widely-used and critical libraries like SQLAlchemy are nice to have.
- whakim 2y agoThe mutable-default-arguments issue is easy for a third-party to linter to catch because it doesn't require any specific knowledge about primary keys and databases. Are there static typing plugins for other common packages that would catch issues like this?
- aragilar 2y agoI personally use pylint, which I've found to be the most aggressive python linter.
- Lerc 2y agoThe bad thing is what they did, not that the disclosed it. I agree that this is probably to their disadvantage, but I would much rather have people admitting their faults than hiding them. If everyone did this the world would be better. Of course the best solution is to not have faults but that is like saying that the solution to being poor is to have lots of money. It's much easier to say than do.
- theamk 2y agoThe bad thing is their engineering culture and not anything technical. We all make mistakes, the question is how we fix them. Look a the last sentences of the post: > Yes we should have done more testing. Yes we shouldn't have copy pasted code. Yes we shouldn't have pushed directly to main. Regardless, I don't regret the experience. None of those are unconditionally bad! Every project I've worked on could use more testing; we all copy-pasted code at least occasionally, and pushing to main is fine in some circumstances. The real problem is that they went live, but their tooling (or knowledge how to use it) was so bad it took 5 days to solve the simple issue; and meanwhile, they kept pushing new code ("10-20 commits/day") while their customers were suffering. This is what really causes the reputation hit.
- kidme5 2y agothat blog post is all they have so not much to worry about company wise
- joemazerino 2y agoI appreciate the author's honesty. Its better to see transparently what happened so customers know the problem is fixed.
- hehdhdjehehegwv 2y agoThis is embarrassing, I’d honestly consider pulling this post for your reputation.
- hitekker 2y agoThat's an alright takeaway: the team made a rookie mistake and then they made a PR mistake by oversharing. Otherwise, I think this comment thread is a classic example why company engineering blogs choose to be boring. Better ten articles that have some useful information, than a single article that allows the commentariat to pile on and ruin your reputation.
- hobs 2y agoA mistake is interesting if the mistake itself or the RCA is interesting - using sloppy methods isn't really that interesting on its face.
- sillysaurusx 2y agoI think it’s an unfair takeaway. I have over a decade of experience and still had to stare at the line to find the bug. If that makes them incompetent, I stand with them. It’s a bug I’ve seen people make in other contexts, not just chatbots. The AI angle is probably why people are piling on. There’s a latent fear that AI will take our jobs, and this is a great way to skewer home that we’re still needed. For now. The one thing I will say is that it probably wouldn’t take me days to track it down. But that’s only because I have lots of experience dealing with bugs the way that The Wolf deals with backs of cars. When you’re trying to run a startup on top of everything else, it can be easy to miss. I’m happy they gave us a glimpse of early stage growing pains, and I don’t think this was a PR fumble. It shows that lots of people want what they’re making, which is roughly the only thing that matters.
- marginalia_nu 2y agoEh, I think it speaks fairly well for them. On the one hand it does seem like a fairly inexperienced organization with some pretty undercooked release and testing processes, but on the other hand all that stuff is ultimately fixable. This is a relatively harmless way of learning that lesson. Admitting a problem is the first step toward fixing it. A culture of ass-covering is much harder to fix, and will definitely get in the way of addressing these types of issues
- Aurornis 2y ago> Better ten articles that have some useful information, than a single article that allows the commentariat to pile on and ruin your reputation. Pile-on aside, the problem with this blog article is that it doesn't really have much of a useful takeaway. They didn't even really talk the offending line in detail. They didn't really talk about what did to fix their engineering pipelines. It was just a story about how they let ChatGPT write some code, the code was buggy, and the bug was hard to spot because they relied on customers e-mailing them about it in a way that only happened when they were sleeping. It's not really a postmortem, it's a story about fast and loose startup times. Which could be interesting in itself, except it's being presented more as an engineering postmortem blog minus the actionable lessons. That's why everyone is confused about why this company posted this as a lesson: The lesson is obvious and, frankly, better left as a quiet story for the founders to chuckle about to their friends.
- deleted 2y ago[deleted]
- hunterrr 2y agoThese criticisms about engineering PR are too heavy handed. Great engineers solve problems and describe problems without finger pointing to place blame. In fact I think that the worst engineers I’ve worked with are the ones most often reaching for someone to place it on.
- deleted 2y ago[deleted]
- Kesty 2y agoCEO thoughts: "Oh post-morten are always well received, I should write one for that very really basic bug we had and how we took 5 days to find it, and forget to mention how we fix it or how we have changed our structure so that it never happen again" Also the CEO: "remember to be defensive on reddit comments saying how we are a small 1 million dollar backed startup and how it's normal do to this king of rookies mistake to be fast."