3 ms·
Maybe these larger models are better at producing concise, internally-consistent, non-polluting (of the wider codebase) features. I just spent 3 days reviewing
by ianmarcinkowski 2mo ago
Maybe these larger models are better at producing concise, internally-consistent, non-polluting (of the wider codebase) features.
I just spent 3 days reviewing the code of a vibe coding maximalist at work. They needed to touch 96 front-end files, ~50 back-end files. We now have `is_uk_company` spread throughout the entire codebase every time we need to output a date format. I would say 30% of the code can be replaced by a moderately more-wise approach of "you ain't gonna need this", 20% is actively polluting and/or harmful to the coherence of the feature.
I'm pretty sure this plan came from a braindump of a verbal meeting they had with our product team, using OpenAI's top models. There were 2 initial spreadsheets that were ABSOLUTELY INDECIPHERABLE, and now we have a new one labeled "FeatureX - Phase 2 to 4".
Just the date fields is a hilarious example of why I'm skeptical. Python (babel.date) and Javascript (the browser's locale) both have locale-aware date rendering. Using a library for country-based date and time parsing would have eliminated ~12 country-specific change sites... but instead we got `is_uk` and raw date format strings (`%d/%m/%y`) in-line inside already-enormous functions. Also, at a meta level, the backend code shouldn't usually be formatting the dates (data export may be an exception), we should be sending ISO dates and having the frontend format the code if anyone gives a crap.
I'm ~$500/day and I spent 3 days reviewing this stuff. Our CTO spent ~2 days this week reviewing this feature, I'm not sure what his daily rate would end up being. Just the date-formatting part cost me half a day... that's at least $250. A few hundred dollars on Xhigh tokens burned $2500-3500 in human review effort, and I was delayed from:
1. Getting an entire sentiment analysis feature deployed by helping our interns
2. Getting a load balancer solution working to automate a manual process
3. Improving our deterministic Cypress test suite
4. Getting a demo feature launched for our sales team that would bring in more money
- patrickmay 2mo ago> Just the date fields is a hilarious example of why I'm skeptical. Python (babel.date) and Javascript (the browser's locale) both have locale-aware date rendering. Using a library for country-based date and time parsing would have eliminated ~12 country-specific change sites... but instead we got `is_uk` and raw date format strings (`%d/%m/%y`) in-line inside already-enormous functions. Also, at a meta level, the backend code shouldn't usually be formatting the dates (data export may be an exception), we should be sending ISO dates and having the frontend format the code if anyone gives a crap. Did you reject the PR or is that not a career-enhancing approach in your organization?
- ianmarcinkowski 2mo agoI've worked with my teammate to see why this is suboptimal and negotiated a lot of concessions. I didn't re-review the code before we pushed it to staging for QA to test over the weekend... I'm worried about what I'll find when I look on Monday.