3 ms·
Considering the many hundreds of technical comments over at the PR (https://github.com/nodejs/node/pull/61478 https://github.com/nodejs/node/pull/61478), the 8
by joshkel 7mo ago
Considering the many hundreds of technical comments over at the PR (https://github.com/nodejs/node/pull/61478 https://github.com/nodejs/node/pull/61478), the 8 reviewers thanked by name in the article, and the stellar reputations of those involved, seems likely.
- keeganpoppen 7mo ago[flagged]
- grey-area 7mo agoMy mistake 19k lines. At 2 mins per line that’s (19000*2)/60/7=90 7-hour days to review it all, are you sure it was all read? I mean they couldn’t be bothered to write it, so what are the chances they read it all? For someone’s website or one business maybe the risk is worth it, for a widely used software project that many others build on it is horrifying to see that much plausible code generated by an LLM.
- pull_my_finger 7mo agoWhen you review code, do you spend 2 minutes per line? That seems like a huge exaggeration of effort required
- seattle_spring 7mo agoI probably review about 1k LoC worth of PRs / day from my coworkers. It certainly doesn't take me 33 hours (!!) to do so, so I must be one of those rockstar 10x superhero ninja engineers I keep hearing about.
- dirkc 7mo agoAre your coworkers producing the code using LLMs? And what level of trust do you place in them?
- ThunderSizzle 7mo agoFor half my coworkers, their LLM code is better than their code.
- girvo 7mo agoThat’s depressing. For 80% of my coworkers their LLM code is horrible. Only the seniors seem to use it well and not just spit out garbage
- ThunderSizzle 7mo agoI think that goes back to whether they are programmers vs engineers. Engineers will focus on professionalism of the end product, even if they used AI to generate most of the product. And I'm not going by "title", but by mindset. Most of my fellow engineers are not - they are just programmers - as in, they don't care about the non-coding part of the job at all.
- grey-area 7mo agoDepends - if it is from a human I find I can trust it a lot more. If it is large blobs from LLMs I find it takes more effort. But it was just a guess at an average to give an estimate of the effort required. I’d hope they spent more than 2 mins on some more complex bits. Are you genuinely confident in a framework project that lands 19kloc generated PRs in one go? I’d worry about hidden security footguns if nothing else and a lot of people use this for their apps. Thankfully I don't use it, but if I did I'd find this really troubling. It also has security implications - if this is normalised in node.js it would be very easy to slip in deniable exploits into large prs. It is IMO almost impossible to properly review a PR that big for security and correctness.
- weird-eye-issue 7mo ago> I mean they couldn’t be bothered to write it, so what are the chances they read it all? What kind of logic is this?
- grey-area 7mo agoIt’s much harder to read code carefully than to write it. Particularly code generated by LLMs which is mostly correct but then sometimes awful.
- pas 7mo agousually yes, but that's why there are tests, and there's a long road before people start depending on this code (if ever). people will try it, test it, report bugs, etc. and it's not like super carefully written code is magically perfect. we know that djb can release things that are close to that, but almost nobody is like him at all!
- ovflowd 7mo agoThe PR has been open for 3 months, and all the reviewers involved have actually read the whole code and are experts on the matter.