3 ms·

You're probably right about the effort - but I expect that such "boilerplate" code contains (or leads to) much more than 20% of the bugs. This 90% of code is n

You're probably right about the effort - but I expect that such "boilerplate" code contains (or leads to) much more than 20% of the bugs. This 90% of code is not genuine word-for-word boilerplate (copy/pasted from a known good source). This code is typically constructed fresh each time; or worse, copied from somewhere similar and quickly tweaked for names/types! (I do it, and I see it done all the time.) I expect that the remaining 10% non-boilerplate code, taking 80% of the effort, is much more carefully considered, and less likely to contain those clumsy/forgetful off-by-one or buffer-overflow bugs.

fiddlerwoaroof 4y ago
Yeah, one of the interesting results in empirical studies of defect rates is that defect rate is influenced by lines of code more than other factors like “static types”. Similarly, analyses of defects have discovered that they tend to occur at the end of repetitive sequences of code, because the developer has sort of switched into autopilot mode. I think the obvious conclusion here (and my experience bears this out to some extent) is that languages and libraries that force boilerplate on you produce buggier code than languages and libraries that abstract the boilerplate away.
kbenson 4y ago
There's a reason many languages have large overflowing repositories of modules (or there are well known libraries) that can be downloaded and used that provide boilerplate solutions for many things. Most people don't like writing that boilerplate once they know how to do it and have done it a few times, and would rather just call a function do_that_thing_need_done_on(input1, input2). If it can't be factored out like that and is actual language boilerplate beyond a few lines, that's a failure of the language. If these AI models are suggesting the code that could be called in a library/module instead of the the code to actually include and call a well known and trusted library or module, I'm not sure that's progress. At least when someone notices a bug or better way to do it and updates that module or library, consumers of that module can update and benefit from it, or at a minimum see that there were bugs in the version they're running they might want to address at some point.
- another-dave 4y ago
  > If these AI models are suggesting the code that could be called in a library/module instead of the the code to actually include and call a well known and trusted library or module, I'm not sure that's progress. I think the judgement call of when to use a library & what library to use is quite subjective, even for humans to get right. If I'm doing JSON deserialisation it might suggest I use Gson library which would be much better than rolling your own. But the original authors are saying that you should prefer Moshi over Gson — I think it'd be hard for an AI to reach that conclusion though (though maybe not if it's doing something like tracking migrations in OS projects from Gson->Moshi). With something a little more trivial — I don't want it to add in a dependency on left-pad, even though it has 2.5M weekly downloads so is arguably both well-known and trusted :) You could probably set a threshold for how complex code is before it's suggested to be swapped out for a lib, but then is my code simple because I'm ignoring edge cases I should support, or because I've trimmed the fat on what I'm choosing to support (e.g. i18n, date handling, email validation etc.)
  - kbenson 4y ago
    I agree it's not always cut and dry which module to use, or whether to use a module for something extremely simple (which is why I mentioned it being more than a few lines, which should weed out stuff like left-pad I would hope), but I think knowing there is a module and it's suggesting it might be a good first step. The only thing worse than using a module that has a bug/security problem for a function that's just a few lines and not used again in the codebase is when the content of that function is copied in place instead of being included and nobody has an easy way of knowing whether that's the code that was suggested and included in their project. Worst of both worlds.

fiddlerwoaroof 4y ago

Yeah, one of the interesting results in empirical studies of defect rates is that defect rate is influenced by lines of code more than other factors like “static types”. Similarly, analyses of defects have discovered that they tend to occur at the end of repetitive sequences of code, because the developer has sort of switched into autopilot mode. I think the obvious conclusion here (and my experience bears this out to some extent) is that languages and libraries that force boilerplate on you produce buggier code than languages and libraries that abstract the boilerplate away.

kbenson 4y ago

There's a reason many languages have large overflowing repositories of modules (or there are well known libraries) that can be downloaded and used that provide boilerplate solutions for many things. Most people don't like writing that boilerplate once they know how to do it and have done it a few times, and would rather just call a function do_that_thing_need_done_on(input1, input2). If it can't be factored out like that and is actual language boilerplate beyond a few lines, that's a failure of the language. If these AI models are suggesting the code that could be called in a library/module instead of the the code to actually include and call a well known and trusted library or module, I'm not sure that's progress. At least when someone notices a bug or better way to do it and updates that module or library, consumers of that module can update and benefit from it, or at a minimum see that there were bugs in the version they're running they might want to address at some point.

another-dave 4y ago
> If these AI models are suggesting the code that could be called in a library/module instead of the the code to actually include and call a well known and trusted library or module, I'm not sure that's progress. I think the judgement call of when to use a library & what library to use is quite subjective, even for humans to get right. If I'm doing JSON deserialisation it might suggest I use Gson library which would be much better than rolling your own. But the original authors are saying that you should prefer Moshi over Gson — I think it'd be hard for an AI to reach that conclusion though (though maybe not if it's doing something like tracking migrations in OS projects from Gson->Moshi). With something a little more trivial — I don't want it to add in a dependency on left-pad, even though it has 2.5M weekly downloads so is arguably both well-known and trusted :) You could probably set a threshold for how complex code is before it's suggested to be swapped out for a lib, but then is my code simple because I'm ignoring edge cases I should support, or because I've trimmed the fat on what I'm choosing to support (e.g. i18n, date handling, email validation etc.)
- kbenson 4y ago
  I agree it's not always cut and dry which module to use, or whether to use a module for something extremely simple (which is why I mentioned it being more than a few lines, which should weed out stuff like left-pad I would hope), but I think knowing there is a module and it's suggesting it might be a good first step. The only thing worse than using a module that has a bug/security problem for a function that's just a few lines and not used again in the codebase is when the content of that function is copied in place instead of being included and nobody has an easy way of knowing whether that's the code that was suggested and included in their project. Worst of both worlds.