3 ms·

> Basically every study I have ever read making some claim about programming (efficacy of IDEs, TDD, static typing, pair programming, formal CS education, ai as

> Basically every study I have ever read making some claim about programming (efficacy of IDEs, TDD, static typing, pair programming, formal CS education, ai assistants, etc...) has been a house of cards that falls apart with even modest prodding. Isn't this true about most things in software? I mean is there anything quantifiable about "microservices" vs "monolith"? Test-driven development, containers, whatever? I mean all of these things are in some way good, in some contexts, but it seems impossible to quantify benefits of any of them. I'm a believer that most decisions made in software are somewhat arbitrary, driven by trends and popularity and it seems like little effort is expended to come to overarching, data-backed results. When they are, they're rare and, like you said, fall apart under investigation or scrutiny. I've always found this strange about software in general. Even every time there's a "we rewrote $THING in $NEW_LANG" and it improved memory use/speed/latency whatever, there's a chorus of (totally legitimate) criticism and inquiry about how things were measured, what attempts were made to optimize the original solutions, if changes were made along the way outside of the language choice that impacted performance etc etc.

jaredklewis 1y ago
I think we agree. To be clear I am not arguing that tools and practices like TDD, microservices, ai assistants, and so on have no effect. They almost certainly have an effect (good or bad). It’s just the unfortunate reality that quantitatively measuring these effects in a meaningful way seems to basically be impossible (or at least I’ve never see it done). With enough resources I can’t think of any reason it shouldn’t be possible, but apparently those resources are not available because there are no good studies on these topics. Thus my skepticism of the “studies” referenced earlier in the thread.

jaredklewis 1y ago

I think we agree. To be clear I am not arguing that tools and practices like TDD, microservices, ai assistants, and so on have no effect. They almost certainly have an effect (good or bad). It’s just the unfortunate reality that quantitatively measuring these effects in a meaningful way seems to basically be impossible (or at least I’ve never see it done). With enough resources I can’t think of any reason it shouldn’t be possible, but apparently those resources are not available because there are no good studies on these topics. Thus my skepticism of the “studies” referenced earlier in the thread.