4 ms·
Evidence-based Software Engineering: based on the publicly available data
- deleted 5y ago[deleted]
- bckr 5y agoI'm glad this has been published. It would be tempting to delay such a publication until more data is available. However, the author(s?) are doing it in the way of honest science, and hopefully this will catalyze further work on EBSE. Who knows, maybe the big companies will decide that the rising tide effect is worthwhile and fill in the gaps.
- ExcavateGrandMa 5y agoAh! yes of course.
- juskrey 5y agoOh well, all of my 20 years of software engineering are a tale of fighting in the dark before anything works and heavy precaution after, without having any remote evidence of nearly everything readily available.
- moksly 5y agoThat’s the most interesting find of the book isn’t it? We’ve taken a sort of controversial approach to software development in the public sector of Denmark because we don’t have enough resources. Small projects are build to run their own little lives with as little post-deployment development as possible. Because of this they aren’t build with best practices, whatever theories are the right way to do things in a current age, none of them are automatically tested because their functionally is so small it’s always going to be obvious where the flaws are and the only way for them to get additional development post deployment is if they turn out to be really good, and even then, we don’t expect them to live after 5-10 years. It’s way of development goes against every theory on software development you’ll find being taught at universities and academies, I know, because I’m an external CS examiner at these places, and we’ve really only chosen to do this because we have 3 developers in an organisation with 10.000 employees. Now the data shows us, that we’re actually better off doing the wrong thing.
- whatshisface 5y agoSmall programs are by far the best approach. The only time that doesn't work is when the problem itself requires a very large program. That is when you need things like automated tests.
- InfiniteRand 5y agoIn theory the obvious solution would be to make everything modular small programs but then you pay the price of coordination (ie, apis between modules, dependency, maintaining uniform practices etc.) so... shrug
- 908B64B197 5y ago> because their functionally is so small it's always going to be obvious where the flaws That, sadly, won't scale. > we're actually better off doing the wrong thing. You aren't actually doing the wrong thing. At 3 employees the maximum complexity your team can handle is pretty low. It's also trivial to pass on institutional knowledge since well... the whole team can share one extra large pizza.
- m1sta_ 5y agoHeavy development processes don't scale when it comes to volume of projects.
- moksly 5y ago> That, sadly, won't scale. We never really need it to. That’s one of the few benefits of writing software in the public sector. We don’t need to expect an explosive growth in users. We’ve seen 10.000 additional citizens move to our area over 5 years, so there is growth, but it’s far from everyone of them who would interact with a single solution that we wouldn’t buy from the private sector through public procurement.
- 908B64B197 5y agoI meant feature-wise. My message was unclear. There's a point where a team of three maintaining tribal knowledge is not enough to deliver certain project above a threshold of complexity. But you mention procurement, so I assume your team does a buy vs build pretty much systematically.
- gawi 5y agoI really like the cover where you can find illustrations of anti-patterns such as stovepipes, gas factories, reinventing the wheel, etc. Can you names the others?
- harveywi 5y agoA cover when the document is digital.
- swiley 5y agoI wish more stuff came with fun illustrations as canonical "covers." Also the file looks like it's probably exactly what they gave to the printers (the left and right margins are different for odd/even pages.)
- bckr 5y agoOh God, now I must know. I see a ticking time-bomb and the blind leading the blind. Not sure if those are canonical. About three dozen other vignettes that I definitely can't pin to anti-patterns. Like man-eating plants, teaching magic to rabbits, gearheads, touching a lightbulb... someone please help.
- lifeisstillgood 5y ago>>> based on an analysis of all the publicly available data This aim is not as ambitious as it sounds, because there is not a great deal of data publicly available. Amusing and revealing at the same time. Bet they did not take long to agree on that senrence
- AnimalMuppet 5y agoI love the idea, and the honest assessment of the quality of the available data. But good grief, cut to the chase already! This reads like they felt they had to give a thorough discussion to every possibly relevant tangential idea. I recommend skimming rather than straight-up reading.
- whatshisface 5y agoThey had an entire chapter on general statistical knowledge! I wouldn't mind an abridged version that assumed all prerequisite general math knowledge and covered only the topic material.
- denton-scratch 5y agoI've been skimming forward. There are three chapters on general statistical knowledge - maybe 1/3 of the book. There are chapters on Probability, Statistics, and Regression. None of those chapters illuminates the subject at hand; they provide an explanation of the author's techniques. It's background. There are several other "background" chapters, that aren't about statistics, but also aren't about the subject at hand. So it looks as if I read most of the meat yesterday. I read most of the chapter on Regression, because I didn't know much about it, and the author relies rather heavily on regression analysis.
- derefr 5y agoI'd love to read a book that teaches concepts "in anger" — where you're given a big problem statement with lots of subproblems; and then, in each chapter, the author: • introduces a subproblem, that would be solvable but for one tool you're currently missing; • introduces the tool you'll need to solve the subproblem; • explains the tool in the process of using the tool to solve the subproblem. Only explaining as much of the tool as is necessary to solve the subproblem — more can be explained in later subproblems. Sadly, I find that most books like this first introduce all the tools you'll need; teach them all thoroughly (including teaching stuff you won't need at all, because it's part of the explanation as traditionally taught); and then go about solving all of the subproblems using all of the tools, in a pretty random order. So the first half ends up boring, and the second half ends up messy and hard-to-follow.
- neves 5y agoThis well written book from O'Reilly covers the same subject: https://www.oreilly.com/library/view/making-software/9780596808310/ https://www.oreilly.com/library/view/making-software/9780596... It is from 2010, I don't know if their is really new things on the subject
- muench 5y agoOne of the editors of that book wrote about Derek Jones’s book “I think (at least I hope) this will influence software engineering education [more] than anything I've ever done” https://twitter.com/gvwilson/status/1326190553738977281?s=20 https://twitter.com/gvwilson/status/1326190553738977281?s=20
- jdlshore 5y agoI’m not so sure about that book. It includes Steve McConnell’s claims about 10x productivity, which Laurent Bossavit thouroughly and convincingly debunked in his book, The Leprechauns of Software Engineering. A bunch of citations isn’t the same as evidence. When I dig into software studies, I find all kinds of issues, ranging from subjects of convenience (the professor used their undergrad class for their study) to industry studies that don’t define their terms. (e.g., MS did a large empirical study of TDD, but never defined how TDD was practiced—this is relevant given the misunderstandings of TDD that exist, particularly at Microsoft). I didn’t read the rest of the book—just looked at the blurb, winced at the inclusion of McConnell repeating his 10x claim, and read his chapter—but my default stance is to be highly skeptical of academic software engineering, and even more skeptical of popularizations. Edit: I’ve just glanced through OP’s book, and it looks like an impressive piece of scholarship. I remain skeptical of academia’s ability to teach much of interest regarding software engineering, though. (They spend a lot of words on Waterfall and traditional project management. Their words on the Agile-kinda stuff companies actually use today? One paragraph, paraphrased: “there is no data, we have nothing to say.” Not the authors’ fault, but sums up the irrelevance of academia to real-world SE, in my mind.)
- feoren 5y agoI'm not sure how you can "debunk" something that so many people have directly observed, but our ability to measure a software engineer's productivity is so awful that I'm willing to consider the possibility that we're all just wrong. Are 10x engineers just good at appearing productive then?
- dang 5y agoRelated previous threads: Evidence-based software engineering: book released - https://news.ycombinator.com/item?id=25073218 https://news.ycombinator.com/item?id=25073218 - Nov 2020 (118 comments) Evidence-based Software Engineering – book beta - https://news.ycombinator.com/item?id=23769930 https://news.ycombinator.com/item?id=23769930 - July 2020 (22 comments) Evidence-based Software Engineering [pdf] - https://news.ycombinator.com/item?id=3213482 https://news.ycombinator.com/item?id=3213482 - Nov 2011 (9 comments) Evidence based software development - https://news.ycombinator.com/item?id=2707618 https://news.ycombinator.com/item?id=2707618 - June 2011 (1 comment) What we actually know about software development, and why we believe it’s true - https://news.ycombinator.com/item?id=901600 https://news.ycombinator.com/item?id=901600 - Oct 2009 (54 comments) I'm pretty sure there were others in years past... maybe someone will find one.
- denton-scratch 5y agoI picked up this book yesterday, and I'm now about 2/3 the way through (I skipped or skimmed about 1/4 of what I've got through - I'm reading for fun, and the returns seem to be diminishing). I appreciate the author's frankness about the limitations he's working under: if there's no data available about some topic, he doesn't discuss that topic. In fact he's pretty rigourous about 'following the data'. I think I would like just a little bit more speculation about e.g. developer motivations. And it's a shame there's not enough data to support an analysis of modern development paradigms such as TDD, pair-programming and so on. Of course that's not the author's fault; anyway, in a sense, the absence of data is itself interesting. I've found the book interesting. I think there are too many 'data dumps', i.e. descriptions of datasets that don't cast much light on anything. I think I'd find a review or abstract that distils the book useful and engaging.
- klausnrooster 5y ago"Those seeking discussions written in the style of a stimulant for mathematical orgasms, will not find satisfaction here." Hahaha, classic. Like his style in general.