7 ms·

Software Engineering's Greatest Hits [video]

180 points by satchet 5y ago

fbr 5y ago
Link to the slides: https://third-bit.com/talks/greatest-hits/ https://third-bit.com/talks/greatest-hits/
citrin_ru 5y ago
> Perl is as hard for novices to learn as a language with a randomly-designed syntax Make something accessible to novices IMHO is a wrong goal in many cases. A tool like a programming language used day to day for many months or years. What important is how efficiently you can use it after spending enough time to learn it (I would say a month or two should be enough for most languages if you know already a few, but it depends). Of course we cannot ignore how hard is to learn a language otherwise we will have languages like C++ which require years to master it (and even coders with years of experience fall pray to numerous hidden traps of the language).
- bakuninsbart 5y ago
  An interesting research question would be if the things that make it harder for a novice to understand code are exactly the same things that make experts more productive. Curly braces was used as an example in the talk, and for me at least they are very useful to understand code structure. I still remember that one of the hardest things for me to get started with in Java was the seemingly endless types. There was a general overload with phrases I had never seen before coming from Python. While I am not too strongly opinionated on static vs. dynamic typing today, static typing is very popular among expert programmers.
  - oriolid 5y ago
    The best explanation for Java language design I have heard that it's not designed to be productive but to prevent mediocre developers from shooting themselves in the foot.
    - xen0 5y ago
      And yet, the Enterprise Java meme.
      - oriolid 5y ago
        Did you mean AbstractCorporationCategoryLanguageMemeFactory? Anyway, at the time Enterprise Java was developed, it was commonly believed that the best way to do large scale development is to hire busload of outsourced workers and give them tools that don't let them do too much damage. If you let them have C++, you'd end up with Series 60.
- bsenftner 5y ago
  After the "toy" languages of Basic and Pascal, my first real programming language was PDP11 Macro-11 Assembly. At the end of that programming class, the final assignment combined all the prior assignments and to the entire class's surprise we had a C compiler. From that laying a foundation experience, I felt extremely comfortable when learning and working in C for a good 20 years after graduation. C++ appeared first as a pre-processor for C, and then took over as the primary compiled language with C treated as a less favored step child. Over the years, my experience has been those that tried to embrace the entire language shot themselves in the feet over and over again: template programming has always been unstable and difficult, operator overloading is a recipe for mistakes, and complex object hierarchies over designed to a fault. Nobody appears to follow the Keep It Simple Stupid (KISS) principal anymore. If there's complexity, it should be innate to the product, and not reflected by how to product is constructed internally. The portion of the talk where it is discussed how many parameters and features of software are never used hit my experience the sharpest. Keep Is Simple!!!
  - mikewarot 5y ago
    >the "toy" languages of Basic and Pascal What makes Pascal a toy language?
    - bsenftner 5y ago
      Too many guards and training wheel preventions? Lack of adherence to a loose standard causing incompatible language variations from different vendors? Pascal editors often enforced a code layout style, with different editors enforcing different styles? No serious work done in pascal?
- Zababa 5y ago
  I think that's a false dichotomy in many cases. For example, compiler errors. Elm and Rust have really good errors, and that benefits everyone. Once that part is done, I agree with you. I also think you may overestimate the depth of knowledge that the average programmer has about his most used programming language. Languages evolve a lot, sometimes faster than how you used them. People also don't use the same programming language during all their life.
- sidpatil 5y ago
  I agree. This is something I tried to explain to others in my last job, with little success. A person using a language long-term is only a novice for so long. After they've figured out the basics and become comfortable solving problems, they become interested in efficiency. Designing a language to cater to novices is like designing a bicycle with only low gears—it's easy to get started, but increasingly difficult to move fast. I.e. the Turing tarpit [1]. I'm biased, and admittedly don't have a lot of perspective on this topic, but I have seen the effects of poorly-designed languages on the application development process and its output. [1] https://en.wikipedia.org/wiki/Turing_tarpit https://en.wikipedia.org/wiki/Turing_tarpit
- netvarun 5y ago
  I agree and well articulated! Your comment reminds me of the quote, “Everything should be made as simple as possible, but no simpler.” (Einstein??)
- bmn__ 5y ago
  > > Perl is as hard for novices to learn as a language with a randomly-designed syntax Stefik is a hack fraud, and he claimed another victim with Wilson. HN readers, beware. https://www.nu42.com/2013/11/are-perl-users-unable-to-write-programs.html https://www.nu42.com/2013/11/are-perl-users-unable-to-write-... http://redd.it/39buec http://redd.it/39buec http://redd.it/38i1pr http://redd.it/38i1pr
ulnarkressty 5y ago
Would be great to see a similar talk about hiring software engineers. Do the studies shown even map to the hiring process?
- mikewarot 5y ago
  Since most people don't handle errors, I think including some testing functional knowledge of error handling might turn out to be surprisingly effective at gauging how effective a programmer truly is.
kqr 5y ago
I was very impressed by Greg Wilson's 2009 presentation, which seems to have disappeared from Video, but I think corresponds to these slides: https://www.slideshare.net/gvwilson/bits-of-evidence-2338367 https://www.slideshare.net/gvwilson/bits-of-evidence-2338367 It truly changed the way I think about not just software development, but it changed my entire view of the world, which is more than I can say of any other presentation. Granted, I was still at a fairly impressionable age of 17 when I listened to it, but still. Definitely looking forward to listening to this one too.
- gtirloni 5y ago
  That video seems to be gone for good but here's a nice discussion about that: https://youtu.be/qDY-Lk1rsQY https://youtu.be/qDY-Lk1rsQY
- phazy 5y ago
  I managed to find an archived audio recording of his talk: https://web.archive.org/web/20120118143443/http://craigagreen.com/downloads/gregwilson.mp3 https://web.archive.org/web/20120118143443/http://craigagree... Even if the quality is not the best, you should still be able to understand most of the presentation.
jbverschoor 5y ago
Spot-on presentation and reflects my experience
infocollector 5y ago
TDD slide has an error it seems. In slide 18 Fucci2016 is referred for a replication study with 39 professionals. That study only uses students. That slide quotes Fucci2016, but I do not see that quote in the paper. He talks about TDD and hatemail and is not careful about references? Perhaps this was just a small error - the reference is incorrect. I am not sure. It seems believable that Test-first vs Test-last might have similar impacts. I just checked one thing in this talk and it turned out to problematic. Has anyone else done fact-checking on his stories?
- Erikun 5y ago
  It seems he took the title of a different Fucci 2016 paper, it should say "A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?" This paper contains both the quote and is done on 39 professionals. This seems like an easy mistake to make and not something I feel has an impact on the trustworthiness of his talk
  - infocollector 5y ago
    Thanks! Seems like short cycles of development including interleaving testing - is the key in this paper. TDD might just achieve this by a fluke, but it should help that - unless you do this TLD or even test addition in cycles. Any thoughts on that?
    - preommr 5y ago
      > TDD might just achieve this by a fluke I am confused by this comment and the overall sentiment in the video. Short dev cycles isn't an accidental byproduct - it's literally one of the key parts of tdd. The first google result for tdd has this: > Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case that defines a desired improvement or new function, then produces the minimum amount of code to pass that test, and finally refactors the new code to acceptable standards.
sokoloff 5y ago
I chuckled slightly on slide 28 at the offhand realization that the longest time in the study was in Java.
mikewarot 5y ago
Well, this has a lot of practical knowledge on offer, and I intend to watch it a few more times for it all to sink in. I've always believed that anyone can learn anything, if they and their teacher both believe it, and put in the effort. I'm glad to see that bias confirmed. I was really surprised that nobody seems to actually handle errors. This does reflect my experience, in what I thought (until now) was an exception. In about 1985 I was taking a vendor taught course in PL/N at the Norand company (who made hand-held computers used for inventory management). We got to the section of the course about error handling. They described the syntax for checking errors, and how to format things, etc. I asked what happened when you got an error, and both instructors had never been asked that question, and both had ZERO clue about it. For all the arguments here about in and out of band error handling, this was shocking, and definitely did not confirm any bias.
- dharmaturtle 5y ago
  I'm a big fan of "Railway Oriented Programming" [1]. Basically your code has two paths - the "Ok" route and the "Error" route. Your code won't compile unless you handle the Error case. However, it requires language level support to use easily - you need mutually exclusive types like discriminated unions. It's harder to do in most OO languages like C#/Java/Javascript, but you can do it with some effort. Which is why most don't. [1]: https://fsharpforfunandprofit.com/rop/ https://fsharpforfunandprofit.com/rop/
  - mpweiher 5y ago
    While ROP is better than some current ways, I vastly prefer dataflow (pipes/filters) as a technique. The fundamental problem, IMHO, with error-handling is the call/return nature of most of our programming languages. That means you have to return something from you function/procedure. If you have nothing to return (error) or nothing to return yet (async), you have a problem, particularly with intermediate functions that don't really know what happened below them and don't really have enough of the context from their callers to handle the error. Exceptions allow you to skip the these intermediate layers. ROP makes you handle all those layers, but gives you some tools to help make sure you did it correctly. Filters, on the other hand, do not return results. Instead they actively pass their results on to the next filter in line. That means that in the case of an error, they can do what functions/procedures cannot: simply not produce a result at all. The next filter will be none-the-wiser. You can then have the filter have an out-of-band mechanism for actually dealing with the error, analogous to Unix stderr.
kova12 5y ago
What he said about developer productivity might be technically wrong, but misleading. Certain types of work require one to be at certain level of mastery, not just in software by the way. One who performs at that level can do the job, but just barely. Below that level you are a burden, not an asset, whoever worked in the industry knows such people, they don't do anything productive, but mostly get in the way. If we then assume that this level is zero, then everything above it it is positive, and or measurement of how much one engineer is better than another is exaggerated. It's like when you are playing an RPG. Quest that requires level 5 is pretty much impossible for level 4. Doable for level 5. Level 6 can pass one easy. Level 10 would zoom through it. He's not 10 times higher in level than level 5, but certainly would get the job done 10 times faster and more reliably. That's where this comes from. Small advantages can mean difference between failing the project, barely pulling the project, or acing one; all of which translates into business losing or making money.
amirkdv 5y ago
> Lots of code metrics have been proposed ... But nothing works better than counting lines of code [with citations] Here's an anecdoate, not to dispute the empirical finding. I recently discovered McCabe (cyclomatic) complexity and found it remarkably good! In a tiny experiment, I ran it on a 50k Python code base with a threshold of 8, and the output almost exactly matched all of the already existing `FIXME: refactor this` comments. According to McCabe the threshold to look out for is 10. --- p.s: if you write Python, you already probably have the mccabe package installed through flake8. Use it like so: $ python -m mccabe --min 8 foo.py

180 points by satchet 5y ago

fbr 5y ago

Link to the slides: https://third-bit.com/talks/greatest-hits/ https://third-bit.com/talks/greatest-hits/

> Perl is as hard for novices to learn as a language with a randomly-designed syntax Make something accessible to novices IMHO is a wrong goal in many cases. A tool like a programming language used day to day for many months or years. What important is how efficiently you can use it after spending enough time to learn it (I would say a month or two should be enough for most languages if you know already a few, but it depends). Of course we cannot ignore how hard is to learn a language otherwise we will have languages like C++ which require years to master it (and even coders with years of experience fall pray to numerous hidden traps of the language).

bakuninsbart 5y ago
An interesting research question would be if the things that make it harder for a novice to understand code are exactly the same things that make experts more productive. Curly braces was used as an example in the talk, and for me at least they are very useful to understand code structure. I still remember that one of the hardest things for me to get started with in Java was the seemingly endless types. There was a general overload with phrases I had never seen before coming from Python. While I am not too strongly opinionated on static vs. dynamic typing today, static typing is very popular among expert programmers.
- oriolid 5y ago
  The best explanation for Java language design I have heard that it's not designed to be productive but to prevent mediocre developers from shooting themselves in the foot.
  - xen0 5y ago
    And yet, the Enterprise Java meme.
    - oriolid 5y ago
      Did you mean AbstractCorporationCategoryLanguageMemeFactory? Anyway, at the time Enterprise Java was developed, it was commonly believed that the best way to do large scale development is to hire busload of outsourced workers and give them tools that don't let them do too much damage. If you let them have C++, you'd end up with Series 60.
bsenftner 5y ago
After the "toy" languages of Basic and Pascal, my first real programming language was PDP11 Macro-11 Assembly. At the end of that programming class, the final assignment combined all the prior assignments and to the entire class's surprise we had a C compiler. From that laying a foundation experience, I felt extremely comfortable when learning and working in C for a good 20 years after graduation. C++ appeared first as a pre-processor for C, and then took over as the primary compiled language with C treated as a less favored step child. Over the years, my experience has been those that tried to embrace the entire language shot themselves in the feet over and over again: template programming has always been unstable and difficult, operator overloading is a recipe for mistakes, and complex object hierarchies over designed to a fault. Nobody appears to follow the Keep It Simple Stupid (KISS) principal anymore. If there's complexity, it should be innate to the product, and not reflected by how to product is constructed internally. The portion of the talk where it is discussed how many parameters and features of software are never used hit my experience the sharpest. Keep Is Simple!!!
- mikewarot 5y ago
  >the "toy" languages of Basic and Pascal What makes Pascal a toy language?
  - bsenftner 5y ago
    Too many guards and training wheel preventions? Lack of adherence to a loose standard causing incompatible language variations from different vendors? Pascal editors often enforced a code layout style, with different editors enforcing different styles? No serious work done in pascal?
Zababa 5y ago
I think that's a false dichotomy in many cases. For example, compiler errors. Elm and Rust have really good errors, and that benefits everyone. Once that part is done, I agree with you. I also think you may overestimate the depth of knowledge that the average programmer has about his most used programming language. Languages evolve a lot, sometimes faster than how you used them. People also don't use the same programming language during all their life.
sidpatil 5y ago
I agree. This is something I tried to explain to others in my last job, with little success. A person using a language long-term is only a novice for so long. After they've figured out the basics and become comfortable solving problems, they become interested in efficiency. Designing a language to cater to novices is like designing a bicycle with only low gears—it's easy to get started, but increasingly difficult to move fast. I.e. the Turing tarpit [1]. I'm biased, and admittedly don't have a lot of perspective on this topic, but I have seen the effects of poorly-designed languages on the application development process and its output. [1] https://en.wikipedia.org/wiki/Turing_tarpit https://en.wikipedia.org/wiki/Turing_tarpit
netvarun 5y ago
I agree and well articulated! Your comment reminds me of the quote, “Everything should be made as simple as possible, but no simpler.” (Einstein??)
bmn__ 5y ago
> > Perl is as hard for novices to learn as a language with a randomly-designed syntax Stefik is a hack fraud, and he claimed another victim with Wilson. HN readers, beware. https://www.nu42.com/2013/11/are-perl-users-unable-to-write-programs.html https://www.nu42.com/2013/11/are-perl-users-unable-to-write-... http://redd.it/39buec http://redd.it/39buec http://redd.it/38i1pr http://redd.it/38i1pr

ulnarkressty 5y ago

Would be great to see a similar talk about hiring software engineers. Do the studies shown even map to the hiring process?

mikewarot 5y ago
Since most people don't handle errors, I think including some testing functional knowledge of error handling might turn out to be surprisingly effective at gauging how effective a programmer truly is.

kqr 5y ago

I was very impressed by Greg Wilson's 2009 presentation, which seems to have disappeared from Video, but I think corresponds to these slides: https://www.slideshare.net/gvwilson/bits-of-evidence-2338367 https://www.slideshare.net/gvwilson/bits-of-evidence-2338367 It truly changed the way I think about not just software development, but it changed my entire view of the world, which is more than I can say of any other presentation. Granted, I was still at a fairly impressionable age of 17 when I listened to it, but still. Definitely looking forward to listening to this one too.

gtirloni 5y ago
That video seems to be gone for good but here's a nice discussion about that: https://youtu.be/qDY-Lk1rsQY https://youtu.be/qDY-Lk1rsQY
phazy 5y ago
I managed to find an archived audio recording of his talk: https://web.archive.org/web/20120118143443/http://craigagreen.com/downloads/gregwilson.mp3 https://web.archive.org/web/20120118143443/http://craigagree... Even if the quality is not the best, you should still be able to understand most of the presentation.

jbverschoor 5y ago

Spot-on presentation and reflects my experience

infocollector 5y ago

TDD slide has an error it seems. In slide 18 Fucci2016 is referred for a replication study with 39 professionals. That study only uses students. That slide quotes Fucci2016, but I do not see that quote in the paper. He talks about TDD and hatemail and is not careful about references? Perhaps this was just a small error - the reference is incorrect. I am not sure. It seems believable that Test-first vs Test-last might have similar impacts. I just checked one thing in this talk and it turned out to problematic. Has anyone else done fact-checking on his stories?

Erikun 5y ago
It seems he took the title of a different Fucci 2016 paper, it should say "A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?" This paper contains both the quote and is done on 39 professionals. This seems like an easy mistake to make and not something I feel has an impact on the trustworthiness of his talk
- infocollector 5y ago
  Thanks! Seems like short cycles of development including interleaving testing - is the key in this paper. TDD might just achieve this by a fluke, but it should help that - unless you do this TLD or even test addition in cycles. Any thoughts on that?
  - preommr 5y ago
    > TDD might just achieve this by a fluke I am confused by this comment and the overall sentiment in the video. Short dev cycles isn't an accidental byproduct - it's literally one of the key parts of tdd. The first google result for tdd has this: > Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case that defines a desired improvement or new function, then produces the minimum amount of code to pass that test, and finally refactors the new code to acceptable standards.

sokoloff 5y ago

I chuckled slightly on slide 28 at the offhand realization that the longest time in the study was in Java.

mikewarot 5y ago

Well, this has a lot of practical knowledge on offer, and I intend to watch it a few more times for it all to sink in. I've always believed that anyone can learn anything, if they and their teacher both believe it, and put in the effort. I'm glad to see that bias confirmed. I was really surprised that nobody seems to actually handle errors. This does reflect my experience, in what I thought (until now) was an exception. In about 1985 I was taking a vendor taught course in PL/N at the Norand company (who made hand-held computers used for inventory management). We got to the section of the course about error handling. They described the syntax for checking errors, and how to format things, etc. I asked what happened when you got an error, and both instructors had never been asked that question, and both had ZERO clue about it. For all the arguments here about in and out of band error handling, this was shocking, and definitely did not confirm any bias.

dharmaturtle 5y ago
I'm a big fan of "Railway Oriented Programming" [1]. Basically your code has two paths - the "Ok" route and the "Error" route. Your code won't compile unless you handle the Error case. However, it requires language level support to use easily - you need mutually exclusive types like discriminated unions. It's harder to do in most OO languages like C#/Java/Javascript, but you can do it with some effort. Which is why most don't. [1]: https://fsharpforfunandprofit.com/rop/ https://fsharpforfunandprofit.com/rop/
- mpweiher 5y ago
  While ROP is better than some current ways, I vastly prefer dataflow (pipes/filters) as a technique. The fundamental problem, IMHO, with error-handling is the call/return nature of most of our programming languages. That means you have to return something from you function/procedure. If you have nothing to return (error) or nothing to return yet (async), you have a problem, particularly with intermediate functions that don't really know what happened below them and don't really have enough of the context from their callers to handle the error. Exceptions allow you to skip the these intermediate layers. ROP makes you handle all those layers, but gives you some tools to help make sure you did it correctly. Filters, on the other hand, do not return results. Instead they actively pass their results on to the next filter in line. That means that in the case of an error, they can do what functions/procedures cannot: simply not produce a result at all. The next filter will be none-the-wiser. You can then have the filter have an out-of-band mechanism for actually dealing with the error, analogous to Unix stderr.

kova12 5y ago

What he said about developer productivity might be technically wrong, but misleading. Certain types of work require one to be at certain level of mastery, not just in software by the way. One who performs at that level can do the job, but just barely. Below that level you are a burden, not an asset, whoever worked in the industry knows such people, they don't do anything productive, but mostly get in the way. If we then assume that this level is zero, then everything above it it is positive, and or measurement of how much one engineer is better than another is exaggerated. It's like when you are playing an RPG. Quest that requires level 5 is pretty much impossible for level 4. Doable for level 5. Level 6 can pass one easy. Level 10 would zoom through it. He's not 10 times higher in level than level 5, but certainly would get the job done 10 times faster and more reliably. That's where this comes from. Small advantages can mean difference between failing the project, barely pulling the project, or acing one; all of which translates into business losing or making money.

amirkdv 5y ago

> Lots of code metrics have been proposed ... But nothing works better than counting lines of code [with citations] Here's an anecdoate, not to dispute the empirical finding. I recently discovered McCabe (cyclomatic) complexity and found it remarkably good! In a tiny experiment, I ran it on a 50k Python code base with a threshold of 8, and the output almost exactly matched all of the already existing `FIXME: refactor this` comments. According to McCabe the threshold to look out for is 10. --- p.s: if you write Python, you already probably have the mccabe package installed through flake8. Use it like so: $ python -m mccabe --min 8 foo.py