9 ms·

> "In a competitive market for development work and staff, paying people to learn from mistakes that have already been made by many others is an unaffordable lu

by aard 6y ago

> "In a competitive market for development work and staff, paying people to learn from mistakes that have already been made by many others is an unaffordable luxury; an engineering approach, derived from evidence, is a lot more cost-effective than craft development." I disagree with this characterization. Yet again, we developers are being told to just follow the program. That programming is not an artistic or craft endeavor that benefits from experience and intuition. It is like working in a factory where coders just bang out widgets on an assembly line. And the self-appointed thinkers will optimize the process for us. What is at risk by not allowing developers to "learn from mistakes" is autonomy. Striping developers of their autonomy is the primary cause of poor performance, not an inability to execute so-called "best practices" Attempts to codify the process of software development always fail, because coding is a design process not a manufacturing one. Developers do their jobs in many different ways, many of which are equally effective. There is more than one way to skin a cat -- especially in creative work. > "The labour of the cognitariate is the means of production of software systems" This false assumption is at the base of the problem. The work of the compiler (or scp) are the means of software production. Coding is design. Once the design is complete, the results are compiled and copied to their target environments. In software, production is negligible. Which promotes the misconception that developers are producing software. In actuality they are designing software. The difference may seem subtle, but it is crucial.

Jtsummers 6y ago
It really depends on the kind of systems and work you're doing. In my previous office, there was an entire group (about 100-150) of programmers whose job was incredibly rote. That is, you could take a novice out of college and get them up to speed in about 1-3 months to do even their most complex work. However, the other groups were much less factory-ish, though rarely anything truly novel. Only a small cadre of programmers were working on anything that really required novelty and creativity. It's a spectrum and that has to be understood by all participants in the discussion. Managers want everything to be like that first group, because it's so consistent and predictable. They want to know that a project will take 1000 man-hours and be right 95% of the time. Many programmers want to see themselves part of the last group. The reality is, most of us are in the middle. There are aspects of the job that are almost mechanical, and aspects which require greater creativity or "craft". If we can clarify that for the managers, it can go a long way to ending some of the nonsense.
- 0x445442 6y ago
  I recognized this 20 years ago but my recognition was crude. What I noticed early on, in almost every place I worked, was most of what could be called office politics for software devs/engineers boiled down to maneuvering to work on API's or even some place lower on the call stack.
StillBored 6y ago
Sorry to be the killjoy here, but This is a big mischaracterization of what "software engineering" is attempting. In this case it is an attempt to document what should be learned from experience and intuition. Because, the vast majority of software engineering IS "factory" work because its the grunt work of building what should be a well understood system, with well understood tooling. This doesn't mean that there isn't plenty of space for creative problem solving, particularly when the system is underspeced, it just means that the correct solution most of the time is the boring one. Most companies don't want "software artists" anymore than they want "artistic bricklayers" or "artistic aerospace engineers". What they want is predictable, maintainable error free software that still works when the "artist" moves on. That doesn't mean that your not free to upload a ton of artistic software to your github, or have opinions about how something should be designed. It just means that a professional should choose the accepted method over the fun one when given the chance. And why shouldn't they? In very few cases are "software engineers" being paid to work on their own pet projects, the end result is going to be something that the company owns and is responsible for, not the individual working for said company.
- relativeadv 6y ago
  "What they want is predictable, maintainable error free software that still works when the "artist" moves on." How about fast software? To get fast software today. You're going to need some artists.
  - WJW 6y ago
    I think the evidence shows that "fast" software is not actually desired by the majority of companies, otherwise they would put more emphasis on using compiled languages, optimisation and reducing technical debt. Sure, everybody will take a speed improvement if it's free but most companies will go for predictable, maintainable and error free software over speed every time.
    - jimbokun 6y ago
      What matters for companies is whether they can accomplish tasks faster than they can do them today. And it's rare that the difference between executing Python or C code is the bottleneck in that process.
majormajor 6y ago
> What is at risk by not allowing developers to "learn from mistakes" is autonomy. Striping developers of their autonomy is the primary cause of poor performance, not an inability to execute so-called "best practices" I've seen a lot of the opposite. Yes, coding is a design practice, but I've had to clean up a lot of messes resulting from just plain bad design because nobody involved - generally ~25 year olds with very little experience out of school - knew that there were lessons from the past they could learn about what designs would and wouldn't work. I agree with you that programming is an endeavor that benefits from experience, and wish that people would realize that means they can learn from the experience of others. Sure, intuition is involved too, but one common thing I've seen in shitty code I've had to salvage is that people often don't apply their intuition to "how could this code fail" or "how easy will this be to modify in the future"? That said... taking a look at this book... I don't see much in the description or table of contents that would teach those folks whose work I'm decrying above much useful about writing good software. It has sections on reliability, project estimation, and development methodology as separate things - plus a lot of non-software-design stuff. But to me, the flow is different - estimation, reliability, and delivery will all suffer if you don't have the right fundamental design skills. You can't get much better at any of those without some deeper underlying changes. It seems to have a lot of discussion of studies adjacent to software-related things, but I'm not sold on them saying much meaningful about software design.
- hutzlibu 6y ago
  "and delivery will all suffer if you don't have the right fundamental design skills" Some people will never get them. And maybe do not have to, if they are not involved in the complex design process(too much), but rather the implementation and testing of well defined small modules. So I believe it is very important to let people do, what their talents are.
TrinaryWorksToo 6y ago
Actually, the book agrees with you. There's very little evidence to support having a ridgid set of rules to follow when programming. There are a few high-level take-aways from the recurring patterns seen in the analysis; these include: • there is little or no evidence for many existing theories of software engineering, • most software has a relatively short lifetime, e.g., source code is deleted, packages are withdrawn or replaced, and software systems cease to be supported. A cost/benefit analysis of an investment intended to reduce future development costs needs to include the possibility that there is no future development; see fig 4.24, fig 5.7, fig 5.52, fig 5.69, fig 6.9, fig 3.31. Changes within the ecosystems in which software is built also has an impact on the viability of existing code; see fig 4.13, fig 4.22, fig 4.59, fig 4.61, fig 11.76, • software developers should not be expected to behave according to this or that mathematical ideal. People come bundled with the collection of cognitive abilities and predilections that enabled their ancestors, nearly all of whom lived in stone-age communities, to reproduce; see chapter 2.
- aard 6y ago
  This is good to know. I should jump into the book. I may have gotten the wrong impression from the summary -- the quotes that I shared. I think a scientific effort to look at the many practices that are currently in vogue is very much needed. So many are taken as gospel truths and lorded over people in the name of science-- but they really aren't supported by any rigorous science at all. If this book points that out, than I am all for it. Especially if the book has more a system thinking approach. Many studies isolate one practice (pair programming, code reviews, etc..) and can show benefits, but they ignore the systems they function in. Apparently opposite approaches, supported by the right personalities and environments can often be equally effective. From your comment, it looks like there may be some analysis like that too.
  - deleted 6y ago
    [deleted]
  - TrinaryWorksToo 6y ago
    I like reading about ESE (empircal/evidence-based software engineering) for precisely these reasons. For example, someone found that SOLID programming didn't offer much of a benefit to code readability. This is my favorite talk on it: https://www.hillelwayne.com/talks/what-we-know-we-dont-know/ https://www.hillelwayne.com/talks/what-we-know-we-dont-know/
908B64B197 6y ago
I read that sentence as "Failures should be documented. If something failed in the past and the underlying issue why it failed is still there, it's useless to attempt it". Ex: Using language Y for X was a disaster because of Y didn't have hardware acceleration and we were unable to reach goal Z. Before attempting to use language Y in production again, make sure platform support has improved.
- WJW 6y ago
  I wonder how many confounding factors there are though. At one point when I had just joined a companyy, a colleague in the process of leaving told me very confidently "at our scale it is not possible to do X anymore". Obviously, we got X working the very next week. What if using language Y for X was a disaster because of poor programming skills or micromanagers? In my experience it is very rarely as clear cut as "Y does not have hardware acceleration support" and there is not really enough rigour in the software engineering process to really figure out where a failure came from.
  - 908B64B197 6y ago
    I also agree with that. I think it's more about infusing an engineering mindset where there is an analysis of past failures.
mekoka 6y ago
I didn't get the feeling that the author disagreed that programming has a crafty nature. But since building software also has some engineering aspects, he is pointing out that you (as an organization or project leader) could wait for the graduates that you hire fresh out of uni to mature into accomplished "craftsmen", or you could just observe what has already worked for others, based on evidence and mindlessly apply those patterns. The mileage may vary on your returns from the latter approach, but it will likely be cheaper and will remove some uncertainty from your projections. Speaking as someone who has observed the kind of massive technical debt you can incur from letting your programmers mature one mistake at a time and ten reinvented wheels later, I would certainly not be too quick to dismiss the proposal. > "The labour of the cognitariate is the means of production of software systems" I fail to understand why you would disagree here. That sentence to me means that producing software systems requires brain work. Whether thinking, designing, coding, testing, debugging, reviewing, it's all part of the "labour of the cognitariate". Compiling isn't.
- aard 6y ago
  You have a valid point here. Maybe I was too hasty to judge the book by it's summary. From other comments, it sounds like this book takes a pretty reasonable approach. I am frequently skeptical of efforts to show the "one true way" of programming, because it usually results in some poorly vetted process being forced on my at work. So, I was probably too quick to jump to conclusions. It actually sounds like this book is good about showing that a lot of our current assumptions about good process are faulty. The point I was trying to make about the "labour of the cognitariate" was that developer doesn't really produce software, they create the blue-print for the software (source code), and then compilers or interpreters produce the actually software. It may seem like an overly semantic point, but I think it is an important distinction to make. It changes they way you think about software development.
dreamer7 6y ago
Taking a chess analogy - every new chess player is taught the same basic principles - control the center, develop your pieces and king safety. Also, some endgames are theoretical. A queen and king against a king and pawn on the 7th rank will be solved the same way by everyone including Grandmasters. At GM-level there's also significant opening theory which has to be learnt. Yet, everyone has fun playing chess and even Grandmasters continue to be mesmerized by it because of the complexity - there is always something new to learn. Also, GMs have access to the top chess engines and can be studying using the same engine but have radically different styles. They live for the parts of their games that are novel and so do we. If all of our development was rigidly defined, we would just automate it. We put up with the boring parts of our development because we know there'll be parts of it that we can design. If you are a hobbyist developer or a student, by all means, explore on your own and learn from your own mistakes. But that is not efficient if you want to get to the forefront. No GM today will refuse to learn opening theory or study the endgames. We will still create magic and have fun.
deleted 6y ago
[deleted]
bane 6y ago
Many years ago I would 110% agree that developing software was more art than science, and that the average software developer was working in a creative medium. But those days are generally over for the vast majority of software engineering needs. But this was in the days when memory was available in kb, the OS was a thin veneer over the hardware, your choices of languages were more which dialect of an machine language you worked in, and the fact that software worked at all was a miracle. These days, with virtually unlimited memory, storage, and extremely high software reuse, the industry has been effectively bootstrapped to the point that the average software engineer is more of a tradesman than an artist - a plumber or an electrician, who's job it is to fit standard components into the space provided for by the business logic. While there are elements of creativity to that job, it's no longer a truly creative medium for most jobs. This in a broad sense was the miracle of Java. That's not to say that there still isn't the room for beautiful, elegant, code. But those tasks are rare, and they tend to target some low-level highly reusable code that the digital plumbers can reuse over and over without any nod to artistry. If assembly line levels of engineering are ever achievable in software engineering, then it's most likely that software engineering can simply be mostly or entirely automated and the "engineer" disappears. But that's not where the field is yet. When that's achieved (maybe in the near future!), a user of a computer can simply ask the computer to build software to perform some function based on some conversational Q&A and and entire, scalable software stack will appear out of the ether. "Computer, I need to..." and software will appear that does that.
pc86 6y ago
Sometimes software development is just cranking out a widget that you need for some line-of-business app, that isn't going to need to scale, and will never see the cloud, and will just sit there used and unchanged until the tech underneath it changes. There is a lot of software that fits into this category, done by a lot of people. Not everything gets an "Artisanally crafted with <3 by $NAME" footer on it, or is solving some novel problem of mathematics or CS. > Striping developers of their autonomy is the primary cause of poor performance, not an inability to execute so-called "best practices" Citation needed for this one, I've worked with more than a couple mid-level and "senior" developers who I would have sworn had never read a single article about best practices for stuff they did every day. > In software, production is negligible. Which promotes the misconception that developers are producing software. In actuality they are designing software. The difference may seem subtle, but it is crucial. This I agree with entirely but only for probably the top 20-25% or so of software. (Top as in most creative or most novel, not best).

by aard 6y ago

Jtsummers 6y ago

It really depends on the kind of systems and work you're doing. In my previous office, there was an entire group (about 100-150) of programmers whose job was incredibly rote. That is, you could take a novice out of college and get them up to speed in about 1-3 months to do even their most complex work. However, the other groups were much less factory-ish, though rarely anything truly novel. Only a small cadre of programmers were working on anything that really required novelty and creativity. It's a spectrum and that has to be understood by all participants in the discussion. Managers want everything to be like that first group, because it's so consistent and predictable. They want to know that a project will take 1000 man-hours and be right 95% of the time. Many programmers want to see themselves part of the last group. The reality is, most of us are in the middle. There are aspects of the job that are almost mechanical, and aspects which require greater creativity or "craft". If we can clarify that for the managers, it can go a long way to ending some of the nonsense.

0x445442 6y ago
I recognized this 20 years ago but my recognition was crude. What I noticed early on, in almost every place I worked, was most of what could be called office politics for software devs/engineers boiled down to maneuvering to work on API's or even some place lower on the call stack.

StillBored 6y ago

Sorry to be the killjoy here, but This is a big mischaracterization of what "software engineering" is attempting. In this case it is an attempt to document what should be learned from experience and intuition. Because, the vast majority of software engineering IS "factory" work because its the grunt work of building what should be a well understood system, with well understood tooling. This doesn't mean that there isn't plenty of space for creative problem solving, particularly when the system is underspeced, it just means that the correct solution most of the time is the boring one. Most companies don't want "software artists" anymore than they want "artistic bricklayers" or "artistic aerospace engineers". What they want is predictable, maintainable error free software that still works when the "artist" moves on. That doesn't mean that your not free to upload a ton of artistic software to your github, or have opinions about how something should be designed. It just means that a professional should choose the accepted method over the fun one when given the chance. And why shouldn't they? In very few cases are "software engineers" being paid to work on their own pet projects, the end result is going to be something that the company owns and is responsible for, not the individual working for said company.

relativeadv 6y ago
"What they want is predictable, maintainable error free software that still works when the "artist" moves on." How about fast software? To get fast software today. You're going to need some artists.
- WJW 6y ago
  I think the evidence shows that "fast" software is not actually desired by the majority of companies, otherwise they would put more emphasis on using compiled languages, optimisation and reducing technical debt. Sure, everybody will take a speed improvement if it's free but most companies will go for predictable, maintainable and error free software over speed every time.
  - jimbokun 6y ago
    What matters for companies is whether they can accomplish tasks faster than they can do them today. And it's rare that the difference between executing Python or C code is the bottleneck in that process.

majormajor 6y ago

> What is at risk by not allowing developers to "learn from mistakes" is autonomy. Striping developers of their autonomy is the primary cause of poor performance, not an inability to execute so-called "best practices" I've seen a lot of the opposite. Yes, coding is a design practice, but I've had to clean up a lot of messes resulting from just plain bad design because nobody involved - generally ~25 year olds with very little experience out of school - knew that there were lessons from the past they could learn about what designs would and wouldn't work. I agree with you that programming is an endeavor that benefits from experience, and wish that people would realize that means they can learn from the experience of others. Sure, intuition is involved too, but one common thing I've seen in shitty code I've had to salvage is that people often don't apply their intuition to "how could this code fail" or "how easy will this be to modify in the future"? That said... taking a look at this book... I don't see much in the description or table of contents that would teach those folks whose work I'm decrying above much useful about writing good software. It has sections on reliability, project estimation, and development methodology as separate things - plus a lot of non-software-design stuff. But to me, the flow is different - estimation, reliability, and delivery will all suffer if you don't have the right fundamental design skills. You can't get much better at any of those without some deeper underlying changes. It seems to have a lot of discussion of studies adjacent to software-related things, but I'm not sold on them saying much meaningful about software design.

hutzlibu 6y ago
"and delivery will all suffer if you don't have the right fundamental design skills" Some people will never get them. And maybe do not have to, if they are not involved in the complex design process(too much), but rather the implementation and testing of well defined small modules. So I believe it is very important to let people do, what their talents are.

TrinaryWorksToo 6y ago

Actually, the book agrees with you. There's very little evidence to support having a ridgid set of rules to follow when programming. There are a few high-level take-aways from the recurring patterns seen in the analysis; these include: • there is little or no evidence for many existing theories of software engineering, • most software has a relatively short lifetime, e.g., source code is deleted, packages are withdrawn or replaced, and software systems cease to be supported. A cost/benefit analysis of an investment intended to reduce future development costs needs to include the possibility that there is no future development; see fig 4.24, fig 5.7, fig 5.52, fig 5.69, fig 6.9, fig 3.31. Changes within the ecosystems in which software is built also has an impact on the viability of existing code; see fig 4.13, fig 4.22, fig 4.59, fig 4.61, fig 11.76, • software developers should not be expected to behave according to this or that mathematical ideal. People come bundled with the collection of cognitive abilities and predilections that enabled their ancestors, nearly all of whom lived in stone-age communities, to reproduce; see chapter 2.

aard 6y ago
This is good to know. I should jump into the book. I may have gotten the wrong impression from the summary -- the quotes that I shared. I think a scientific effort to look at the many practices that are currently in vogue is very much needed. So many are taken as gospel truths and lorded over people in the name of science-- but they really aren't supported by any rigorous science at all. If this book points that out, than I am all for it. Especially if the book has more a system thinking approach. Many studies isolate one practice (pair programming, code reviews, etc..) and can show benefits, but they ignore the systems they function in. Apparently opposite approaches, supported by the right personalities and environments can often be equally effective. From your comment, it looks like there may be some analysis like that too.
- deleted 6y ago
  [deleted]
- TrinaryWorksToo 6y ago
  I like reading about ESE (empircal/evidence-based software engineering) for precisely these reasons. For example, someone found that SOLID programming didn't offer much of a benefit to code readability. This is my favorite talk on it: https://www.hillelwayne.com/talks/what-we-know-we-dont-know/ https://www.hillelwayne.com/talks/what-we-know-we-dont-know/

908B64B197 6y ago

I read that sentence as "Failures should be documented. If something failed in the past and the underlying issue why it failed is still there, it's useless to attempt it". Ex: Using language Y for X was a disaster because of Y didn't have hardware acceleration and we were unable to reach goal Z. Before attempting to use language Y in production again, make sure platform support has improved.

WJW 6y ago
I wonder how many confounding factors there are though. At one point when I had just joined a companyy, a colleague in the process of leaving told me very confidently "at our scale it is not possible to do X anymore". Obviously, we got X working the very next week. What if using language Y for X was a disaster because of poor programming skills or micromanagers? In my experience it is very rarely as clear cut as "Y does not have hardware acceleration support" and there is not really enough rigour in the software engineering process to really figure out where a failure came from.
- 908B64B197 6y ago
  I also agree with that. I think it's more about infusing an engineering mindset where there is an analysis of past failures.

mekoka 6y ago

I didn't get the feeling that the author disagreed that programming has a crafty nature. But since building software also has some engineering aspects, he is pointing out that you (as an organization or project leader) could wait for the graduates that you hire fresh out of uni to mature into accomplished "craftsmen", or you could just observe what has already worked for others, based on evidence and mindlessly apply those patterns. The mileage may vary on your returns from the latter approach, but it will likely be cheaper and will remove some uncertainty from your projections. Speaking as someone who has observed the kind of massive technical debt you can incur from letting your programmers mature one mistake at a time and ten reinvented wheels later, I would certainly not be too quick to dismiss the proposal. > "The labour of the cognitariate is the means of production of software systems" I fail to understand why you would disagree here. That sentence to me means that producing software systems requires brain work. Whether thinking, designing, coding, testing, debugging, reviewing, it's all part of the "labour of the cognitariate". Compiling isn't.

aard 6y ago
You have a valid point here. Maybe I was too hasty to judge the book by it's summary. From other comments, it sounds like this book takes a pretty reasonable approach. I am frequently skeptical of efforts to show the "one true way" of programming, because it usually results in some poorly vetted process being forced on my at work. So, I was probably too quick to jump to conclusions. It actually sounds like this book is good about showing that a lot of our current assumptions about good process are faulty. The point I was trying to make about the "labour of the cognitariate" was that developer doesn't really produce software, they create the blue-print for the software (source code), and then compilers or interpreters produce the actually software. It may seem like an overly semantic point, but I think it is an important distinction to make. It changes they way you think about software development.

dreamer7 6y ago

Taking a chess analogy - every new chess player is taught the same basic principles - control the center, develop your pieces and king safety. Also, some endgames are theoretical. A queen and king against a king and pawn on the 7th rank will be solved the same way by everyone including Grandmasters. At GM-level there's also significant opening theory which has to be learnt. Yet, everyone has fun playing chess and even Grandmasters continue to be mesmerized by it because of the complexity - there is always something new to learn. Also, GMs have access to the top chess engines and can be studying using the same engine but have radically different styles. They live for the parts of their games that are novel and so do we. If all of our development was rigidly defined, we would just automate it. We put up with the boring parts of our development because we know there'll be parts of it that we can design. If you are a hobbyist developer or a student, by all means, explore on your own and learn from your own mistakes. But that is not efficient if you want to get to the forefront. No GM today will refuse to learn opening theory or study the endgames. We will still create magic and have fun.

deleted 6y ago

[deleted]

bane 6y ago

Many years ago I would 110% agree that developing software was more art than science, and that the average software developer was working in a creative medium. But those days are generally over for the vast majority of software engineering needs. But this was in the days when memory was available in kb, the OS was a thin veneer over the hardware, your choices of languages were more which dialect of an machine language you worked in, and the fact that software worked at all was a miracle. These days, with virtually unlimited memory, storage, and extremely high software reuse, the industry has been effectively bootstrapped to the point that the average software engineer is more of a tradesman than an artist - a plumber or an electrician, who's job it is to fit standard components into the space provided for by the business logic. While there are elements of creativity to that job, it's no longer a truly creative medium for most jobs. This in a broad sense was the miracle of Java. That's not to say that there still isn't the room for beautiful, elegant, code. But those tasks are rare, and they tend to target some low-level highly reusable code that the digital plumbers can reuse over and over without any nod to artistry. If assembly line levels of engineering are ever achievable in software engineering, then it's most likely that software engineering can simply be mostly or entirely automated and the "engineer" disappears. But that's not where the field is yet. When that's achieved (maybe in the near future!), a user of a computer can simply ask the computer to build software to perform some function based on some conversational Q&A and and entire, scalable software stack will appear out of the ether. "Computer, I need to..." and software will appear that does that.

pc86 6y ago

Sometimes software development is just cranking out a widget that you need for some line-of-business app, that isn't going to need to scale, and will never see the cloud, and will just sit there used and unchanged until the tech underneath it changes. There is a lot of software that fits into this category, done by a lot of people. Not everything gets an "Artisanally crafted with <3 by $NAME" footer on it, or is solving some novel problem of mathematics or CS. > Striping developers of their autonomy is the primary cause of poor performance, not an inability to execute so-called "best practices" Citation needed for this one, I've worked with more than a couple mid-level and "senior" developers who I would have sworn had never read a single article about best practices for stuff they did every day. > In software, production is negligible. Which promotes the misconception that developers are producing software. In actuality they are designing software. The difference may seem subtle, but it is crucial. This I agree with entirely but only for probably the top 20-25% or so of software. (Top as in most creative or most novel, not best).