7 ms·

Absolutely. The biggest offenders in software engineering are the things people integrate into their identity (i.e. "I'm an Agile manager", "I'm a Java architec

by andreyf 16y ago

Absolutely. The biggest offenders in software engineering are the things people integrate into their identity (i.e. "I'm an Agile manager", "I'm a Java architect", "I'm a Lisp hacker"). Another red flag is when there are no clear ways of testing a hypothesis (i.e. "NoSQL databases are better for this task", "This management methodology is best for our organization", "Outsourcing is great if you do so-and-so"). Most, if not all, of the opinions regarding industrial software engineering are based in anecdotal experience, without the tiniest thought given to controlled experiments. C'est la vie. PS: what's most confusing, for me, is that it seems impossible to fix all of the variables necessary to answer a question like "what technology is best for this real-world problem?". I've yet to hear of any good ways of systematically measuring the effect of technology choices on programmer productivity.

moultano 16y ago
We aren't doing physics here. If we can't observe the effects of something easily, it probably isn't that important.
- natep 16y ago
  Actually, that is similar to the position of a physicist, just without the words "easily" and "probably". For instance, what happened before the big bang has no effect on the current universe, which is why there's so much research into the first moments of the big bang, and not into any point in time before then. But to address your comment specifically, I couldn't disagree more. Unless, of course you're just trolling by making a rediculous statement without any backing evidence in a discussion about pseudoscience, in which case, well done. First, all of the parent's examples are hypotheses that are hard to test, not hard to observe. It's hard to test just how much better NoSQL/SQL is for a given task, but not hard to see that there are differences in performance if you were to, say, implement both systems. I'd like to see you change the management methodology for a company and not see a change. Same goes for his outsourcing example. Second, I do not think there is a logical connection between your conclusion and your premise. Since you haven't given a single iota or proof for your claim, I don't think I need any proof to counter it. Finally, even if we aren't doing "physics" (what ever your definition of that is), why does that mean pseudoscience is acceptable?
  - Tichy 16y ago
    Maybe there is no strict ordering for those technologies. Or maybe what works best depends on the individuals involved. Also, there is the "no silver bullet" theory.
    - natep 16y ago
      You and I agree. Sorry if I didn't make that clear, but the way I read the post at the top of this thread, each claim could be backed with appropriate evidence for and against (as Feynman says every scientist's duty is). For a sufficiently well defined situation X, the claim that "Y is the most appropriate solution for X" can and should be tested appropriately. The "appropriate" amount of testing would also depend on the situation X. The 'difficulty of observation' may be a factor here, but is not the only one. If the situation doesn't require 'hard' evidence (e.g. the consequences are minor) that's fine, but I wouldn't say any of the above examples fall under that category, and people making decisions based on 'soft' evidence should do so knowing that it is what it is. Also, whether there is or isn't a strict ordering, dependence on the individuals, or a silver bullet would all be facts turned up by sufficient application of the scientific method. This relates to Feynman's points on how it's necessary to do experiments where you "don't learn anything new" and again, how every scientist is ethically bound to report results that contradict their theory.
- InclinedPlane 16y ago
  That's the kind of thinking that leads you into wondering how the hell your code base got so bad, how you fell so far behind the competition, etc. A lot of critically important aspects of modern software engineering and craftsmanship are extremely subjective. The difference between success and failure in software can often come down to having the right group of people with good judgment in charge.
stellar678 16y ago
This is also the reason that fields like nutrition science aren't generally all that useful if what you want to do something like offer diet advice. There are just too many shifting pieces to offer a systematic explanation of what's going on, at least with the experimental limits we have right now. You can only track so many variables at once.
- bertm 16y ago
  This presents an interesting paradox though because at the end of the day a decision is required. Do I eat thing X or thing Y? And in what proportion? So when a set of things is not well understood but a decision is required, many non science strategies enter. So what can be done about this?
  - scscsc 16y ago
    You listen to science and not to crackpots who pretend to do science. Note that most advice regarding nutrition appearing in newspapers/blogs is total crap (as opposed to science). If you can't distinguish science from crap (or can't be bothered to search for science) I would suggest you go with what seems reasonable to yourself.
    - stellar678 16y ago
      That's the whole point though - science so far can't give much useful advice on what to eat, the topic is just too complex. Science can give hedged explanations of what's been observed in relation to a particular nutrient, but that's a far shot from dietary advice. What seems to end up happening is "we've observed a correlation between saturated fat consumption and heart disease" makes it out of the science world and gets turned into "avoid saturated fat at all costs" (and buy our replacement products). Then years later we realize it's a lot more complicated than that.
      - scscsc 16y ago
        It was known at the time that correlation is not causation. That is a classic example of taking dietary advice from the incompetent.
fragmede 16y ago
I'd argue that code contests like Google Code Jam are the closest to 'controlled experiments' that we have in software engineering. There's a set of specs of what is to be implemented, and a limited time in which to do it. (With different contest structure/judging criteria to testing different things like, speed, adaptability/maintainability, etc.) By trying to entice as many programmer/programmer teams to enter, differences in individual programmer is hopefully lessened. Even your question, "(What is) the effect of technology choices on programmer productivity?", tries to treat programmers as interchangeable when tech choice A is better for all programmers. Programmer productivity, like intelligence, is something that if you'd have a license to print money if you could accurately and precisely measure.
fragmede 16y ago
I'd argue that code contests like Google Code Jam are the closest to 'controlled experiments' that we have in software engineering. There's a set of specs of what is to be implemented, and a limited time in which to do it. (With different contest structure/judging criteria to testing different things like, speed, adaptability/maintainability, etc.) By trying to entice as many programmer/programmer teams to enter, differences in individual programmer is hopefully lessened. Even your question, "(What is) the effect of technology choices on programmer productivity?", tries to treat programmers as interchangeable as if tech choice A is would make all programmers more productive. Programmer productivity, like intelligence, is something that if you'd have a license to print money if you could accurately and precisely measure.
rbanffy 16y ago
Be careful. Software Engineering is not a science. Computer science is a science but, as it is explained so well in the SICP videos I watched, compsci's relations with computers is the same as astronomy's with telescopes. Software engineering can - and should - use scientific method, but it has more to do with carpentry than with physics. And I, personally, regard medicine as the biggest offender. Homeopathy should be a criminal offense.

by andreyf 16y ago

moultano 16y ago

We aren't doing physics here. If we can't observe the effects of something easily, it probably isn't that important.

natep 16y ago
Actually, that is similar to the position of a physicist, just without the words "easily" and "probably". For instance, what happened before the big bang has no effect on the current universe, which is why there's so much research into the first moments of the big bang, and not into any point in time before then. But to address your comment specifically, I couldn't disagree more. Unless, of course you're just trolling by making a rediculous statement without any backing evidence in a discussion about pseudoscience, in which case, well done. First, all of the parent's examples are hypotheses that are hard to test, not hard to observe. It's hard to test just how much better NoSQL/SQL is for a given task, but not hard to see that there are differences in performance if you were to, say, implement both systems. I'd like to see you change the management methodology for a company and not see a change. Same goes for his outsourcing example. Second, I do not think there is a logical connection between your conclusion and your premise. Since you haven't given a single iota or proof for your claim, I don't think I need any proof to counter it. Finally, even if we aren't doing "physics" (what ever your definition of that is), why does that mean pseudoscience is acceptable?
- Tichy 16y ago
  Maybe there is no strict ordering for those technologies. Or maybe what works best depends on the individuals involved. Also, there is the "no silver bullet" theory.
  - natep 16y ago
    You and I agree. Sorry if I didn't make that clear, but the way I read the post at the top of this thread, each claim could be backed with appropriate evidence for and against (as Feynman says every scientist's duty is). For a sufficiently well defined situation X, the claim that "Y is the most appropriate solution for X" can and should be tested appropriately. The "appropriate" amount of testing would also depend on the situation X. The 'difficulty of observation' may be a factor here, but is not the only one. If the situation doesn't require 'hard' evidence (e.g. the consequences are minor) that's fine, but I wouldn't say any of the above examples fall under that category, and people making decisions based on 'soft' evidence should do so knowing that it is what it is. Also, whether there is or isn't a strict ordering, dependence on the individuals, or a silver bullet would all be facts turned up by sufficient application of the scientific method. This relates to Feynman's points on how it's necessary to do experiments where you "don't learn anything new" and again, how every scientist is ethically bound to report results that contradict their theory.
InclinedPlane 16y ago
That's the kind of thinking that leads you into wondering how the hell your code base got so bad, how you fell so far behind the competition, etc. A lot of critically important aspects of modern software engineering and craftsmanship are extremely subjective. The difference between success and failure in software can often come down to having the right group of people with good judgment in charge.

stellar678 16y ago

This is also the reason that fields like nutrition science aren't generally all that useful if what you want to do something like offer diet advice. There are just too many shifting pieces to offer a systematic explanation of what's going on, at least with the experimental limits we have right now. You can only track so many variables at once.

bertm 16y ago
This presents an interesting paradox though because at the end of the day a decision is required. Do I eat thing X or thing Y? And in what proportion? So when a set of things is not well understood but a decision is required, many non science strategies enter. So what can be done about this?
- scscsc 16y ago
  You listen to science and not to crackpots who pretend to do science. Note that most advice regarding nutrition appearing in newspapers/blogs is total crap (as opposed to science). If you can't distinguish science from crap (or can't be bothered to search for science) I would suggest you go with what seems reasonable to yourself.
  - stellar678 16y ago
    That's the whole point though - science so far can't give much useful advice on what to eat, the topic is just too complex. Science can give hedged explanations of what's been observed in relation to a particular nutrient, but that's a far shot from dietary advice. What seems to end up happening is "we've observed a correlation between saturated fat consumption and heart disease" makes it out of the science world and gets turned into "avoid saturated fat at all costs" (and buy our replacement products). Then years later we realize it's a lot more complicated than that.
    - scscsc 16y ago
      It was known at the time that correlation is not causation. That is a classic example of taking dietary advice from the incompetent.

fragmede 16y ago

I'd argue that code contests like Google Code Jam are the closest to 'controlled experiments' that we have in software engineering. There's a set of specs of what is to be implemented, and a limited time in which to do it. (With different contest structure/judging criteria to testing different things like, speed, adaptability/maintainability, etc.) By trying to entice as many programmer/programmer teams to enter, differences in individual programmer is hopefully lessened. Even your question, "(What is) the effect of technology choices on programmer productivity?", tries to treat programmers as interchangeable when tech choice A is better for all programmers. Programmer productivity, like intelligence, is something that if you'd have a license to print money if you could accurately and precisely measure.

fragmede 16y ago

I'd argue that code contests like Google Code Jam are the closest to 'controlled experiments' that we have in software engineering. There's a set of specs of what is to be implemented, and a limited time in which to do it. (With different contest structure/judging criteria to testing different things like, speed, adaptability/maintainability, etc.) By trying to entice as many programmer/programmer teams to enter, differences in individual programmer is hopefully lessened. Even your question, "(What is) the effect of technology choices on programmer productivity?", tries to treat programmers as interchangeable as if tech choice A is would make all programmers more productive. Programmer productivity, like intelligence, is something that if you'd have a license to print money if you could accurately and precisely measure.

rbanffy 16y ago

Be careful. Software Engineering is not a science. Computer science is a science but, as it is explained so well in the SICP videos I watched, compsci's relations with computers is the same as astronomy's with telescopes. Software engineering can - and should - use scientific method, but it has more to do with carpentry than with physics. And I, personally, regard medicine as the biggest offender. Homeopathy should be a criminal offense.