3 ms·

No, I think his point is valid. Some of the biggest errors are subtle failures in the implementation of business logic. Cases where what the code does and what

by awj 9y ago

No, I think his point is valid. Some of the biggest errors are subtle failures in the implementation of business logic. Cases where what the code does and what it looks like it does fail to line up. Static typing can be brought to bear on those (encode your business logic in the type system), but what parts that fails to cover still need good testing. Plus, how do you test that your types correctly encode your business logic? I'm still more in the static camp than out of it, but I can recognize that the problem is not as cut and dried as we want to believe. There's are reasons this pendulum has swung back and forth over the decades, and I don't see anything to suggest those reasons have changed.

platz 9y ago
> how do you test that your types correctly encode your business logic types, as most commonly used do not _directly_ encode business logic; they just provide constraints i.e. it is not an all-or-nothing situation. You can still write unit tests.
- yogthos 9y ago
  What you have to demonstrate is that static typing catches a statistically significant amount of additional errors over plain unit tests. Nobody has been able to show this to be the case so far in practice.
  - platz 9y ago
    Requiring statistically significant studies pretty much guarantees no provable claims can be made for any side; since that amounts to a social science laden with shaky assumptions and ambiguous modeling. Since it's is a standard of proof that no-one can achieve, i don't see why mention it here. But to go along with this for a moment, I liked your response to this paper here https://www.reddit.com/r/Clojure/comments/73q8c2/a_largescale_study_of_programming_languages_and/dnsqypt/ https://www.reddit.com/r/Clojure/comments/73q8c2/a_largescal... ; Though I'm not sure how we're deciding what is "statistically significant" in contrast to what the researchers have decided is statistically significant. Were their calculations incorrect? However, you claim to reveal their "real conclusion", in contrast to their Actual Conclusion, in which they do indicate that "Among functional languages, static typing is also somewhat better than dynamic typing." Still, I do not place much stock in these studies for the aforementioned reasons. I also think you have to be careful about where you have placed the "burden of proof". (You've placed it on requiring poof that static types provide X benefit. One can just as well turn this around and require poof that eschewing static types provide X benefit; so we cannot a-priori place such demands in either direction).
    - yogthos 9y ago
      The null hypothesis has to be that both approaches are equally effective. There simply isn't any evidence to suggest otherwise. At this point, it's premature to discuss whether static typing or dynamic typing affords benefits. The first step would be to look at a large set of real world open source projects written in different languages. If we see empirical evidence that projects written in certain types of languages consistently perform better in a particular area, such as reduction in defects, we can make a hypothesis as to why that is. For example, if there was statistical evidence to indicate that using Haskell reduces defects, a hypothesis could be made that the the Haskell type system plays a role here. That hypothesis could then be further tested, and that would tell us whether it's correct or not. I agree that these studies are hard to do, and that they'll always be imperfect. However, that's still the best tool we have for approaching this empirically.
      - platz 9y ago
        > The null hypothesis has to be that both approaches are equally effective. There simply isn't any evidence to suggest otherwise. Sure, I think that is in the ballpark of an acceptable null hypothesis, for a given experiment. I'm not sure one can identify a null hypothesis that is universal among all experiments, but this captures the idea, sure.
- awj 9y ago
  Ehh, semantics. "How do you ensure that your constraints are correct?" is effectively the same question. Any nontrivial set of constraints will need to be validated, and validating those constraints is a pretty big challenge that the compiler by definition cannot help you with.

by awj 9y ago

platz 9y ago

> how do you test that your types correctly encode your business logic types, as most commonly used do not _directly_ encode business logic; they just provide constraints i.e. it is not an all-or-nothing situation. You can still write unit tests.

yogthos 9y ago
What you have to demonstrate is that static typing catches a statistically significant amount of additional errors over plain unit tests. Nobody has been able to show this to be the case so far in practice.
- platz 9y ago
  Requiring statistically significant studies pretty much guarantees no provable claims can be made for any side; since that amounts to a social science laden with shaky assumptions and ambiguous modeling. Since it's is a standard of proof that no-one can achieve, i don't see why mention it here. But to go along with this for a moment, I liked your response to this paper here https://www.reddit.com/r/Clojure/comments/73q8c2/a_largescale_study_of_programming_languages_and/dnsqypt/ https://www.reddit.com/r/Clojure/comments/73q8c2/a_largescal... ; Though I'm not sure how we're deciding what is "statistically significant" in contrast to what the researchers have decided is statistically significant. Were their calculations incorrect? However, you claim to reveal their "real conclusion", in contrast to their Actual Conclusion, in which they do indicate that "Among functional languages, static typing is also somewhat better than dynamic typing." Still, I do not place much stock in these studies for the aforementioned reasons. I also think you have to be careful about where you have placed the "burden of proof". (You've placed it on requiring poof that static types provide X benefit. One can just as well turn this around and require poof that eschewing static types provide X benefit; so we cannot a-priori place such demands in either direction).
  - yogthos 9y ago
    The null hypothesis has to be that both approaches are equally effective. There simply isn't any evidence to suggest otherwise. At this point, it's premature to discuss whether static typing or dynamic typing affords benefits. The first step would be to look at a large set of real world open source projects written in different languages. If we see empirical evidence that projects written in certain types of languages consistently perform better in a particular area, such as reduction in defects, we can make a hypothesis as to why that is. For example, if there was statistical evidence to indicate that using Haskell reduces defects, a hypothesis could be made that the the Haskell type system plays a role here. That hypothesis could then be further tested, and that would tell us whether it's correct or not. I agree that these studies are hard to do, and that they'll always be imperfect. However, that's still the best tool we have for approaching this empirically.
    - platz 9y ago
      > The null hypothesis has to be that both approaches are equally effective. There simply isn't any evidence to suggest otherwise. Sure, I think that is in the ballpark of an acceptable null hypothesis, for a given experiment. I'm not sure one can identify a null hypothesis that is universal among all experiments, but this captures the idea, sure.
awj 9y ago
Ehh, semantics. "How do you ensure that your constraints are correct?" is effectively the same question. Any nontrivial set of constraints will need to be validated, and validating those constraints is a pretty big challenge that the compiler by definition cannot help you with.