4 ms·
The type system question is different from the psychological examples: the problem is not people misinterpreting evidence, but that reliable empirical evidence
by tikhonj 10y ago
The type system question is different from the psychological examples: the problem is not people misinterpreting evidence, but that reliable empirical evidence simply does not exist. Papers on the matter are sparse, completely uneven and full of methodological issues.
Personally, I'd argue that "statically typed" vs "dynamically typed" does not even make sense as a single question. There's more difference between Haskell and Java than between Java and Python, and an experiment comparing two identical languages with and without static typing won't tell us much beyond those two languages. (I recall seeing at least one paper that did this; it's probably worth reading, but not for making broader conclusions.)
Moreover, there simply isn't a compelling way to measure most of the things that programmers actually care about like expressiveness, productivity or safety. Existing measures (like counting bugs, lines of code over time, experiments on small tasks often performed by students... etc) are limited, full of confounding variables and quite indirect in measuring what people actually care about. I've looked through various studies and experiments in software engineering and while some are compelling, many are little more than dressed-up anecdotes or "experience reports".
It's especially hard to study these things in the contexts that matter. What we care about is experienced programmers who've used specific technologies for years applying them in teams, at scale. What's easy to experiment on is people who've just learned something using it on tiny tasks. Observational studies that aim at the broader industry context are interesting but hard to generalize because of confounding variables and difficulty of measurement.
In the absence of this sort of evidence, people have to make decisions somehow, and it's not surprising that they overstate their confidence. We see this in pretty much everything else that doesn't have a strong empirical basis like questions around organizing workplaces, teams and processes. Just look at opinions people have about open offices, specific agile processes or interview procedures!
Another side to the question is that languages inevitably have a strong aesthetic component, and talking about aesthetics is difficult. But you're certainly not going to convince anyone on aesthetic matters with an experiment or observational study, any more than you can expect to accomplish anything like that in the art world!