Lots of excellent science is done without double-blind experiments. Not all experiments need double blind. All of the Pluto flyby researchers know their data co

Lots of excellent science is done without double-blind experiments. Not all experiments need double blind. All of the Pluto flyby researchers know their data comes from Pluto. You'll mostly see the phrase 'gold standard for science' in biomedical and social science fields where subjective evaluation plays, or may play, an important role. While these are studies are definitely in social science, they mostly stuck with objective values which cannot easily be affected by an un-blinded experimenter. Here's what a quick scan of some of the papers, which you could have done yourself. The paper "Static type systems (sometimes) have a positive impact on the usability of undocumented software" uses randomly assigned test subjects and a measurement - time - which does not have a subjective component that would be affected by blinding the experimenter. The same with "How Do API Documentation and Static Typing Affect API Usability?", which also looked at other objective parameters. "An Experiment About Static and Dynamic Type Systems" also used time, as well as the ability to pass a set of test cases which were uniformly applied. They considered using code reviews, but the subjectiveness requires a large number of reviewers to hope to get an interpretable number. Had they gone the code review route, yes, I think the reviewers would also need to be blinded. "Work In Progress: an Empirical Study of Static Typing in Ruby" was a pilot study which had no experimental numbers, and was mostly meant for hypothesis generation and working out kinks in the study protocol. "Haskell vs. Ada vs. C++ vs. Awk vs. ... An Experiment in Software Prototyping Productivity" used LoC, development time, and subjective reviewers, noting the issues with subjective reviews, but curiously omitting the low statistical confidence. Blinding wouldn't have made a difference in the reviews as all programs but Haskell were written in a different languages. (Two were in Haskell.) FWIW, double-blind studies are not immune to p-hacking, for that you need a preregistered study. I guess a pre-registered triple-blind study would be platinum standard?