This is very true. Systematic bias in observational data collection (astronomy etc.) as well as systematic bias in experimental data collection (particle physi

This is very true. Systematic bias in observational data collection (astronomy etc.) as well as systematic bias in experimental data collection (particle physics etc.) isn't accounted for in statistical analysis of that data. A classic example I recall is a Feynmann story, where a group of researchers were getting very statistically sound and repeatable results of very unusual and unexplained particle track behavior in a cloud chamber. Feynmann looked at the data and said "you probably have a tiny piece of metal in the cloud chamber somewhere" and that turned out to be the explanation. Similar examples in the social sciences include systematic bias in the preparation and administration of IQ tests to different groups of people (see Charles Murray's 'Bell Curve' vs. Stephen J. Gould's 'The Mismeasure of Man'). Hundreds of other examples can be found across all scientific disciplines, unfortunately. To quote the smartest PI I ever worked for "There's a lot of BS in statistical analysis".