4 ms·
At my day job I work a lot with data, from a multitude of sources. It amazes me how hard it is to build data products (you could think of it as an analog to sci
by mtrn 11y ago
At my day job I work a lot with data, from a multitude of sources. It amazes me how hard it is to build data products (you could think of it as an analog to scientific experiments) and know exactly what parts of the processing pipeline influenced what parts in the result.
On the other hand: When I hear that many scientists use Excel for complicated processing chains and are sometimes themselves hardly able to reproduce anything a few month after the paper was published, I believe we hit some new low point in scientific activity. Combined with an infrastructure focused on research impact and number of papers published, this is really a sad situation.
- x5n1 11y agoReal science is random, see No Method. When you make it a career that has requirements and expectations you set this sort of process to take place. Still even through this process, if enough people are doing it, there is bound to be some breakthrough science. But most of the people are wasting their time cooking data to make their careers go further, rather than make progress in science.
- deleted 11y ago[deleted]
- jessaustin 11y agoYou're right that this phenomenon undermines confidence in scientific results. It also undermines truth in those results. Garbage in, garbage out.
- aledalgrande 11y agoI have not worked directly in research, but that has been my experience when trying to reproduce results from papers: it hardly works even after following each step the researcher did (confirmed via email) and a lot of conditions that are not true in the real world have to be met.
- tdaltonc 11y agoI assume you're referring to "Against Method" by Feyerabend? His argument is absolutely not that science is "random". It's that the methods of science cannot be predicted, prescribed, or universal, even over very short time scales and between similar fields. Even if Feyerabend is right [0], it's still perfectly reasonable to expect studies to be repeatable. [0] I'm sure he is but I think that the degree and practical significance of his observation is questionable.
- type0 11y agoYeah, I have seen this in person. What is more sad, is that the scientists using Excel are aware of the limitation but they just don't want to spend time learning to work with R or Python and csv files and version control systems, after all they rather spend this time for grant seeking. For them convenience trumps reproducibility, they just redo the analysis until one one could get the "right" result. This indeed is very bad for academia and drives smart people away from postgraduate school. I also encountered use of proprietary software that scientist written themselves just to prevent reproducibility and make it harder for competing research groups. How this people can get published in high impact papers is beyond my understanding.