3 ms·

Publishing in much less visible venues than nature, the problem to me is that we do not often have to the actual resources to live by our own standards. Only fe

by riedel 2y ago

Publishing in much less visible venues than nature, the problem to me is that we do not often have to the actual resources to live by our own standards. Only few publications are altered by direct instructions by a lab lead IMHO. But I think still quite a lot are deeply flawed. Mostly it is the pressure on PhD students to get results which at least in our domain makes research likely not to be reproducible. And everyone else in the lab just wants to believe in the findings. IMHO we need huge bug bounty programs for research to incentivise finding the defects additionally to theoretical reproducibility. In the 20 years I have been in CS research, I haven't seen a reviewer trying to reproduce a result.

tonetegeatinst 2y ago
Some of what you mentioned flew over my head, but I agree that in many cases journals don't seem to even validate stuff. Some of the machine learning papers in the AI field are a bit better, they are disclosing the training data and in some cases might even link the juniper notebook for reproducibility. Modern AI seems to lack the open honesty in what training data was used, which while I fundamentally dislike, I also see it as anticompetitive. The inability to validate the methodology used, or to scrutinize the training data is what disturbs me. While I personally think that AI that has been dumbed down with safeguards and censored is harmful not only to the user but also the development of future progress, the inability to analyze what data is being trained on is concerning as backdoors and bias can be purposely introduced that harms both people and privacy. Take a look at this chemistry video, disclosure that I'm subbed to this channel and many like it. https://www.youtube.com/watch?v=hVJXOi933cc https://www.youtube.com/watch?v=hVJXOi933cc One major thing that some might gloss over is the sheer cost of stuff. The benefit if seen from PHD researchers seems to be the ability to research topics or ideas that have no immediate commercial use. I agree they defiantly focus and pressure these people on publishing results....and that between science youtubers and businesses people, that the implications of the research gets stretched to seem like a miracle or some grand discovery. While those are cool, its also just as important to make the little discovery because it broadens our understanding of reality and how everything works and what is possible. Check out this really good explanation regarding this problem. https://www.youtube.com/watch?v=czjisEGe5Cw https://www.youtube.com/watch?v=czjisEGe5Cw

by riedel 2y ago

tonetegeatinst 2y ago

Some of what you mentioned flew over my head, but I agree that in many cases journals don't seem to even validate stuff. Some of the machine learning papers in the AI field are a bit better, they are disclosing the training data and in some cases might even link the juniper notebook for reproducibility. Modern AI seems to lack the open honesty in what training data was used, which while I fundamentally dislike, I also see it as anticompetitive. The inability to validate the methodology used, or to scrutinize the training data is what disturbs me. While I personally think that AI that has been dumbed down with safeguards and censored is harmful not only to the user but also the development of future progress, the inability to analyze what data is being trained on is concerning as backdoors and bias can be purposely introduced that harms both people and privacy. Take a look at this chemistry video, disclosure that I'm subbed to this channel and many like it. https://www.youtube.com/watch?v=hVJXOi933cc https://www.youtube.com/watch?v=hVJXOi933cc One major thing that some might gloss over is the sheer cost of stuff. The benefit if seen from PHD researchers seems to be the ability to research topics or ideas that have no immediate commercial use. I agree they defiantly focus and pressure these people on publishing results....and that between science youtubers and businesses people, that the implications of the research gets stretched to seem like a miracle or some grand discovery. While those are cool, its also just as important to make the little discovery because it broadens our understanding of reality and how everything works and what is possible. Check out this really good explanation regarding this problem. https://www.youtube.com/watch?v=czjisEGe5Cw https://www.youtube.com/watch?v=czjisEGe5Cw