3 ms·

Its worse when you realize that most stuff published is focused on results, not disproving an idea, but rather proving some idea or discovery. Even worse is ho

Its worse when you realize that most stuff published is focused on results, not disproving an idea, but rather proving some idea or discovery. Even worse is how limiting it is, when the cost of materials and equipment means the only people who can fact check your paper via trying to reproduce your results are basically other institutions, or in the extremely rare case it might be an industry partner. I'm interested in semiconductors from design to fabrication. For design I can use openlane on Linux but AFAIK that dosnt support small feature sizes, and I will need to manually build my own lab. I can't afford any expensive equipment so my best bet is just get a high paying job in my field I'm a student in, which is cyber security, and hope I can save money over the years and manage to buy up used equipment. This also means I basically have to spend a large portion of my life working to afford a basic lab and equipment that's old and outdated. And safety, well its obviously important but at a certain point i can't justify the fancy safety equipment if it costs so much. Especially if it has a limited life like the HF acid exposure kits that have the cream to help save your life. Or SCBA isn't cheap and has a maintenance cost. Unless I am willing to dedicate my life to this niche, or can find a cybersecurity job that involves chip fans.....I'm SOL. TLDR, certain research and discovery is limited to well funded universities which means you centralized the progress to a specific group: higher education labs and industry funded research partners

riedel 2y ago
Publishing in much less visible venues than nature, the problem to me is that we do not often have to the actual resources to live by our own standards. Only few publications are altered by direct instructions by a lab lead IMHO. But I think still quite a lot are deeply flawed. Mostly it is the pressure on PhD students to get results which at least in our domain makes research likely not to be reproducible. And everyone else in the lab just wants to believe in the findings. IMHO we need huge bug bounty programs for research to incentivise finding the defects additionally to theoretical reproducibility. In the 20 years I have been in CS research, I haven't seen a reviewer trying to reproduce a result.
- tonetegeatinst 2y ago
  Some of what you mentioned flew over my head, but I agree that in many cases journals don't seem to even validate stuff. Some of the machine learning papers in the AI field are a bit better, they are disclosing the training data and in some cases might even link the juniper notebook for reproducibility. Modern AI seems to lack the open honesty in what training data was used, which while I fundamentally dislike, I also see it as anticompetitive. The inability to validate the methodology used, or to scrutinize the training data is what disturbs me. While I personally think that AI that has been dumbed down with safeguards and censored is harmful not only to the user but also the development of future progress, the inability to analyze what data is being trained on is concerning as backdoors and bias can be purposely introduced that harms both people and privacy. Take a look at this chemistry video, disclosure that I'm subbed to this channel and many like it. https://www.youtube.com/watch?v=hVJXOi933cc https://www.youtube.com/watch?v=hVJXOi933cc One major thing that some might gloss over is the sheer cost of stuff. The benefit if seen from PHD researchers seems to be the ability to research topics or ideas that have no immediate commercial use. I agree they defiantly focus and pressure these people on publishing results....and that between science youtubers and businesses people, that the implications of the research gets stretched to seem like a miracle or some grand discovery. While those are cool, its also just as important to make the little discovery because it broadens our understanding of reality and how everything works and what is possible. Check out this really good explanation regarding this problem. https://www.youtube.com/watch?v=czjisEGe5Cw https://www.youtube.com/watch?v=czjisEGe5Cw

riedel 2y ago

Publishing in much less visible venues than nature, the problem to me is that we do not often have to the actual resources to live by our own standards. Only few publications are altered by direct instructions by a lab lead IMHO. But I think still quite a lot are deeply flawed. Mostly it is the pressure on PhD students to get results which at least in our domain makes research likely not to be reproducible. And everyone else in the lab just wants to believe in the findings. IMHO we need huge bug bounty programs for research to incentivise finding the defects additionally to theoretical reproducibility. In the 20 years I have been in CS research, I haven't seen a reviewer trying to reproduce a result.

tonetegeatinst 2y ago
Some of what you mentioned flew over my head, but I agree that in many cases journals don't seem to even validate stuff. Some of the machine learning papers in the AI field are a bit better, they are disclosing the training data and in some cases might even link the juniper notebook for reproducibility. Modern AI seems to lack the open honesty in what training data was used, which while I fundamentally dislike, I also see it as anticompetitive. The inability to validate the methodology used, or to scrutinize the training data is what disturbs me. While I personally think that AI that has been dumbed down with safeguards and censored is harmful not only to the user but also the development of future progress, the inability to analyze what data is being trained on is concerning as backdoors and bias can be purposely introduced that harms both people and privacy. Take a look at this chemistry video, disclosure that I'm subbed to this channel and many like it. https://www.youtube.com/watch?v=hVJXOi933cc https://www.youtube.com/watch?v=hVJXOi933cc One major thing that some might gloss over is the sheer cost of stuff. The benefit if seen from PHD researchers seems to be the ability to research topics or ideas that have no immediate commercial use. I agree they defiantly focus and pressure these people on publishing results....and that between science youtubers and businesses people, that the implications of the research gets stretched to seem like a miracle or some grand discovery. While those are cool, its also just as important to make the little discovery because it broadens our understanding of reality and how everything works and what is possible. Check out this really good explanation regarding this problem. https://www.youtube.com/watch?v=czjisEGe5Cw https://www.youtube.com/watch?v=czjisEGe5Cw