10 ms·

To quote someone I know in Alzheimer research - “I’m pretty sure it’s all fake” It’s funny because more than once I’ve had conversations off-the-record with pe

To quote someone I know in Alzheimer research - “I’m pretty sure it’s all fake” It’s funny because more than once I’ve had conversations off-the-record with people who had the same concerns. What I think is most troubling is how long it took and how many papers support the claims. In other words, there are thousands of papers supporting beta-amyloid and/or building off the data / theory. All of that data and everyone conducting that data should be in question. Yes, sometimes you’ll see correlations even though the mechanism is not what you’d expect. That said, for 16 years? Lol No, in reality, this should indicate there is systemic fraud in the industry. I don’t think everyone’s a fraud, it’s as much about what doesn’t get published as what does get published. How does this happen? I commented on incentives prior here: https://news.ycombinator.com/reply?id=32213123 https://news.ycombinator.com/reply?id=32213123 I personally don’t see how we can advance science with the current credentialing and paper citation == academic success we’ve seen grow the last 50 years. In many ways, fundamental research has stalled, with applications of research advancing (giving us the illusion of advancing the fundamentals). To fix this problem, an overhaul is needed. Most importantly, the gate keeping mechanisms should be removed.

kurupt213 4y ago
What’s one of the first things they teach when dealing with statistical analysis? Correlation is not causation
beanjuice 4y ago
What do you mean by "the gate keeping mechanisms" should be removed? Do you mean the difficulty of publishing? Or tenure? Or degrees?
- voakbasda 4y ago
  All of those gate keeping mechanisms have systemic issues and need to be reformed at the very least. "Removed" might be a bit strong, but I definitely see how one could reach that perspective. Regardless, they certainly have lost the credibility to claim any monopolistic authority over the domains they purport to serve.
  - smcin 4y ago
    a) It's easy to suggest gatekeeping mechanisms like degrees, tenture, journals controlling publication "should all be reformed", without proposing any specific alternatives; but they all exist to prevent an obvious set of anti-patterns; if you eliminate them you just get different anti-patterns. Can you make a specific proposal of what's a better system to replace them with? b) In any case, the $$ incentives and root-causes are distorted, no point in scapegoating the gatekeeping mechanisms. The US pharmaceutical industry (for example) is 3.2% of GDP, and is 48% of the global pharmaceutical market. That amount of $$ is obviously going to distort policy at all levels. c) Scientific fraud should be criminalized, there should be stiff penalties and they should actually be enforced (actual prison, fines, losing job and tenure, retracting PhDs). Otherwise you're playing a race-to-the-bottom in a consequence-free environment, against someone else who's going to do unethical things. d) Until you address the root-causes/incentives, people's behavior won't change, why would it. Compare to the recent similar scandal on "self-reported data" for US News college rankings. [https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=%22college%20ranking%22&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...]
- smegsicle 4y ago
  revolutionary ideas will always come from outside, because institutions naturally can only form around what they already assume to be true... but taking that to mean institutions serve no purpose is terribly shortsighted one thing that does have to go though is the implication that the peer-review process is 'science' and anything outside of it is 'not science', which is just embarrassing
stuckinhell 4y ago
I'm not sure how less gatekeeping mechanisms would be helpful ? Wouldn't that just lead to more crap being produced, and informal power structures and cliques forming ? It sounds like research needs to be held accountable to some kind of standard for accuracy or quality.
- dokein 4y ago
  One issue is that the current gatekeepers (peer reviewers for journals, grant proposal scoring committees, promotion committees, etc.) are often the people most prominent in their field. On one hand this makes sense for obvious reasons (an expert is the most equipped to judge their field), but on the other hand things like the amyloid hypothesis get 'baked-in' because, well, it's pretty hard to ask those same individuals to highly rank a large grant proposal that goes against their own theory. So I think the answer is gatekeeping needs to be different -- not less.
  - Beldin 4y ago
    Do note that the more the gate is kept, the stronger the incentive for cheating. Moreover, peer review is overrated. An experiment by the NIPS conference in 2014 found that ~60% of submissions are in the "gray middle" - not obviously great or obviously crap. In more detail: they split the PC in two, and had 10% of papers (166) reviewed by both halves. Each half had to accept ~37 papers. The halves disagreed on 21 accepted papers [1]. So yeah, if your submission is sciencey, it's the flip of a coin whether peer review will accept it. [1] https://hunch.net/?p=467864 https://hunch.net/?p=467864
- slowmovintarget 4y ago
  Agree. I don't think we want /r/science moderators determining what is accepted research. But there must be some way to democratize the process short of waiting for the perfect AGI to moderate publications.
- naasking 4y ago
  It seems plausible that informal power structures may be easier to disrupt with better data than formal power structures.
pca006132 4y ago
Well you don't need to go through the gate keeping mechanism, you can just publish a blog. The gate keeping mechanism is like a web of trust that retain only those that are considered valuable in the community, so researchers can have a bounded number of articles to read, at least this is what I think.
- lazide 4y ago
  It's exactly the same problem 'web of trust' has though - incentives for the gate keepers are NOT aligned with accuracy necessarily.
  - pca006132 4y ago
    Yeah, but I think the problem is not really in the web of trust but in the incentive for researchers. If we are still stuck in this publish or perish trend, even if we replace those publishers with journals based on 'open communities', this kind of things will still happen. However, if we abandon the use of such indices (H-index, citation count, whatever), what should we use to evaluate grant applications? Sales pitch like startups asking for VC money? I don't know, I think the system is problematic but I can't really think of something else that is feasible.
fny 4y ago
One of the big questions I have is what happens if there's no significant financial incentive tied to a therapy. Lithium seems quite promising, for example. However, it's an abundant salt that requires very little research (meager funding opportunities) and can't be patented (non-existent commercialization opportunities.)
- AlotOfReading 4y ago
  You can still IP protect delivery mechanisms and branding. Insulin and tums are examples, respectively. I don't see what the potential for issues here is unless it's an ultra-rare disease where the total market isn't worth the certification costs. That doesn't describe Alzheimer's.
323 4y ago
I think one of the fundamental issues is that in science it is absolutely forbidden to accuse someone of fraud. If you do that you will be cancelled to use a modern term. And no peer reviewer dares question the integrity of the data or researcher, it's taken as gospel. So while many might know about, nobody dares say a thing.
- sillysaurusx 4y ago
  For what it’s worth, this hasn’t been my experience in ML. But I’ve had a limited window into academia. We were always questioning each other’s results. Especially when something seemed too good to be true. “Integrity” can take many forms, and it’s surprisingly easy to fool yourself when you’re doing work in the field. So the default assumption was that we were all fooling ourselves, not each other, unless proven otherwise.
  - 323 4y ago
    But were you accusing others in public of intentional fraudulent activities? Because it's a different thing than "you have bad statistics here" or "wrong assumptions here".
    - sillysaurusx 4y ago
      That’s true.
  - PeterisP 4y ago
    ML is a bit different because if the authors provide sufficient data, then any fraud is trivially obvious on replication and if they don't provide sufficient data, then that is a reasonable criticism on its own without having to go into motives. However, in other areas where you have "real world" experiments, you don't even expect the experiments to replicate - two clinical trials on different sets of patients won't necessarily yield the same results, and different results when repeating a biological experiment does not necessarily imply fraud; we know that in this domain (unlike ML) we sometimes do have unknown confounders that experiments don't control for.
  - 4y ago
searine 4y ago
>In other words, there are thousands of papers supporting beta-amyloid and/or building off the data / theory. It is important to note that the fraud doesn't encompass all of beta-amyloid, but is specific to a subtype called Ab*56. The plaques are there in autopsy tissue. We can see them. What is causing them is still uncertain. >No, in reality, this should indicate there is systemic fraud in the industry. Disagree. This indicates that the field can successfully identify and remove bad/fraudulent lines of inquiry in a relatively short amount of time (a decade). >A complete overhaul is needed This is extreme. Fraud happens. Scientific process is designed to root out unverifiable theories, by design. The process is working as intended, why burn it down?
- arbitrage 4y ago
  The process is not working as intended. 16 years to detect and correct major fraud in the field? That's simply inexcusable, and a sign that things are definitely not working properly.
- zzleeper 4y ago
  It's specific to the most important subtype, which single handedly rescued the entire beta-amyloid theory from the waste bin it was headed to in the early 2000s. Even 5-10 years ago I heard stories of researchers criticizing the "beta-amyloid cabal" for blocking every other avenue for research, as they were the ones in control of the journals (the referees, editors) and grants.
- mike_hearn 4y ago
  "the field can successfully identify and remove bad/fraudulent lines of inquiry in a relatively short amount of time" It took 16 years and only happened due to a couple of neuroscientists spotting a short selling opportunity, which allowed them to fund an investigation. Phrased differently, the field itself did not find this fraud. Outsiders did, when motivated by non-academic systems. "The process is working as intended, why burn it down?" The mis-allocation of hundreds of millions of dollars on the basis of a Photoshop, with the only outcome being that everyone involves says "no comment", is not by any measure the process working as intended.
sillysaurusx 4y ago
It’s okay to recognize a problem without proposing a solution. I agree that there are alarming levels of fraud. But heading straight to “tear down the journals” seems mistaken. New progress is rarely made by burning down old structures. New work tends to transcend old work. So the best way to remove the gate keeping mechanisms is to make them irrelevant. I suspect that a few more generations of tech progress might make that a reality. It seems like a matter of time till some teenage upstart is hacking on their brain the same way I hack on ML, for better or worse. And teenagers tend to notice correlations that the old guard miss. We’re just not there yet. And that’s fine. Our options are to work within the confines of existing systems, or build new systems. Removing mechanisms is a bit like removing a dam because it’s leaking. The best strategy is probably to fix the leaks.
- lettergram 4y ago
  > But heading straight to “tear down the journals” seems mistaken. Ultimately, I think journals can be reformed but it’ll break their business model. Effectively, you can publish without reviews. Then have credentialed people anywhere anonymously making comments / challenges. You can also let the public similarly comment. Similar to movie reviews - people in and out of the industry could both review. On top of that, allocate 25% of grants for replication which could be done in a somewhat public manner and attached to as a report on the paper. Things like that. That said, Einstein had his work initially rejected. There weren’t strong journals at the time as a gate keeper. Just people would publish and publicly debate — I don’t think there’s an issue with that now. Particularly, With the internet.
  - Beldin 4y ago
    > Effectively, you can publish without reviews. Yes, you can. If you want an academic career, you'll also need to publish in prestigious outlets though. I would love a better model for academia, but I haven't heard a convincing one, let alone that gaining sufficient traction.
gonehome 4y ago
These trends have a long history. Halsted's 'radical mastectomy' was similar. He had an incorrect theory of cancer (centrifugal spiral) that caused a lot of unnecessary harm, but it was not possible for people to push back against it because of his status in the community (iirc pushback eventually came from someone doing research in London where Halsted was less powerful). The emperor of all maladies is a great book about cancer generally which touches on this. Mendel's peas are another example - breakthrough that went ignored for 40 years because Mendel was a nobody and biologists of the time had their own nonsense theories not backed by real empiricism. Phlogiston, Elan Vital, etc. - a little different since they weren't even pretending to be empirically true. Just people making stuff up. Falling into made up tribal nonsense is the default state of humanity - even for scientists, it's hard to think independently to overcome that. At least scientists are supposed to have that as a goal, but focusing on this difficulty and how we're all affected by it does lead to better thinking imo - at least it helps recognize these failures as they're happening more quickly.
- ZeroGravitas 4y ago
  Isn't "Phlogiston" basically negative oxygen? Which is why it actually had reasonable empirical support at the time. I've read that the guy we now credit as discovering oxygen, was actually trying to (and died believing he did) create 'de-phlogistoned' air, and because there was no phlogistons in it, things burned really well in it as it sucked the phlogistons out of the burning matter even faster than normal air. The key difference is that for 'negative oxygen' to be a thing it would need negative mass as oxidised (or de-phlogistoned) materials gained weight. Just looked it up on Wikpedia, in case I was repeating urban legends: > During his lifetime, Priestley's considerable scientific reputation rested on his invention of carbonated water, his writings on electricity, and his discovery of several "airs" (gases), the most famous[7] being what Priestley dubbed "dephlogisticated air" (oxygen). Priestley's determination to defend phlogiston theory and to reject what would become the chemical revolution eventually left him isolated within the scientific community. https://en.wikipedia.org/wiki/Joseph_Priestley https://en.wikipedia.org/wiki/Joseph_Priestley
- causi 4y ago
  To be fair, there are a lot of exceptions to Mendel's laws and he falsified some of his data so it matched his expectations.
tptacek 4y ago
There's somebody on this thread who works in the field of Alzheimers research who says this story is exaggerated.
matthewdgreen 4y ago
> I personally don’t see how we can advance science with the current credentialing and paper citation == academic success we’ve seen grow the last 50 years. As a scientist I would be thrilled to see someone remove these systems. The question then is: what to replace it with that would work better? None of our resources are infinite. It's even harder when the fundamental problem is research fraud.
- hinkley 4y ago
  I'm more worried about, "how do we illustrate the clear need for a change without pouring gasoline directly onto the anti-intellectual bonfires that already exist?" Every one of these little scandals points us to opportunities to improve, but also is confirmation bias for people who think science is just one system of superstitions amongst a field of viable contenders.
  - idontpost 4y ago
    > I'm more worried about, "how do we illustrate the clear need for a change without pouring gasoline directly onto the anti-intellectual bonfires that already exist?" Asking this question is in and of itself directly pouring gasoline onto the bonfires. Just work on fixing the damn problems. Trying to PR them is part of the problem.
    - matthewdgreen 4y ago
      It doesn’t really matter either way. At a meta-level this isn’t about Alzheimer’s, and it isn’t about “fixing science”. Nobody in this discussion has any constructive idea how to cure this horrifying disease, or how to build a scientific establishment that isn’t vulnerable to fraud. If that’s all we cared about this would be a boring policy discussion about how to fix peer review. But nobody here is interested in having that discussion: this discussion is about whether we should discard the input of the scientific community as hopelessly corrupt.
- joemphilips 4y ago
  I'm surprised that no one has mentioned prediction market(PM) as the alternative. This is old paper but IMO it is still persuasive. https://mason.gmu.edu/~rhanson/gamble.html https://mason.gmu.edu/~rhanson/gamble.html
SubiculumCode 4y ago
that is quite the accusations, verging on slanderous, based on hearsay.
rossdavidh 4y ago
I think this problem is much more severe in cases where a more accurate assessment would be, "we have no idea what is going on or how to stop it, even though we desperately want to". People are more likely to grasp at straws if there is nothing other than straw available. This does not, of course, mean that it's not still a big problem; far from it.
not2b 4y ago
Back when I used to serve on conference review committees I'd talk about the drunk and the lamppost problem: finding the keys in the dark is too hard, so he looks under the lamppost where the light is better (the authors make a simplifying assumption that allows for an elegant approach but the solution is then irrelevant to solving the original problem). In this case curing Alzheimer's is too hard, but we can make drugs that clear up the plaques that are associated with Alzheimer's and not pay attention to whether doing that actually helps patients: we can document that the plaques are reduced and publish on that basis. So I suspect that a lot of the research is correct but of no value, because the wrong problem is being "solved".
charlieflowers 4y ago
Cognitive bias kept me from seeing this at first, but this is really a sign, not just of a broken set of incentives, but _decline_. If you're old enough, imagine how we (USA-centric) would mock flawed research coming out of the Soviet Union. Well, here we are. And unfortunately there are plenty of signs of decline once it dawns on you that's what's happening.
- rjbwork 4y ago
  Indeed. Our institutions are severely degraded, if not horribly broken. I hope it is not irreparably so, but I am not optimistic. The signs of societal and imperial decline are everywhere these days and we do not seem to be able to meet the challenges, many existential, that lay before us. I'm not sure what to do.

kurupt213 4y ago

What’s one of the first things they teach when dealing with statistical analysis? Correlation is not causation

beanjuice 4y ago

What do you mean by "the gate keeping mechanisms" should be removed? Do you mean the difficulty of publishing? Or tenure? Or degrees?

voakbasda 4y ago
All of those gate keeping mechanisms have systemic issues and need to be reformed at the very least. "Removed" might be a bit strong, but I definitely see how one could reach that perspective. Regardless, they certainly have lost the credibility to claim any monopolistic authority over the domains they purport to serve.
- smcin 4y ago
  a) It's easy to suggest gatekeeping mechanisms like degrees, tenture, journals controlling publication "should all be reformed", without proposing any specific alternatives; but they all exist to prevent an obvious set of anti-patterns; if you eliminate them you just get different anti-patterns. Can you make a specific proposal of what's a better system to replace them with? b) In any case, the $$ incentives and root-causes are distorted, no point in scapegoating the gatekeeping mechanisms. The US pharmaceutical industry (for example) is 3.2% of GDP, and is 48% of the global pharmaceutical market. That amount of $$ is obviously going to distort policy at all levels. c) Scientific fraud should be criminalized, there should be stiff penalties and they should actually be enforced (actual prison, fines, losing job and tenure, retracting PhDs). Otherwise you're playing a race-to-the-bottom in a consequence-free environment, against someone else who's going to do unethical things. d) Until you address the root-causes/incentives, people's behavior won't change, why would it. Compare to the recent similar scandal on "self-reported data" for US News college rankings. [https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=%22college%20ranking%22&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...]
smegsicle 4y ago
revolutionary ideas will always come from outside, because institutions naturally can only form around what they already assume to be true... but taking that to mean institutions serve no purpose is terribly shortsighted one thing that does have to go though is the implication that the peer-review process is 'science' and anything outside of it is 'not science', which is just embarrassing

stuckinhell 4y ago

I'm not sure how less gatekeeping mechanisms would be helpful ? Wouldn't that just lead to more crap being produced, and informal power structures and cliques forming ? It sounds like research needs to be held accountable to some kind of standard for accuracy or quality.

dokein 4y ago
One issue is that the current gatekeepers (peer reviewers for journals, grant proposal scoring committees, promotion committees, etc.) are often the people most prominent in their field. On one hand this makes sense for obvious reasons (an expert is the most equipped to judge their field), but on the other hand things like the amyloid hypothesis get 'baked-in' because, well, it's pretty hard to ask those same individuals to highly rank a large grant proposal that goes against their own theory. So I think the answer is gatekeeping needs to be different -- not less.
- Beldin 4y ago
  Do note that the more the gate is kept, the stronger the incentive for cheating. Moreover, peer review is overrated. An experiment by the NIPS conference in 2014 found that ~60% of submissions are in the "gray middle" - not obviously great or obviously crap. In more detail: they split the PC in two, and had 10% of papers (166) reviewed by both halves. Each half had to accept ~37 papers. The halves disagreed on 21 accepted papers [1]. So yeah, if your submission is sciencey, it's the flip of a coin whether peer review will accept it. [1] https://hunch.net/?p=467864 https://hunch.net/?p=467864
slowmovintarget 4y ago
Agree. I don't think we want /r/science moderators determining what is accepted research. But there must be some way to democratize the process short of waiting for the perfect AGI to moderate publications.
naasking 4y ago
It seems plausible that informal power structures may be easier to disrupt with better data than formal power structures.

pca006132 4y ago

Well you don't need to go through the gate keeping mechanism, you can just publish a blog. The gate keeping mechanism is like a web of trust that retain only those that are considered valuable in the community, so researchers can have a bounded number of articles to read, at least this is what I think.

lazide 4y ago
It's exactly the same problem 'web of trust' has though - incentives for the gate keepers are NOT aligned with accuracy necessarily.
- pca006132 4y ago
  Yeah, but I think the problem is not really in the web of trust but in the incentive for researchers. If we are still stuck in this publish or perish trend, even if we replace those publishers with journals based on 'open communities', this kind of things will still happen. However, if we abandon the use of such indices (H-index, citation count, whatever), what should we use to evaluate grant applications? Sales pitch like startups asking for VC money? I don't know, I think the system is problematic but I can't really think of something else that is feasible.

fny 4y ago

One of the big questions I have is what happens if there's no significant financial incentive tied to a therapy. Lithium seems quite promising, for example. However, it's an abundant salt that requires very little research (meager funding opportunities) and can't be patented (non-existent commercialization opportunities.)

AlotOfReading 4y ago
You can still IP protect delivery mechanisms and branding. Insulin and tums are examples, respectively. I don't see what the potential for issues here is unless it's an ultra-rare disease where the total market isn't worth the certification costs. That doesn't describe Alzheimer's.

323 4y ago

I think one of the fundamental issues is that in science it is absolutely forbidden to accuse someone of fraud. If you do that you will be cancelled to use a modern term. And no peer reviewer dares question the integrity of the data or researcher, it's taken as gospel. So while many might know about, nobody dares say a thing.

sillysaurusx 4y ago
For what it’s worth, this hasn’t been my experience in ML. But I’ve had a limited window into academia. We were always questioning each other’s results. Especially when something seemed too good to be true. “Integrity” can take many forms, and it’s surprisingly easy to fool yourself when you’re doing work in the field. So the default assumption was that we were all fooling ourselves, not each other, unless proven otherwise.
- 323 4y ago
  But were you accusing others in public of intentional fraudulent activities? Because it's a different thing than "you have bad statistics here" or "wrong assumptions here".
  - sillysaurusx 4y ago
    That’s true.
- PeterisP 4y ago
  ML is a bit different because if the authors provide sufficient data, then any fraud is trivially obvious on replication and if they don't provide sufficient data, then that is a reasonable criticism on its own without having to go into motives. However, in other areas where you have "real world" experiments, you don't even expect the experiments to replicate - two clinical trials on different sets of patients won't necessarily yield the same results, and different results when repeating a biological experiment does not necessarily imply fraud; we know that in this domain (unlike ML) we sometimes do have unknown confounders that experiments don't control for.
- 4y ago

searine 4y ago

>In other words, there are thousands of papers supporting beta-amyloid and/or building off the data / theory. It is important to note that the fraud doesn't encompass all of beta-amyloid, but is specific to a subtype called Ab*56. The plaques are there in autopsy tissue. We can see them. What is causing them is still uncertain. >No, in reality, this should indicate there is systemic fraud in the industry. Disagree. This indicates that the field can successfully identify and remove bad/fraudulent lines of inquiry in a relatively short amount of time (a decade). >A complete overhaul is needed This is extreme. Fraud happens. Scientific process is designed to root out unverifiable theories, by design. The process is working as intended, why burn it down?

arbitrage 4y ago
The process is not working as intended. 16 years to detect and correct major fraud in the field? That's simply inexcusable, and a sign that things are definitely not working properly.
zzleeper 4y ago
It's specific to the most important subtype, which single handedly rescued the entire beta-amyloid theory from the waste bin it was headed to in the early 2000s. Even 5-10 years ago I heard stories of researchers criticizing the "beta-amyloid cabal" for blocking every other avenue for research, as they were the ones in control of the journals (the referees, editors) and grants.
mike_hearn 4y ago
"the field can successfully identify and remove bad/fraudulent lines of inquiry in a relatively short amount of time" It took 16 years and only happened due to a couple of neuroscientists spotting a short selling opportunity, which allowed them to fund an investigation. Phrased differently, the field itself did not find this fraud. Outsiders did, when motivated by non-academic systems. "The process is working as intended, why burn it down?" The mis-allocation of hundreds of millions of dollars on the basis of a Photoshop, with the only outcome being that everyone involves says "no comment", is not by any measure the process working as intended.

sillysaurusx 4y ago

It’s okay to recognize a problem without proposing a solution. I agree that there are alarming levels of fraud. But heading straight to “tear down the journals” seems mistaken. New progress is rarely made by burning down old structures. New work tends to transcend old work. So the best way to remove the gate keeping mechanisms is to make them irrelevant. I suspect that a few more generations of tech progress might make that a reality. It seems like a matter of time till some teenage upstart is hacking on their brain the same way I hack on ML, for better or worse. And teenagers tend to notice correlations that the old guard miss. We’re just not there yet. And that’s fine. Our options are to work within the confines of existing systems, or build new systems. Removing mechanisms is a bit like removing a dam because it’s leaking. The best strategy is probably to fix the leaks.

lettergram 4y ago
> But heading straight to “tear down the journals” seems mistaken. Ultimately, I think journals can be reformed but it’ll break their business model. Effectively, you can publish without reviews. Then have credentialed people anywhere anonymously making comments / challenges. You can also let the public similarly comment. Similar to movie reviews - people in and out of the industry could both review. On top of that, allocate 25% of grants for replication which could be done in a somewhat public manner and attached to as a report on the paper. Things like that. That said, Einstein had his work initially rejected. There weren’t strong journals at the time as a gate keeper. Just people would publish and publicly debate — I don’t think there’s an issue with that now. Particularly, With the internet.
- Beldin 4y ago
  > Effectively, you can publish without reviews. Yes, you can. If you want an academic career, you'll also need to publish in prestigious outlets though. I would love a better model for academia, but I haven't heard a convincing one, let alone that gaining sufficient traction.

gonehome 4y ago

These trends have a long history. Halsted's 'radical mastectomy' was similar. He had an incorrect theory of cancer (centrifugal spiral) that caused a lot of unnecessary harm, but it was not possible for people to push back against it because of his status in the community (iirc pushback eventually came from someone doing research in London where Halsted was less powerful). The emperor of all maladies is a great book about cancer generally which touches on this. Mendel's peas are another example - breakthrough that went ignored for 40 years because Mendel was a nobody and biologists of the time had their own nonsense theories not backed by real empiricism. Phlogiston, Elan Vital, etc. - a little different since they weren't even pretending to be empirically true. Just people making stuff up. Falling into made up tribal nonsense is the default state of humanity - even for scientists, it's hard to think independently to overcome that. At least scientists are supposed to have that as a goal, but focusing on this difficulty and how we're all affected by it does lead to better thinking imo - at least it helps recognize these failures as they're happening more quickly.

ZeroGravitas 4y ago
Isn't "Phlogiston" basically negative oxygen? Which is why it actually had reasonable empirical support at the time. I've read that the guy we now credit as discovering oxygen, was actually trying to (and died believing he did) create 'de-phlogistoned' air, and because there was no phlogistons in it, things burned really well in it as it sucked the phlogistons out of the burning matter even faster than normal air. The key difference is that for 'negative oxygen' to be a thing it would need negative mass as oxidised (or de-phlogistoned) materials gained weight. Just looked it up on Wikpedia, in case I was repeating urban legends: > During his lifetime, Priestley's considerable scientific reputation rested on his invention of carbonated water, his writings on electricity, and his discovery of several "airs" (gases), the most famous[7] being what Priestley dubbed "dephlogisticated air" (oxygen). Priestley's determination to defend phlogiston theory and to reject what would become the chemical revolution eventually left him isolated within the scientific community. https://en.wikipedia.org/wiki/Joseph_Priestley https://en.wikipedia.org/wiki/Joseph_Priestley
causi 4y ago
To be fair, there are a lot of exceptions to Mendel's laws and he falsified some of his data so it matched his expectations.

tptacek 4y ago

There's somebody on this thread who works in the field of Alzheimers research who says this story is exaggerated.

matthewdgreen 4y ago

> I personally don’t see how we can advance science with the current credentialing and paper citation == academic success we’ve seen grow the last 50 years. As a scientist I would be thrilled to see someone remove these systems. The question then is: what to replace it with that would work better? None of our resources are infinite. It's even harder when the fundamental problem is research fraud.

hinkley 4y ago
I'm more worried about, "how do we illustrate the clear need for a change without pouring gasoline directly onto the anti-intellectual bonfires that already exist?" Every one of these little scandals points us to opportunities to improve, but also is confirmation bias for people who think science is just one system of superstitions amongst a field of viable contenders.
- idontpost 4y ago
  > I'm more worried about, "how do we illustrate the clear need for a change without pouring gasoline directly onto the anti-intellectual bonfires that already exist?" Asking this question is in and of itself directly pouring gasoline onto the bonfires. Just work on fixing the damn problems. Trying to PR them is part of the problem.
  - matthewdgreen 4y ago
    It doesn’t really matter either way. At a meta-level this isn’t about Alzheimer’s, and it isn’t about “fixing science”. Nobody in this discussion has any constructive idea how to cure this horrifying disease, or how to build a scientific establishment that isn’t vulnerable to fraud. If that’s all we cared about this would be a boring policy discussion about how to fix peer review. But nobody here is interested in having that discussion: this discussion is about whether we should discard the input of the scientific community as hopelessly corrupt.
joemphilips 4y ago
I'm surprised that no one has mentioned prediction market(PM) as the alternative. This is old paper but IMO it is still persuasive. https://mason.gmu.edu/~rhanson/gamble.html https://mason.gmu.edu/~rhanson/gamble.html

SubiculumCode 4y ago

that is quite the accusations, verging on slanderous, based on hearsay.

rossdavidh 4y ago

I think this problem is much more severe in cases where a more accurate assessment would be, "we have no idea what is going on or how to stop it, even though we desperately want to". People are more likely to grasp at straws if there is nothing other than straw available. This does not, of course, mean that it's not still a big problem; far from it.

not2b 4y ago

Back when I used to serve on conference review committees I'd talk about the drunk and the lamppost problem: finding the keys in the dark is too hard, so he looks under the lamppost where the light is better (the authors make a simplifying assumption that allows for an elegant approach but the solution is then irrelevant to solving the original problem). In this case curing Alzheimer's is too hard, but we can make drugs that clear up the plaques that are associated with Alzheimer's and not pay attention to whether doing that actually helps patients: we can document that the plaques are reduced and publish on that basis. So I suspect that a lot of the research is correct but of no value, because the wrong problem is being "solved".

charlieflowers 4y ago

Cognitive bias kept me from seeing this at first, but this is really a sign, not just of a broken set of incentives, but _decline_. If you're old enough, imagine how we (USA-centric) would mock flawed research coming out of the Soviet Union. Well, here we are. And unfortunately there are plenty of signs of decline once it dawns on you that's what's happening.

rjbwork 4y ago
Indeed. Our institutions are severely degraded, if not horribly broken. I hope it is not irreparably so, but I am not optimistic. The signs of societal and imperial decline are everywhere these days and we do not seem to be able to meet the challenges, many existential, that lay before us. I'm not sure what to do.