4 ms·

Philosophy doesn't have any ROI. It lives off critical examination of ideas, which is why research on it has to be done in a vibrant community. In a nutshell, i

Philosophy doesn't have any ROI. It lives off critical examination of ideas, which is why research on it has to be done in a vibrant community. In a nutshell, it needs a research topic to be popular enough to stir up some criticisms of it and have enough experts who can evaluate it. Otherwise, that research program is doomed. Without critical evaluation you cannot have research. That requires enough of a critical threshold of people working on the topic and a community. Generally, science lives off skepticism. Skepticism requires a decent number of skeptics who try to show that you're wrong. That requires your research to be sufficiently popular for others. Without that control, it becomes crackpotery very fast. You've got it the wrong way 'round.

lemonwaterlime 1y ago
I see. You're focused on philosophy only. My point is about the other areas of academia where you "doubt the situation is different." There, the dynamics I described are real. Either way, there's a certain perspective and bias one has from being inside a system. Telling someone they've got things the "wrong way 'round" risks being biased in the Plato's Cave kind of way.
- jonathanstrange 1y ago
  No, I wasn't talking about philosophy only. Check out my other reply. It's a more practical take that makes the point clearer. Funding misfit research makes no sense no matter how you square it.
godelski 1y ago
> Philosophy doesn't have any ROI. Maybe don't generalize then? *Especially* on a tech forum? I'm going to back lemonwaterlime here. The truth of the matter is that even as scientists we aren't very good at predicting what technology or research will be impactful. There's a laundry list of ground breaking research and technologies that have failed to be funded or rejected. This includes mRNA research as well as many Nobel prize winning research. It's so common there's multiple cliches describing the phenomena! It's pretty simple to see why lemonwaterlime's claim has merit: you can't cause paradigm shifts by maintaining the current paradigm. I'm sure there's another aspect that you see in your own field: momentum. When winning awards is used as merit for winning more awards then you create a feedback loop that reinforces a narrowing of ideas. This is especially problematic when journals and conferences measure their quality by their rejection rates. One CANNOT determine quality or impact factor via a rejection rate. I left my PhD with a distaste for academia because I cannot fathom how so many researchers evaluate works comparatively and with so little care (as if it is an unproductive task that simply needs to be completed rather than the seriousness it deserves). You can only evaluate a work by its own validity, not by your personal belief if it is more impactful or not than your imagination of the quality of other submissions. It is insane and self-destructive. Reviewers should not be incentivized to reject works, but rather to write meaningful comments to improve works. It's far too easy to criticize a work, and in the current system metareviewers and ACs are biased to scrutinize rejects far less than accepts. I'm sorry, it just isn't working. Here's all you can do in reviewing. For a grant, paper, or whatever Objective: - Contains factual errors - Contains writing errors Subjective: - The validity of the work - The impact of the work - Properly communicated to the intended audience - Contains the appropriate citations - Is sufficient quality for the prestige of the venue/award When I review papers I will accept a paper if: - It is void of *major* factual errors (that I am aware of) - Is sufficiently communicated that I believe a junior level grad student would be able to replicate (maybe with some effort) - It is not plagiarized My beliefs about impact and how incremental the work is only affect how strongly I will accept a work. Regardless of accept or reject I write lengthy reviews, because my job is to help the authors make their paper the best it can be. I've even received reviewer awards at top venues for this, but let's be real, those awards are effectively meaningless. > Skepticism requires a decent number of skeptics who try to show that you're wrong. Yes, but this *CANNOT* be done meaningfully through review. It can only be done through replication efforts. Something we not only fail to incentivize in the current ecosystem, but actively discourage. It is absolutely bonkers that we discourage what is fundamentally the cornerstone of science (you are not claiming philosophy is science, are you?). It is also ludicrous that we actively discourage publishing negative results. There's so much time and money wasted in what could be resolved through better communication. > Generally, science lives off skepticism. This is incorrect and only encourages the anti-science crowd. Science lives off of replication and the free exploration of ideas. Will you have failures and fund ideas that turn out to be wrong? Of course! What scientist isn't aware of the extreme difficulty of pushing the frontier of human knowledge? Even in the current system we have an absurdly large amount of fraud and an unacceptable failure rate for replication. The system only encourages these things. While we don't want to encourage futile pursuits, let's be real, if you've made it to the level of a PhD you are quite in tune with the nuances of the field and should have a fairly good intuition of what problems need to be resolved. If you do not, you really should not have been awarded your doctorate. Our efforts to reduce waste only serves as a modern day cobra effect. I'm sorry, fundamental research is just inherently risky. But when it succeeds it can have profound impacts across the globe. I'd argue that the invention of a single topic has created magnitudes more value (in varying senses of the word) than all our failures combined. Would you not think that alone, Calculus produces trillions of dollars a year in value and aids in saving millions of lives a year? Or take the more recent mRNA results. How many failures, crackpots, and "waste" does this single success offset? Hundreds? Thousands? Millions? Mind you, Katalin Karikó is one of those "crackpots" you refer to. How many true crackpots are willing to allow in order to ensure such false negatives may succeed? Or inversely, how many high impact falsely labeled crackpots are we willing to sacrifice in order to ensure we do not misallocate funds to the crazies? I'm sure you've thought about countless variations of this question...
- jonathanstrange 1y ago
  I appreciate the detailed reply and wholeheartedly disagree with most of your points. Most foundational research also has no quantifiable ROI, and rightly so. Putting these details aside, let me put it in more practical terms, since both you and the other person who replied seem to be unaware of the unavoidable practicalities of funding evaluation. As someone who has been in various evaluation panels and also based on my own successful applications (the guidelines are always public), I want to stress that "popular" in the context of science evaluation really just means that the majority of evaluators think a proposal has more scientific merits, is original, can be expected to make substantial contributions, can be pulled off, is supported by the candidates CV, and so on. Providing more funding for "misfits" and "outcasts" could mean two different things. First, it could mean that less popular proposals are getting higher chances of obtaining funding without jeopardizing the proposals of which the panel thinks they have most merits. Then this is merely a complicated way of saying that there should be more funding for everyone. I've never met a researcher who disagreed with that. Second, it could mean that unpopular proposals of "misfit" candidates should be given precedence over more popular proposals in certain calls, for example by explicit guidelines or special grants. Although there are grant schemes for that (e.g. special grants for "high risk" projects in the EU Horizon 2020 scheme), this would be almost contradictory as a general guideline in all calls. It just doesn't make sense. Scientific peers must at one point or another evaluate proposals to decide over grants. This cannot be done by politicians or mere administrators. It would be idiotic to ask them to favor proposals they think have no merits. So, Yes: Funding schemes for high risk proposals should exist, and they do exist. But these are the exception, not the rule, just like successful outsiders in science are outliers instead of the rule. Most of science is an extremely collaborative effort. (The same holds for philosophy, which is not a science.) Regarding replication efforts: If someone's area is deeply unpopular and most people think the area or approach has no merits, then nobody will replicate it either. That's exactly my point. There needs to be a high enough threshold of scientists working on the same approach and the corresponding networking and embedding into the scientific community to ensure there are enough replication efforts and there is enough critical evaluation. Otherwise, the "misfit" is bound to publish in fringe journals with no quality control and nobody will ever check whether the work makes sense. That's not good.

lemonwaterlime 1y ago

I see. You're focused on philosophy only. My point is about the other areas of academia where you "doubt the situation is different." There, the dynamics I described are real. Either way, there's a certain perspective and bias one has from being inside a system. Telling someone they've got things the "wrong way 'round" risks being biased in the Plato's Cave kind of way.

jonathanstrange 1y ago
No, I wasn't talking about philosophy only. Check out my other reply. It's a more practical take that makes the point clearer. Funding misfit research makes no sense no matter how you square it.

godelski 1y ago

> Philosophy doesn't have any ROI. Maybe don't generalize then? *Especially* on a tech forum? I'm going to back lemonwaterlime here. The truth of the matter is that even as scientists we aren't very good at predicting what technology or research will be impactful. There's a laundry list of ground breaking research and technologies that have failed to be funded or rejected. This includes mRNA research as well as many Nobel prize winning research. It's so common there's multiple cliches describing the phenomena! It's pretty simple to see why lemonwaterlime's claim has merit: you can't cause paradigm shifts by maintaining the current paradigm. I'm sure there's another aspect that you see in your own field: momentum. When winning awards is used as merit for winning more awards then you create a feedback loop that reinforces a narrowing of ideas. This is especially problematic when journals and conferences measure their quality by their rejection rates. One CANNOT determine quality or impact factor via a rejection rate. I left my PhD with a distaste for academia because I cannot fathom how so many researchers evaluate works comparatively and with so little care (as if it is an unproductive task that simply needs to be completed rather than the seriousness it deserves). You can only evaluate a work by its own validity, not by your personal belief if it is more impactful or not than your imagination of the quality of other submissions. It is insane and self-destructive. Reviewers should not be incentivized to reject works, but rather to write meaningful comments to improve works. It's far too easy to criticize a work, and in the current system metareviewers and ACs are biased to scrutinize rejects far less than accepts. I'm sorry, it just isn't working. Here's all you can do in reviewing. For a grant, paper, or whatever Objective: - Contains factual errors - Contains writing errors Subjective: - The validity of the work - The impact of the work - Properly communicated to the intended audience - Contains the appropriate citations - Is sufficient quality for the prestige of the venue/award When I review papers I will accept a paper if: - It is void of *major* factual errors (that I am aware of) - Is sufficiently communicated that I believe a junior level grad student would be able to replicate (maybe with some effort) - It is not plagiarized My beliefs about impact and how incremental the work is only affect how strongly I will accept a work. Regardless of accept or reject I write lengthy reviews, because my job is to help the authors make their paper the best it can be. I've even received reviewer awards at top venues for this, but let's be real, those awards are effectively meaningless. > Skepticism requires a decent number of skeptics who try to show that you're wrong. Yes, but this *CANNOT* be done meaningfully through review. It can only be done through replication efforts. Something we not only fail to incentivize in the current ecosystem, but actively discourage. It is absolutely bonkers that we discourage what is fundamentally the cornerstone of science (you are not claiming philosophy is science, are you?). It is also ludicrous that we actively discourage publishing negative results. There's so much time and money wasted in what could be resolved through better communication. > Generally, science lives off skepticism. This is incorrect and only encourages the anti-science crowd. Science lives off of replication and the free exploration of ideas. Will you have failures and fund ideas that turn out to be wrong? Of course! What scientist isn't aware of the extreme difficulty of pushing the frontier of human knowledge? Even in the current system we have an absurdly large amount of fraud and an unacceptable failure rate for replication. The system only encourages these things. While we don't want to encourage futile pursuits, let's be real, if you've made it to the level of a PhD you are quite in tune with the nuances of the field and should have a fairly good intuition of what problems need to be resolved. If you do not, you really should not have been awarded your doctorate. Our efforts to reduce waste only serves as a modern day cobra effect. I'm sorry, fundamental research is just inherently risky. But when it succeeds it can have profound impacts across the globe. I'd argue that the invention of a single topic has created magnitudes more value (in varying senses of the word) than all our failures combined. Would you not think that alone, Calculus produces trillions of dollars a year in value and aids in saving millions of lives a year? Or take the more recent mRNA results. How many failures, crackpots, and "waste" does this single success offset? Hundreds? Thousands? Millions? Mind you, Katalin Karikó is one of those "crackpots" you refer to. How many true crackpots are willing to allow in order to ensure such false negatives may succeed? Or inversely, how many high impact falsely labeled crackpots are we willing to sacrifice in order to ensure we do not misallocate funds to the crazies? I'm sure you've thought about countless variations of this question...

jonathanstrange 1y ago
I appreciate the detailed reply and wholeheartedly disagree with most of your points. Most foundational research also has no quantifiable ROI, and rightly so. Putting these details aside, let me put it in more practical terms, since both you and the other person who replied seem to be unaware of the unavoidable practicalities of funding evaluation. As someone who has been in various evaluation panels and also based on my own successful applications (the guidelines are always public), I want to stress that "popular" in the context of science evaluation really just means that the majority of evaluators think a proposal has more scientific merits, is original, can be expected to make substantial contributions, can be pulled off, is supported by the candidates CV, and so on. Providing more funding for "misfits" and "outcasts" could mean two different things. First, it could mean that less popular proposals are getting higher chances of obtaining funding without jeopardizing the proposals of which the panel thinks they have most merits. Then this is merely a complicated way of saying that there should be more funding for everyone. I've never met a researcher who disagreed with that. Second, it could mean that unpopular proposals of "misfit" candidates should be given precedence over more popular proposals in certain calls, for example by explicit guidelines or special grants. Although there are grant schemes for that (e.g. special grants for "high risk" projects in the EU Horizon 2020 scheme), this would be almost contradictory as a general guideline in all calls. It just doesn't make sense. Scientific peers must at one point or another evaluate proposals to decide over grants. This cannot be done by politicians or mere administrators. It would be idiotic to ask them to favor proposals they think have no merits. So, Yes: Funding schemes for high risk proposals should exist, and they do exist. But these are the exception, not the rule, just like successful outsiders in science are outliers instead of the rule. Most of science is an extremely collaborative effort. (The same holds for philosophy, which is not a science.) Regarding replication efforts: If someone's area is deeply unpopular and most people think the area or approach has no merits, then nobody will replicate it either. That's exactly my point. There needs to be a high enough threshold of scientists working on the same approach and the corresponding networking and embedding into the scientific community to ensure there are enough replication efforts and there is enough critical evaluation. Otherwise, the "misfit" is bound to publish in fringe journals with no quality control and nobody will ever check whether the work makes sense. That's not good.