8 ms·

As a scientist, I've seen lots of people who are overconfident in both themselves and overconfident in science. IMO, a good PhD science program helps students m

As a scientist, I've seen lots of people who are overconfident in both themselves and overconfident in science. IMO, a good PhD science program helps students master a discipline, but also to recognize the limits of the discipline. For example, 23andMe thought they would revolutionize drug development with a huge genetics dataset. In practice, genetic information alone is not sufficient to treat the majority of diseases that affect individuals and society. There is too much environmental variation affecting human biology for purely genetic approaches. Understanding the real limits of knowledge is vital to pushing knowledge forward where we can. As a biologist, one of the things I most appreciate about Sabine Hossenfelder (a physicist) is that she highlights the limits of knowledge in her (and adjacent) fields. She gets a lot of push back (and is sometimes wrong), but having the discussion is vital to science. Acknowledging the limits of science is not a negative attitude about science, but a positive one. A clear idea of the current limits of science (both theoretically and practically) is instrumental to pushing through them. For example, the scholarly papers highlighting the replication crisis (https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis) are actually very useful to maintaining the health of science as a human endeavor, not a critique of science. Scientists need a clear understanding of the scientific foundations that they are building on.

NikolaNovak 3y ago
I find that coloquially we use the term "science" in several distinct meanings, two of which are: 1. Science as the body of knowledge 2. Science as a method / approach I also find that mixing the meanings/perspectives/intent up in a single conversation is common, sometimes accidentally sometimes intentionally. In my ignorance (ComSci major, so not a real science :), I would describe myself as extremely positive / confident to "Science, the approach/method" and, through that approach, pragmatic about the "Science the current body of knowledge". In other words: Yes, there are very much limits to what we currently know, and some of what we think we know will turn out to be wrong, subtly or catastrophically. There are definitely huge limits and uncertainties to Science the body of knowledge! But, acknowledging it is kinda the point, and the best way to figure it out that we are currently aware of is through scientific approach. (I've just realized I might even have become a zealot that you describe, because I can't even figure out what a plausible & feasible alternative method is, if your goals are to actually figure things out. To that point, I find humility and skepticism about your current science body of knowledge a crucial part of science the method, something which most other methods lack). I find distinction is crucial especially in political and religious discussion frameworks. Otherwise, I never know if I agree or disagree with statements regarding "limits of science" etc. (This is all further mixed up by zillion of daily popular articles where "Science says that [...]!!!" or "[...], scientists find", which... ugh, oversimplify at best and deceive more likely) What are your thoughts?
- theGnuMe 3y ago
  The science of computing is a real science.
  - seanr88 3y ago
    I'd agree with this although I understand where the idea comes from. It is difficult for people to understand what is and is not a science and it is easy to think that computer science is not a science even when you study it. The way we learn and study topics is divorced from the original method of discovering those topics. The way people learn Computer Science is generally by absorbing the information, not by doing the experiments. So it is difficult for people to understand that the way we have this knowledge is through hypothesis forming and experimentation i.e. Science.
  - deleted 3y ago
    [deleted]
- Sharlin 3y ago
  I think there is a third definition of "science": 3. What is actually happening in academia I think most people exposed to science#3 from the inside can agree that science#2 works – and indeed works surprisingly well – despite science#3, not because of it.
  - mathematicaster 3y ago
    Top comment. My adjacent refrain is "Science is not the same as people trying to do science".
    - biomcgary 3y ago
      The work of philosophers and sociologists of science, Kuhn and Feyerabend in particular, support this perspective.
    - deleted 3y ago
      [deleted]
eh_why_not 3y ago
> As a scientist, I've seen lots of people who are overconfident in both themselves and overconfident in science. I feel that I've never had the first problem, but have definitely had the second. On one hand, there is the natural limitation of Science itself in terms of the type of questions (that are amenable to the scientific method) it can answer. On the other hand, it is still the best way of generating knowledge that we have. My overconfidence was that scientists, as individuals and as a community, would always do the right thing, driven by, and honestly following, the scientific method. But in the past few years I've had to revisit this assumption several times and be reminded to always retain some healthy skepticism. Most recent example is this climate scientist who just published in Nature, and then went ahead afterwards and penned an op-ed [0] saying he actually misrepresented the actual factors in order to get published. [0] https://www.thefp.com/p/i-overhyped-climate-change-to-get-published https://www.thefp.com/p/i-overhyped-climate-change-to-get-pu...
- perfect-blue 3y ago
  > My overconfidence was that scientists, as individuals and as a community, would always do the right thing... This is a great point. I'm shocked by how often I end up working on a project with a colleague who is taking the path of least resistance. In my field, this usually results in using decades old statistical methods than have been proven time and time again to be unsatisfactory. They just don't want to learn new methods or their technical expertise aren't good enough to learn how to implement the new approaches. So they just coast. I'm not sure what to do in these situations other than just try and set a positive example.
dekhn 3y ago
23&Me failed in drug development because they started from a mistaken premise- that the data they collected (very specifically, genotype arrays) would produce data that was correlated with human health closely enough to identify targets (proteins or pathways to disrupt/modify). They have a huge genetics dataset, they don't have a huge genomics dataset, and the underlying relationship between the genome and phenotypes (especially complex disease phenotypes) is a highly nonlinear function. Environment is important but we could still have huge improvements in medical care using genomics. It's easy to obtain and still has a very strong relationship to disease, and is a problem best solved by deep learning. I've watched people tilt against this windmill for 25+ years and it's kind of funny just how bad our labelling of diseases is.
- biomcgary 3y ago
  Excellent point. I work at a biotech startup where one major focus is "how do we fix the disease labels?". I call it the pyrite problem (your gold standard data contains fools gold), but it is known more prosaically as the mislabeling problem.
- dahinds 3y ago
  This is an inaccurate view of what 23andMe is doing. First we did not "fail" in drug development, our collaboration with GSK yielded substantially more programs than we had anticipated when we started. The verdict on whether the strategy is successful or not will not be clear for another 5-10 years because being better at picking targets doesn't shorten the timeline for bringing a drug to market, it mostly means that you should have a modestly higher proportion of programs that end up being successful. Second we did not start with a premise that we could or needed to do a great job of predicting disease from genetics alone, that is not required to identify good targets. You're correct we don't have huge genomics datasets, we largely use the same genomics datasets others use. Identifying targets with genetics in the simplest sense requires using genetics to identify an association at a genomic location, and then using genomics for functional interpretation of that association. And having the largest database of genetic associations enables the first step of that. I'm not sure what you mean by "the underlying relationship between the genome and phenotypes... is a highly nonlinear function". Simple additive models account for most of the heritability of complex phenotypes.
f1shy 3y ago
The most over confident in theirselves and science people I know are PhD. With egos and arrogance greater than the solar system. People that go early to the industry and see the “real world” tend to have a more tamed expectative of what they could achieve and science can offer.
brigadier132 3y ago
Ironically, you are taking this study at face value. This study reminds me of the "Republicans tend to be sociopaths more often" study. That ended up being completely refuted.
crabmusket 3y ago
> Acknowledging the limits of science is not a negative attitude about science, but a positive one. A clear idea of the current limits of science (both theoretically and practically) is instrumental to pushing through them. "The gods did not reveal, from the beginning, all things to us, but in the course of time through seeking we may learn and know things better. But as for certain truth, no man has known it, nor shall he know it, neither of the gods Nor yet of all the things of which I speak. For even if by chance he were to utter the final truth, he would himself not know it: for all is but a woven web of guesses" Xenophanes via Karl Popper
XTHK 3y ago
>one of the things I most appreciate about Sabine Hossenfelder (a physicist) is that she highlights the limits of knowledge in her (and adjacent) fields. I suppose you missed her recent video about economics? Her content is getting more and more clickbaitey by the day. Some really half-baked data creeping into her videos.
- ramraj07 3y ago
  It really is sad indeed. You die a hero (stop posting videos once you’ve exhausted your expertise) or live long enough to become a villain indeed.
- biomcgary 3y ago
  I don't follow Hossenfelder that closely, so I haven't seen any of her videos on economics. I have seen her walk back some of her misunderstanding around global warming, however. Celebrating someone's achievements in a specific domain is not giving them carte blanche across the board. Personally, I think it more productive to celebrate someone's successes while lamenting their failures. If one thinks that mediocrity is the norm (which it is, by definition), then transcending it, even briefly, it something to celebrate and use as inspiration, even while acknowledging that all individuals have flaws (to some degree).
GoblinSlayer 3y ago
>As a biologist, one of the things I most appreciate about Sabine Hossenfelder (a physicist) is that she highlights the limits of knowledge in her (and adjacent) fields. I only heard her arguing others are wrong because she's right. That's not a demonstration of the limits of knowledge.

NikolaNovak 3y ago

I find that coloquially we use the term "science" in several distinct meanings, two of which are: 1. Science as the body of knowledge 2. Science as a method / approach I also find that mixing the meanings/perspectives/intent up in a single conversation is common, sometimes accidentally sometimes intentionally. In my ignorance (ComSci major, so not a real science :), I would describe myself as extremely positive / confident to "Science, the approach/method" and, through that approach, pragmatic about the "Science the current body of knowledge". In other words: Yes, there are very much limits to what we currently know, and some of what we think we know will turn out to be wrong, subtly or catastrophically. There are definitely huge limits and uncertainties to Science the body of knowledge! But, acknowledging it is kinda the point, and the best way to figure it out that we are currently aware of is through scientific approach. (I've just realized I might even have become a zealot that you describe, because I can't even figure out what a plausible & feasible alternative method is, if your goals are to actually figure things out. To that point, I find humility and skepticism about your current science body of knowledge a crucial part of science the method, something which most other methods lack). I find distinction is crucial especially in political and religious discussion frameworks. Otherwise, I never know if I agree or disagree with statements regarding "limits of science" etc. (This is all further mixed up by zillion of daily popular articles where "Science says that [...]!!!" or "[...], scientists find", which... ugh, oversimplify at best and deceive more likely) What are your thoughts?

theGnuMe 3y ago
The science of computing is a real science.
- seanr88 3y ago
  I'd agree with this although I understand where the idea comes from. It is difficult for people to understand what is and is not a science and it is easy to think that computer science is not a science even when you study it. The way we learn and study topics is divorced from the original method of discovering those topics. The way people learn Computer Science is generally by absorbing the information, not by doing the experiments. So it is difficult for people to understand that the way we have this knowledge is through hypothesis forming and experimentation i.e. Science.
- deleted 3y ago
  [deleted]
Sharlin 3y ago
I think there is a third definition of "science": 3. What is actually happening in academia I think most people exposed to science#3 from the inside can agree that science#2 works – and indeed works surprisingly well – despite science#3, not because of it.
- mathematicaster 3y ago
  Top comment. My adjacent refrain is "Science is not the same as people trying to do science".
  - biomcgary 3y ago
    The work of philosophers and sociologists of science, Kuhn and Feyerabend in particular, support this perspective.
  - deleted 3y ago
    [deleted]

eh_why_not 3y ago

> As a scientist, I've seen lots of people who are overconfident in both themselves and overconfident in science. I feel that I've never had the first problem, but have definitely had the second. On one hand, there is the natural limitation of Science itself in terms of the type of questions (that are amenable to the scientific method) it can answer. On the other hand, it is still the best way of generating knowledge that we have. My overconfidence was that scientists, as individuals and as a community, would always do the right thing, driven by, and honestly following, the scientific method. But in the past few years I've had to revisit this assumption several times and be reminded to always retain some healthy skepticism. Most recent example is this climate scientist who just published in Nature, and then went ahead afterwards and penned an op-ed [0] saying he actually misrepresented the actual factors in order to get published. [0] https://www.thefp.com/p/i-overhyped-climate-change-to-get-published https://www.thefp.com/p/i-overhyped-climate-change-to-get-pu...

perfect-blue 3y ago
> My overconfidence was that scientists, as individuals and as a community, would always do the right thing... This is a great point. I'm shocked by how often I end up working on a project with a colleague who is taking the path of least resistance. In my field, this usually results in using decades old statistical methods than have been proven time and time again to be unsatisfactory. They just don't want to learn new methods or their technical expertise aren't good enough to learn how to implement the new approaches. So they just coast. I'm not sure what to do in these situations other than just try and set a positive example.

dekhn 3y ago

23&Me failed in drug development because they started from a mistaken premise- that the data they collected (very specifically, genotype arrays) would produce data that was correlated with human health closely enough to identify targets (proteins or pathways to disrupt/modify). They have a huge genetics dataset, they don't have a huge genomics dataset, and the underlying relationship between the genome and phenotypes (especially complex disease phenotypes) is a highly nonlinear function. Environment is important but we could still have huge improvements in medical care using genomics. It's easy to obtain and still has a very strong relationship to disease, and is a problem best solved by deep learning. I've watched people tilt against this windmill for 25+ years and it's kind of funny just how bad our labelling of diseases is.

biomcgary 3y ago
Excellent point. I work at a biotech startup where one major focus is "how do we fix the disease labels?". I call it the pyrite problem (your gold standard data contains fools gold), but it is known more prosaically as the mislabeling problem.
dahinds 3y ago
This is an inaccurate view of what 23andMe is doing. First we did not "fail" in drug development, our collaboration with GSK yielded substantially more programs than we had anticipated when we started. The verdict on whether the strategy is successful or not will not be clear for another 5-10 years because being better at picking targets doesn't shorten the timeline for bringing a drug to market, it mostly means that you should have a modestly higher proportion of programs that end up being successful. Second we did not start with a premise that we could or needed to do a great job of predicting disease from genetics alone, that is not required to identify good targets. You're correct we don't have huge genomics datasets, we largely use the same genomics datasets others use. Identifying targets with genetics in the simplest sense requires using genetics to identify an association at a genomic location, and then using genomics for functional interpretation of that association. And having the largest database of genetic associations enables the first step of that. I'm not sure what you mean by "the underlying relationship between the genome and phenotypes... is a highly nonlinear function". Simple additive models account for most of the heritability of complex phenotypes.

f1shy 3y ago

The most over confident in theirselves and science people I know are PhD. With egos and arrogance greater than the solar system. People that go early to the industry and see the “real world” tend to have a more tamed expectative of what they could achieve and science can offer.

brigadier132 3y ago

Ironically, you are taking this study at face value. This study reminds me of the "Republicans tend to be sociopaths more often" study. That ended up being completely refuted.

crabmusket 3y ago

> Acknowledging the limits of science is not a negative attitude about science, but a positive one. A clear idea of the current limits of science (both theoretically and practically) is instrumental to pushing through them. "The gods did not reveal, from the beginning, all things to us, but in the course of time through seeking we may learn and know things better. But as for certain truth, no man has known it, nor shall he know it, neither of the gods Nor yet of all the things of which I speak. For even if by chance he were to utter the final truth, he would himself not know it: for all is but a woven web of guesses" Xenophanes via Karl Popper

XTHK 3y ago

>one of the things I most appreciate about Sabine Hossenfelder (a physicist) is that she highlights the limits of knowledge in her (and adjacent) fields. I suppose you missed her recent video about economics? Her content is getting more and more clickbaitey by the day. Some really half-baked data creeping into her videos.

ramraj07 3y ago
It really is sad indeed. You die a hero (stop posting videos once you’ve exhausted your expertise) or live long enough to become a villain indeed.
biomcgary 3y ago
I don't follow Hossenfelder that closely, so I haven't seen any of her videos on economics. I have seen her walk back some of her misunderstanding around global warming, however. Celebrating someone's achievements in a specific domain is not giving them carte blanche across the board. Personally, I think it more productive to celebrate someone's successes while lamenting their failures. If one thinks that mediocrity is the norm (which it is, by definition), then transcending it, even briefly, it something to celebrate and use as inspiration, even while acknowledging that all individuals have flaws (to some degree).

GoblinSlayer 3y ago

>As a biologist, one of the things I most appreciate about Sabine Hossenfelder (a physicist) is that she highlights the limits of knowledge in her (and adjacent) fields. I only heard her arguing others are wrong because she's right. That's not a demonstration of the limits of knowledge.