6 ms·

> it's insane that metascience isn't studied more. It is, probably more than is apparent. See: https://en.wikipedia.org/wiki/Philosophy_of_science https://en.

> it's insane that metascience isn't studied more. It is, probably more than is apparent. See: https://en.wikipedia.org/wiki/Philosophy_of_science https://en.wikipedia.org/wiki/Philosophy_of_science https://en.wikipedia.org/wiki/Sociology_of_scientific_knowledge https://en.wikipedia.org/wiki/Sociology_of_scientific_knowle... There is an interesting recursive problem here, though: what tools do you use to scientifically analyze the scientific process? Whatever tools you use will themselves be hobbled by the same systemic flaws you are trying to understand. Also, as any sociologist will be happy to tell you, incentive structures and other human group behavior gets in the way. It's probably hard to get funding for a study that shows that all the other departments at your university aren't quite the flawless seekers of truth they appear to be.

chromaton 5y ago
>what tools do you use to scientifically analyze the scientific process? Iterating on existing systems to see if you can get results to converge and also testing new systems to see if they also result in known good values.
- licebmi__at__ 5y ago
  That seems broad enough to describe the human history isn’t it? So, “keep doing what you do”?
  - chromaton 5y ago
    Not necessarily. You could just fall back on authority, superstition, or navel-gazing.
    - dpe82 5y ago
      That would probably fall under "testing [new/other] systems to see if they also result in known good values."
- munificent 5y ago
  It's fun to realize how deeply ingrained the current scientific process is in our way of thinking. All of these ideas that you tacitly take for granted are itself mutable parts of the scientific process: * That iterative improvement and hill-climbing is an effective process for improving results. * That replication of experiments and convergence is a truth-generating enterprise. * That truth can be expressed numerically. * That there are some values that are "known good". By what process? According to whom? To be clear, I don't disagree with those. However, these rules aren't baked into the firmament of the universe. They are processes we humans have chosen to apply in our social process of reaching conensus on truth. In other words, this list here isn't physics, it's technology. It's entirely possible to imagine a culture whose truth finding bodies don't take for granted one or more of these rules at all. That culture might be more or less effective (again, according to what metrics?), but it would still be well-defined.
  - xxpor 5y ago
    The only nit I'd pick with the list is: * That truth can be expressed numerically. Isn't the basic point of quantum physics that this isn't true? We can only make guesses with probabilities, but we can't know the actual truth, and therefore can't express it numerically.
    - munificent 5y ago
      Sure, but the probabilities are numbers too. Which is again sort of acknowledging the need to fit quantum mechanics into a numeric framework. Imagine you were studying ice cream flavors. You might design a study like, "We'll ask a lot of people and the flavor that the most people prefer is the best." In other words, the metaprocess you use to design your experiment itself tacitly assumes you need a numeric result. The presumption of comparison and quantifying frames the questions you even think to ask. But you can imagine an alternate culture that when studying ice cream flavors doesn't even ask questions with numeric answers. It could be, "We'll ask a lot of people to try flavors and write poems about the experience." We wouldn't even call this "science". Because there is a hidden border around even the term that affects how we are able to evolve the scientific process.
analog31 5y ago
Indeed, and if you think science is bad, imagine metascience.
- fnord123 5y ago
  When you think about it, it's insane that metametascience isn't studied more. We're going to question every little detail about the scientific process, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" metascience?
  - caenn 5y ago
    Indeed, and if you think metascience is bad, imagine meta-metascience.
    - analog31 5y ago
      Where's tail call optimization when you need it?
      - jonsen 5y ago
        Would be nice if tail call optimization solved the halting problem.
    - IIAOPSW 5y ago
      You've caught an Orobus by the tail!
    - svieira 5y ago
      Meta-metascience has been studied _extensively_ by previous generations, where what this generation calls "science" was called "natural philosophy" and interesting discourse and study was had on such things as: * The study of knowledge (epistemology): https://en.wikipedia.org/wiki/Epistemology https://en.wikipedia.org/wiki/Epistemology * The study of existence and what exists (ontology): https://en.wikipedia.org/wiki/Ontology https://en.wikipedia.org/wiki/Ontology * The study of the purpose of things (teleology): https://en.wikipedia.org/wiki/Teleology https://en.wikipedia.org/wiki/Teleology It turns out that these are hard areas of study and it requires a lot of properly focused leisure time to understand properly. Most people don't consider these things, wing it, and wind up working with a half-baked meta-meta science of their own creation ... oh, I see what you mean :-D
rijoja 5y ago
Yeah so it is not science but rather philosophy and it is called Epistemology and not metascience. Even though I suppose metascience would be an apt description in a way.
fallingknife 5y ago
> what tools do you use to scientifically analyze the scientific process? Engineering. If you can build something that works based on the rules theorized by scientists, they are on to something. e.g. building a skyscraper proves we know the properties of steel to a pretty good margin of error.
- BurningFrog 5y ago
  For many sciences, like psychology, that's not normally an option.
  - sprafa 5y ago
    Psychology is basically applied neuroscience. They are both in their infancy and we are using the best information we can, but I wouldn't call the body of work comparable to other sciences.
  - thu2111 5y ago
    Which raises the question why we study this subject at all, if it's not actually applicable to anything beyond making viral TED talks.
nicoburns 5y ago
I might be biased as someone who studied philosophy. But I wish Philosophy of Science was mandatory in more science degrees. A lot of scientists don't seem to be familiar.
- andi999 5y ago
  Can you point to something good and useful in the Philosophie of Science? I mean let's say physics is as good as it gets (laws in mathematical language, strong experimental evidence, solid results); from my perspective a success. But it has nothing to do with how like popper imagines science. So insisting on like a popper process would also not be benificial for physics in my opinion. But probably there is better stuff, I just couldn't find it. (with a popper mindset nobody would understand why we still study 'falsified' theories like electrodynamics)
  - bsder 5y ago
    > Can you point to something good and useful in the Philosophie of Science? Every scientist should be able to answer the particular tenets that they take on faith. Scientists do take on faith that the universe is causal and that the rules today are the same as the rules yesterday. Yes, scientists double check these assumptions over and over, but they can never "prove" them. This kind of introspection is important for science to set itself apart from religion, for example.
    - andi999 5y ago
      I meant more like book, or a philosopher.
  - nicoburns 5y ago
    Some key things for me would be the ideas that: 1. There is no such thing as objectivity in inquiry. There is always a scientist interpreting things, and they are always looking at things through the lens of their own biases and cultural norms. The best you can hope to do is to be aware of your limitations. 1. Statistical evidence on its own is not always a good basis for believing something to be true. Typically you also want a good theoretical model, and an understanding of the mechanism that underlies the observed phenomena (physics is good at this, psychology not so much). 3. That there is more knowledge than is detectable through statistical methods (currently at least). And that lack of evidence from statistical studies does not necessarily constitute good evidence that a theory is false.

chromaton 5y ago

>what tools do you use to scientifically analyze the scientific process? Iterating on existing systems to see if you can get results to converge and also testing new systems to see if they also result in known good values.

licebmi__at__ 5y ago
That seems broad enough to describe the human history isn’t it? So, “keep doing what you do”?
- chromaton 5y ago
  Not necessarily. You could just fall back on authority, superstition, or navel-gazing.
  - dpe82 5y ago
    That would probably fall under "testing [new/other] systems to see if they also result in known good values."
munificent 5y ago
It's fun to realize how deeply ingrained the current scientific process is in our way of thinking. All of these ideas that you tacitly take for granted are itself mutable parts of the scientific process: * That iterative improvement and hill-climbing is an effective process for improving results. * That replication of experiments and convergence is a truth-generating enterprise. * That truth can be expressed numerically. * That there are some values that are "known good". By what process? According to whom? To be clear, I don't disagree with those. However, these rules aren't baked into the firmament of the universe. They are processes we humans have chosen to apply in our social process of reaching conensus on truth. In other words, this list here isn't physics, it's technology. It's entirely possible to imagine a culture whose truth finding bodies don't take for granted one or more of these rules at all. That culture might be more or less effective (again, according to what metrics?), but it would still be well-defined.
- xxpor 5y ago
  The only nit I'd pick with the list is: * That truth can be expressed numerically. Isn't the basic point of quantum physics that this isn't true? We can only make guesses with probabilities, but we can't know the actual truth, and therefore can't express it numerically.
  - munificent 5y ago
    Sure, but the probabilities are numbers too. Which is again sort of acknowledging the need to fit quantum mechanics into a numeric framework. Imagine you were studying ice cream flavors. You might design a study like, "We'll ask a lot of people and the flavor that the most people prefer is the best." In other words, the metaprocess you use to design your experiment itself tacitly assumes you need a numeric result. The presumption of comparison and quantifying frames the questions you even think to ask. But you can imagine an alternate culture that when studying ice cream flavors doesn't even ask questions with numeric answers. It could be, "We'll ask a lot of people to try flavors and write poems about the experience." We wouldn't even call this "science". Because there is a hidden border around even the term that affects how we are able to evolve the scientific process.

analog31 5y ago

Indeed, and if you think science is bad, imagine metascience.

fnord123 5y ago
When you think about it, it's insane that metametascience isn't studied more. We're going to question every little detail about the scientific process, but we're just going to take on faith that peer-review and publication in journals is an effective method for weeding out "bad" metascience?
- caenn 5y ago
  Indeed, and if you think metascience is bad, imagine meta-metascience.
  - analog31 5y ago
    Where's tail call optimization when you need it?
    - jonsen 5y ago
      Would be nice if tail call optimization solved the halting problem.
  - IIAOPSW 5y ago
    You've caught an Orobus by the tail!
  - svieira 5y ago
    Meta-metascience has been studied _extensively_ by previous generations, where what this generation calls "science" was called "natural philosophy" and interesting discourse and study was had on such things as: * The study of knowledge (epistemology): https://en.wikipedia.org/wiki/Epistemology https://en.wikipedia.org/wiki/Epistemology * The study of existence and what exists (ontology): https://en.wikipedia.org/wiki/Ontology https://en.wikipedia.org/wiki/Ontology * The study of the purpose of things (teleology): https://en.wikipedia.org/wiki/Teleology https://en.wikipedia.org/wiki/Teleology It turns out that these are hard areas of study and it requires a lot of properly focused leisure time to understand properly. Most people don't consider these things, wing it, and wind up working with a half-baked meta-meta science of their own creation ... oh, I see what you mean :-D

rijoja 5y ago

Yeah so it is not science but rather philosophy and it is called Epistemology and not metascience. Even though I suppose metascience would be an apt description in a way.

fallingknife 5y ago

> what tools do you use to scientifically analyze the scientific process? Engineering. If you can build something that works based on the rules theorized by scientists, they are on to something. e.g. building a skyscraper proves we know the properties of steel to a pretty good margin of error.

BurningFrog 5y ago
For many sciences, like psychology, that's not normally an option.
- sprafa 5y ago
  Psychology is basically applied neuroscience. They are both in their infancy and we are using the best information we can, but I wouldn't call the body of work comparable to other sciences.
- thu2111 5y ago
  Which raises the question why we study this subject at all, if it's not actually applicable to anything beyond making viral TED talks.

nicoburns 5y ago

I might be biased as someone who studied philosophy. But I wish Philosophy of Science was mandatory in more science degrees. A lot of scientists don't seem to be familiar.

andi999 5y ago
Can you point to something good and useful in the Philosophie of Science? I mean let's say physics is as good as it gets (laws in mathematical language, strong experimental evidence, solid results); from my perspective a success. But it has nothing to do with how like popper imagines science. So insisting on like a popper process would also not be benificial for physics in my opinion. But probably there is better stuff, I just couldn't find it. (with a popper mindset nobody would understand why we still study 'falsified' theories like electrodynamics)
- bsder 5y ago
  > Can you point to something good and useful in the Philosophie of Science? Every scientist should be able to answer the particular tenets that they take on faith. Scientists do take on faith that the universe is causal and that the rules today are the same as the rules yesterday. Yes, scientists double check these assumptions over and over, but they can never "prove" them. This kind of introspection is important for science to set itself apart from religion, for example.
  - andi999 5y ago
    I meant more like book, or a philosopher.
- nicoburns 5y ago
  Some key things for me would be the ideas that: 1. There is no such thing as objectivity in inquiry. There is always a scientist interpreting things, and they are always looking at things through the lens of their own biases and cultural norms. The best you can hope to do is to be aware of your limitations. 1. Statistical evidence on its own is not always a good basis for believing something to be true. Typically you also want a good theoretical model, and an understanding of the mechanism that underlies the observed phenomena (physics is good at this, psychology not so much). 3. That there is more knowledge than is detectable through statistical methods (currently at least). And that lack of evidence from statistical studies does not necessarily constitute good evidence that a theory is false.