4 ms·

There's more to it than just testing. If there weren't, then I'd be inclined to agree with the author. What many people fail to realize is that the full form of

There's more to it than just testing. If there weren't, then I'd be inclined to agree with the author. What many people fail to realize is that the full form of the scientific method (when applied properly) is: * make an observation * form a hypothesis that explains the observation * design a test to disprove the hypothesis * carry out the test and collect data * analyze the results and either reject the hypothesis or formulate a new test It is the necessity of a hypothesis to be falsifiable, and the intent of the scientist to disprove (not prove) their hypothesis that sets science apart.

ams6110 10y ago
Why then the seeming reluctance (as posted here recently) of journals to publish "failure to replicate" papers?
- kleer001 10y ago
  Emotion. It's not as exciting (or profitable) to publish a paper that basically says "We tried a lot of things, but none of them worked." That makes everyone feel like they're wasting time. Hopefully that will change soon and labs can keep their funding (from govt and corporate sources) even if they publish reams of negative results.
- jballanc 10y ago
  sigh...Do you want the short answer or the long version? The short answer is: peer pressure. The longer version requires a look back at the history of scientific publishing and how it's been perverted in more recent times. Originally, scientists (or natural philosophers as they were known then) would carry out experiments and record their findings in their personal papers. The problem with this is the question of priority: who discovered what first. If you leave this question up to individuals recording their own results, then there's nothing to stop those individuals from backdating their own writings in a false attempt to claim priority. So a tradition grew of communicating results via letters. When you discover something, don't just write it down, send it in a letter to a fellow natural philosopher. That gives you a (presumably) independent means of verifying priority. Of course, this does nothing to prevent two friends from colluding to falsely claim the mantel of priority. Eventually the tradition morphed. Instead of just writing a letter to a colleague, you would write the letter and the colleague would read it to a gathering of natural philosophers (or, if you had the means of travel, you could read your findings, or in some cases demonstrate them, to the gathering in person). This gives you a much more robust means of claiming priority. Eventually, these societies organized themselves, one of the earliest being the Royal Society. It didn't take long before these groups decided that all the letters being read should be recorded in some format. Thus, you have the "Philosophical Transactions of the Royal Society", the first scientific journal... ...in 1665. Since then the enterprise of scientific publishing has changed very little. Why is this bit of history important? Because the point of the original journals was not to be a full record of the experiments being performed, but rather just of discoveries so that their priority could be established. The scientists of the time still communicated their efforts, both successful and not, with each other in myriad ways. Perhaps more importantly, positions at universities were often based on these personal communications (and recommendations from peers). Fast forward 400 or so years and scientific output (and indeed the number of scientists and universities producing this output) undergoes exponential growth. Suddenly, hiring professors based purely on personal communications becomes impractical. Ah, but what's this? There's also an exponential growth in scientific journals, and some candidates for these positions have their name on more by-lines than other candidates. Surely they must excel at science, no? In other words, scientific journals, which were originally just one part of a scientists work, were perverted to become a stand-in for all of a scientists work. Yet, owing to their history as a means of establishing priority for discovery, the papers published continued to be propelled by tradition to emphasize positive results. If a practice that has remained unchanged for 4 and a half centuries isn't ripe for disruption, I don't know what is!
  - infogulch 10y ago
    Published papers became the University's hire metric. So now work is focused on publishing. The "hire metric" problem seems to have poisoned all jobs, science work not exempt.
    - jballanc 10y ago
      Right, but this wouldn't be as big of a problem for science if it weren't for the fact that the tradition of scientific journals was already focused on publishing positive results. There are efforts underway that may eventually change this (Open Data, Open Publishing, pre-print servers, etc.), but it's a big ship to turn.
- twblalock 10y ago
  Because falsifiability is not part of how most science is done. Most scientists design experiments to verify hypotheses, not to falsify/refute them. The relationship between Karl Popper's idealized scientific method, and science as it is actually practiced, is rather tenuous. Many people who study the scientific method reject Popper's views, although many of the commenters here seem to think that Popper had the final word on the subject. The reality is that science is done by people, and human culture and prejudice, including ego and groupthink, are common in science. This is why some scientists don't like to consider submissions of articles that fail to replicate previous experiments. It's a problem for peer review more broadly.
ubernostrum 10y ago
What many people fail to realize is that the full form of the scientific method Now, wait for someone to reply to you with "Actually the true scientific method is..." and then someone to reply to them with a correction, and then people arguing about verification versus falsification, and then people arguing about the problem of induction... And then maybe you realize that it's not an easy question to answer.
- jballanc 10y ago
  I disagree. It is a very easy question to answer so long as you realize that "science" is a subset of "knowledge". It has been accepted for quite some time now that falsifiability is the key factor distinguishing science from everything that's not science. However there are other forms of knowledge (math, for example) that are not science. Consequently, this is why you'll hear many physicists claim that string theory is not science. It certainly looks like science, and some day it might become science, but currently as the bulk of string theory is not falsifiable, it is not, strictly speaking, science.
  - Retra 10y ago
    Math is falsifiable. You can derive innumerable self-contradictory mathematical systems and we discard them because they are demonstrably useless. The demonstrability of uselessness is falsifiability.
    - dragonwriter 10y ago
      > Math is falsifiable. Not in the sense that propositions of science are, where falsifiability refers to empirical falsification, not logical contradiction. What you are doing is called "equivocation". > You can derive innumerable self-contradictory mathematical systems and we discard them because they are demonstrably useless. No, we reject any set that includes self-contradictions; we may choose which self-consistent set to use in a particular domain because of empirical utility, but, to the extent there is falsification involved their, its not of the math or set of axioms, per se, but of its applicability to the problem domain.
mirimir 10y ago
This! The parent article does not contain "fals*". So it's clearly bullshit, in that it doesn't address what the scientific method is. Also, the point about modification based on contrary evidence is actually a weakness of the scientific method.
twblalock 10y ago
> design a test to disprove the hypothesis That's not what scientists normally do. Popper argued that they should do it, not that they actually do it. Many experiments are designed to verify hypotheses, not to falsify them.

ams6110 10y ago

Why then the seeming reluctance (as posted here recently) of journals to publish "failure to replicate" papers?

kleer001 10y ago
Emotion. It's not as exciting (or profitable) to publish a paper that basically says "We tried a lot of things, but none of them worked." That makes everyone feel like they're wasting time. Hopefully that will change soon and labs can keep their funding (from govt and corporate sources) even if they publish reams of negative results.
jballanc 10y ago
sigh...Do you want the short answer or the long version? The short answer is: peer pressure. The longer version requires a look back at the history of scientific publishing and how it's been perverted in more recent times. Originally, scientists (or natural philosophers as they were known then) would carry out experiments and record their findings in their personal papers. The problem with this is the question of priority: who discovered what first. If you leave this question up to individuals recording their own results, then there's nothing to stop those individuals from backdating their own writings in a false attempt to claim priority. So a tradition grew of communicating results via letters. When you discover something, don't just write it down, send it in a letter to a fellow natural philosopher. That gives you a (presumably) independent means of verifying priority. Of course, this does nothing to prevent two friends from colluding to falsely claim the mantel of priority. Eventually the tradition morphed. Instead of just writing a letter to a colleague, you would write the letter and the colleague would read it to a gathering of natural philosophers (or, if you had the means of travel, you could read your findings, or in some cases demonstrate them, to the gathering in person). This gives you a much more robust means of claiming priority. Eventually, these societies organized themselves, one of the earliest being the Royal Society. It didn't take long before these groups decided that all the letters being read should be recorded in some format. Thus, you have the "Philosophical Transactions of the Royal Society", the first scientific journal... ...in 1665. Since then the enterprise of scientific publishing has changed very little. Why is this bit of history important? Because the point of the original journals was not to be a full record of the experiments being performed, but rather just of discoveries so that their priority could be established. The scientists of the time still communicated their efforts, both successful and not, with each other in myriad ways. Perhaps more importantly, positions at universities were often based on these personal communications (and recommendations from peers). Fast forward 400 or so years and scientific output (and indeed the number of scientists and universities producing this output) undergoes exponential growth. Suddenly, hiring professors based purely on personal communications becomes impractical. Ah, but what's this? There's also an exponential growth in scientific journals, and some candidates for these positions have their name on more by-lines than other candidates. Surely they must excel at science, no? In other words, scientific journals, which were originally just one part of a scientists work, were perverted to become a stand-in for all of a scientists work. Yet, owing to their history as a means of establishing priority for discovery, the papers published continued to be propelled by tradition to emphasize positive results. If a practice that has remained unchanged for 4 and a half centuries isn't ripe for disruption, I don't know what is!
- infogulch 10y ago
  Published papers became the University's hire metric. So now work is focused on publishing. The "hire metric" problem seems to have poisoned all jobs, science work not exempt.
  - jballanc 10y ago
    Right, but this wouldn't be as big of a problem for science if it weren't for the fact that the tradition of scientific journals was already focused on publishing positive results. There are efforts underway that may eventually change this (Open Data, Open Publishing, pre-print servers, etc.), but it's a big ship to turn.
twblalock 10y ago
Because falsifiability is not part of how most science is done. Most scientists design experiments to verify hypotheses, not to falsify/refute them. The relationship between Karl Popper's idealized scientific method, and science as it is actually practiced, is rather tenuous. Many people who study the scientific method reject Popper's views, although many of the commenters here seem to think that Popper had the final word on the subject. The reality is that science is done by people, and human culture and prejudice, including ego and groupthink, are common in science. This is why some scientists don't like to consider submissions of articles that fail to replicate previous experiments. It's a problem for peer review more broadly.

ubernostrum 10y ago

What many people fail to realize is that the full form of the scientific method Now, wait for someone to reply to you with "Actually the true scientific method is..." and then someone to reply to them with a correction, and then people arguing about verification versus falsification, and then people arguing about the problem of induction... And then maybe you realize that it's not an easy question to answer.

jballanc 10y ago
I disagree. It is a very easy question to answer so long as you realize that "science" is a subset of "knowledge". It has been accepted for quite some time now that falsifiability is the key factor distinguishing science from everything that's not science. However there are other forms of knowledge (math, for example) that are not science. Consequently, this is why you'll hear many physicists claim that string theory is not science. It certainly looks like science, and some day it might become science, but currently as the bulk of string theory is not falsifiable, it is not, strictly speaking, science.
- Retra 10y ago
  Math is falsifiable. You can derive innumerable self-contradictory mathematical systems and we discard them because they are demonstrably useless. The demonstrability of uselessness is falsifiability.
  - dragonwriter 10y ago
    > Math is falsifiable. Not in the sense that propositions of science are, where falsifiability refers to empirical falsification, not logical contradiction. What you are doing is called "equivocation". > You can derive innumerable self-contradictory mathematical systems and we discard them because they are demonstrably useless. No, we reject any set that includes self-contradictions; we may choose which self-consistent set to use in a particular domain because of empirical utility, but, to the extent there is falsification involved their, its not of the math or set of axioms, per se, but of its applicability to the problem domain.

mirimir 10y ago

This! The parent article does not contain "fals*". So it's clearly bullshit, in that it doesn't address what the scientific method is. Also, the point about modification based on contrary evidence is actually a weakness of the scientific method.

twblalock 10y ago

> design a test to disprove the hypothesis That's not what scientists normally do. Popper argued that they should do it, not that they actually do it. Many experiments are designed to verify hypotheses, not to falsify them.