3 ms·

I don't think reproducibility is about scientific method. It is about trust -- ie is the claim correct and stable. This is also why reproducibility is important

by ivalm 7y ago

I don't think reproducibility is about scientific method. It is about trust -- ie is the claim correct and stable. This is also why reproducibility is important in science. I don't think the scientific method (in the sense of hypothesis->experiment->observation->theory) has anything to do with reproducibility. Frankly, I am not even sure it is necessary for good work, whether it is in ML/DL or in an established basic science. Scientific method is definitely not how most practitioners experience doing research.

hooande 7y ago
is there an alternative to the scientific method that I don't know about? I don't think I've seen anyone question the basic framework before
- Tarq0n 7y ago
  Hypothesis generation is definitely also a common goal of scientific work. There are many situations in which you don't have a good prior to base an experiment on.
- SiempreViernes 7y ago
  "the scientific method" is a somewhat nebulous concept in daily discussions, as evidenced by the guy above defining such that reproducibility is not part of it. Personally, I would count reproducibility as a fairly important part of the scientific method, as it demonstrates you've achieved the fundamental requirement of understanding your experimental setup. Without very detailed understanding of your experiment, the experiment isn't scientific, so by extension I would claim that the GP's "scientific method" isn't scientific.
- samcodes 7y ago
  Eliezer Yudkowsky wrote a lot about (many things, including) refining the scientific method in Rationality [0]. IIRC, his point is that the scientific method puts no constraints on what hypotheses you test. He thinks a more orderly search of hypothesis space is preferable. [0] https://www.readthesequences.com/ https://www.readthesequences.com/
sgt101 7y ago
The gold standard is h->e->o->t->p->o - you use the theory to create a new prediction that is confirmed by observation thus supporting the theory. This is what is so compelling about 20thC physics - so many insights and consequence from the theory that are then supported by subsequent observation and experiment.

by ivalm 7y ago

hooande 7y ago

is there an alternative to the scientific method that I don't know about? I don't think I've seen anyone question the basic framework before

Tarq0n 7y ago
Hypothesis generation is definitely also a common goal of scientific work. There are many situations in which you don't have a good prior to base an experiment on.
SiempreViernes 7y ago
"the scientific method" is a somewhat nebulous concept in daily discussions, as evidenced by the guy above defining such that reproducibility is not part of it. Personally, I would count reproducibility as a fairly important part of the scientific method, as it demonstrates you've achieved the fundamental requirement of understanding your experimental setup. Without very detailed understanding of your experiment, the experiment isn't scientific, so by extension I would claim that the GP's "scientific method" isn't scientific.
samcodes 7y ago
Eliezer Yudkowsky wrote a lot about (many things, including) refining the scientific method in Rationality [0]. IIRC, his point is that the scientific method puts no constraints on what hypotheses you test. He thinks a more orderly search of hypothesis space is preferable. [0] https://www.readthesequences.com/ https://www.readthesequences.com/

sgt101 7y ago

The gold standard is h->e->o->t->p->o - you use the theory to create a new prediction that is confirmed by observation thus supporting the theory. This is what is so compelling about 20thC physics - so many insights and consequence from the theory that are then supported by subsequent observation and experiment.