6 ms·

Source? I don't hear about rampant replication problems in chemistry/physics. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Re

by acover 8y ago

Source? I don't hear about rampant replication problems in chemistry/physics. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis

nordsieck 8y ago
This[0] is a great video talking about problems with lots of studies in biology-adjacent fields including, but not limited to, psychology and nutrition. [0] John Ioannidis: "Reproducible Research: True or False?" https://www.youtube.com/watch?v=GPYzY9I78CI https://www.youtube.com/watch?v=GPYzY9I78CI
- acover 8y ago
  Interesting! Sadly medicine has a replication crisis. I don't think chemistry or physics does.
mcguire 8y ago
I'll just leave a mention of the current state of theoretical physics here: https://www.nature.com/articles/d41586-018-05374-9 https://www.nature.com/articles/d41586-018-05374-9
- tormeh 8y ago
  That's a criticism of a bunch of heuristics we've had for selecting promising theories. No popular theory has been proven wrong. It's just that we've been stuck for a really long time. And we need some kind of heuristics, otherwise we're just going to overfit to our data.
schuetze 8y ago
The replicability of the biomedical sciences is just as bad if not worse than in psychology. [1] http://www.slate.com/articles/health_and_science/future_tense/2016/04/biomedicine_facing_a_worse_replication_crisis_than_the_one_plaguing_psychology.html http://www.slate.com/articles/health_and_science/future_tens... [2] https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970 https://www.nature.com/news/1-500-scientists-lift-the-lid-on...
- acover 8y ago
  Yes, that is covered in my wikipedia link and a comment below. There is a replication crisis in many fields. Not every field.
  - thaumasiotes 8y ago
    There is a replication crisis in every field that is not disciplined by a need to achieve externally-defined goals. Chemistry is in good shape because people apply it in order to e.g. create steel. The difference between a scientist and an engineer is that the scientist answers the question "if I do X, what will happen?", and the engineer answers the question "what do I do in order to get Y to happen?". But if there are no engineers working off of the results, the scientist is free to say anything. Sadly, as pointed out elsewhere in the subthread, this is a sufficient but not a necessary condition. Medicine has plenty of well-defined goals, but it doesn't know how to meet them. The cheap wisdom there is: if you want something badly enough, there will always be someone willing to sell it to you, whether or not they actually have it.
iron0013 8y ago
If you're interested in machine learning/deep learning/artificial intelligence, for example, I've got bad news about a lot of those papers that appear at NIPS...
- Baeocystin 8y ago
  I've yet to see anything from the current boom be anything more than empirical curve fitting in fancy clothes. I've lived through enough AI winters to not be surprised. Still disappointed.
  - thanatropism 8y ago
    UMAP is pretty cool. It's like the realization of a dream that's been hanging around (when it was called "multidimensional scaling") since the mid-20th century.

by acover 8y ago

Source? I don't hear about rampant replication problems in chemistry/physics. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis

nordsieck 8y ago

This[0] is a great video talking about problems with lots of studies in biology-adjacent fields including, but not limited to, psychology and nutrition. [0] John Ioannidis: "Reproducible Research: True or False?" https://www.youtube.com/watch?v=GPYzY9I78CI https://www.youtube.com/watch?v=GPYzY9I78CI

acover 8y ago
Interesting! Sadly medicine has a replication crisis. I don't think chemistry or physics does.

mcguire 8y ago

I'll just leave a mention of the current state of theoretical physics here: https://www.nature.com/articles/d41586-018-05374-9 https://www.nature.com/articles/d41586-018-05374-9

tormeh 8y ago
That's a criticism of a bunch of heuristics we've had for selecting promising theories. No popular theory has been proven wrong. It's just that we've been stuck for a really long time. And we need some kind of heuristics, otherwise we're just going to overfit to our data.

schuetze 8y ago

The replicability of the biomedical sciences is just as bad if not worse than in psychology. [1] http://www.slate.com/articles/health_and_science/future_tense/2016/04/biomedicine_facing_a_worse_replication_crisis_than_the_one_plaguing_psychology.html http://www.slate.com/articles/health_and_science/future_tens... [2] https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970 https://www.nature.com/news/1-500-scientists-lift-the-lid-on...

acover 8y ago
Yes, that is covered in my wikipedia link and a comment below. There is a replication crisis in many fields. Not every field.
- thaumasiotes 8y ago
  There is a replication crisis in every field that is not disciplined by a need to achieve externally-defined goals. Chemistry is in good shape because people apply it in order to e.g. create steel. The difference between a scientist and an engineer is that the scientist answers the question "if I do X, what will happen?", and the engineer answers the question "what do I do in order to get Y to happen?". But if there are no engineers working off of the results, the scientist is free to say anything. Sadly, as pointed out elsewhere in the subthread, this is a sufficient but not a necessary condition. Medicine has plenty of well-defined goals, but it doesn't know how to meet them. The cheap wisdom there is: if you want something badly enough, there will always be someone willing to sell it to you, whether or not they actually have it.

iron0013 8y ago

If you're interested in machine learning/deep learning/artificial intelligence, for example, I've got bad news about a lot of those papers that appear at NIPS...

Baeocystin 8y ago
I've yet to see anything from the current boom be anything more than empirical curve fitting in fancy clothes. I've lived through enough AI winters to not be surprised. Still disappointed.
- thanatropism 8y ago
  UMAP is pretty cool. It's like the realization of a dream that's been hanging around (when it was called "multidimensional scaling") since the mid-20th century.