10 ms·

The paper here seems to make absolutely zero distinction between deductive and inductive reasoning, which should be the entire point. Math is an arbitrary fram

by scoofy 5y ago

The paper here seems to make absolutely zero distinction between deductive and inductive reasoning, which should be the entire point. Math is an arbitrary framework, built from arbitrary axioms. It is deductive. Thus, all proofs are simply deduction. The knowledge here is positive knowledge, we can show things are true, false, or undecidable. There may be errors, but those are errors in execution. Psychology is not built on a framework. It is inductive, we are trying to find axioms that map to the data we collect. Thus, all papers are trying to add/build it's arbitrary framework. The only knowledge here is negative knowledge, falsification, we know only what is a failed hypothesis. There will be errors, both in execution, and there will also be statistical errors in experimental results. The entire point of the replication crisis is that we don't publish or pay attention to results that are boring, so the framework we build is built on skewed data. We don't reject previously popular papers that are now unfalsifiable (the idea that the now unfalsifiable Milgram experiment is still taught in every university psychology dept should be outrageous). The boring results need to be weight statistically against the interesting results, but aren't, etc. Nobody out there is arguing whether or not the axiom of choice is a true axiom? It sort of doesn't matter, it can't matter, because it's arbitrary by definition. You can't have a replication crisis inside of a deductive framework without changing the framework. This doesn't happen too often, but we did see this during the shift from neutonian to einsteinian physics. The study of philosophy of science is fairly obscure, but is the center of this discussion.

netizen-936824 5y ago
I don't believe I'm familiar with an Ingram experiment, any chance you could expand on this? I was also unable to find something with a quick internet search, however Milgram was suggested to me.
- scoofy 5y ago
  Milgram experiment, sorry... must have got hit by autocorrect there: https://en.wikipedia.org/wiki/Milgram_experiment https://en.wikipedia.org/wiki/Milgram_experiment You can add the Stanford Prison Experiment (Zimbardo) to the list of, now unfalsifiable, experiments that inexplicably are regularly taught in university settings when they cannot possibly provide any useful data to the sciences: https://en.wikipedia.org/wiki/Stanford_prison_experiment https://en.wikipedia.org/wiki/Stanford_prison_experiment The idea that many if not most universities create educational programs based on these very obvious problematic studies, leads me to believe that the scientific community can be as guilty of info-tainment bias as the general public. I only wish that I'd been able to continue in academia so that i could have at least a small voice in changing this problematic dynamic we live with. They literally made a movie about the Zimbardo experiment in 2015. It couldn't be more obvious that people care more that it be true than whether it might be false: https://en.wikipedia.org/wiki/The_Stanford_Prison_Experiment_(film) https://en.wikipedia.org/wiki/The_Stanford_Prison_Experiment...
  - zzzeek 5y ago
    what do you mean when you say "now unfalsifiable", is that because the experiments themselves are too unethical to be repeated?
    - netizen-936824 5y ago
      That's correct, however students are being told about the manipulation these days. Especially concerning the prison experiment.
    - mannerheim 5y ago
      The Zimbardo experiment, at least, has actually been falsified, so I don't think it's right to call it unfalsifiable.
stavros 5y ago
Exactly, I can't understand what the author was thinking. If a paper shows that 2+2=4, what would the replication problem be? You write the paper out again and 2+2 turns out to equal 5? "The results can't be replicated" is different from "the logic here is wrong". So different that this article starts from an entirely invalid premise.
- bo1024 5y ago
  From the post: > More seriously, it’s reasonably well-known among mathematicians that published math papers are full of errors.[1] I think the author's point is that actually the distinction is less clear: many math papers can't be (easily) replicated, yet people in math aren't too worried. [1] https://twitter.com/benskuhn/status/1419281164951556097 https://twitter.com/benskuhn/status/1419281164951556097
  - stavros 5y ago
    But there's a difference between "can't get the results a second time" and "didn't even get the results the first time". There isn't even so much a notion of "results", just "logically, Y follows from X". If that's wrong, you don't need to run the experiment again (there is no experiment to run), the logic is just flawed.
    - topaz0 5y ago
      Right -- you need to run the argument again.
    - bo1024 5y ago
      I don’t think it’s quite that simple. A math paper generally says 3 kinds of things: X is true, this is the high level idea of how we can prove X is true, and this is the detailed proof that X is true. As the post mentions, usually when there’s a mistake, X is still true.
      - ConceptJunkie 5y ago
jcranberry 5y ago
I think you're wrong on some of these points, and the author was trying to make these points but was not explicit. I remember reading an article in the intelligencer by a mathematician who essentially said there are certain conjectures where if they were to be proved false rather than be thrown into a sea of uncertainty, mathematicians i would quickly move to investigate a readjustment of basic axioms rather than accept that those conjectures are incorrect. Then there are fields of mathematics around selecting different axioms. Investigating the ramifications of whether you take the undecidable "continuum hypothesis" as true or false. And then theres model theory and such. Presumably they study models of interest and not arbitrary ones. You're mostly correct that the methodology is mostly deductive but the point is that what we choose to use isn't arbitrary because there are things in math which are more important than axioms as they are things believed to be "real".
- scoofy 5y ago
  I don’t mean "arbitrary" in the sense that it doesn’t matter to the engineering of a building. I mean "arbitrary" in the sense that unprovableness within the framework. Axioms are by definition arbitrary in that sense. edit: I'll try to expand on this. Math is weird because it's a sort-of in-framework study. We use math when we do physics, and we use physics when we do engineering. If the physical world ever started disagreeing with our physics, then we need new physics. If the math ever started showing inconsistent results in our physics, then we would need new math. None of that is to say that Neutonian Physics, say, is "wrong in the sense that it's internally inconsistent", only that "it's wrong in the sense that it is not an accurate mapping of a framework to the world we find ourselves in." The first type of wrongness is the type of wrongness we concern ourselves with in deductive studies, the second type, the mapping-error, is inductive, and requires meta-analysis, and is prone to (Hume's) problem of induction, which means it's ultimately unknowable (i.e. Popper).
  - cormacrelf 5y ago
    Here is where you're wrong: > You can't have a replication crisis inside of a deductive framework without changing the framework. You may change the framework, but the replication crisis would be that people don't notice it's changed. A proof can get so huge and complex that it would take a lifetime of study to understand it. One of the article's good points was that "you can replicate a math paper by reading it", but if you cannot read a proof, you cannot replicate it. If nobody can understand it, nobody can replicate it. There would be a big crisis if people started trusting huge papers blindly and using their results without stating them as assumptions. Mathematicians are largely not doing that, so there isn't really a crisis. But they might! And you would not notice that the framework had become inconsistent for a while. Therein would lie the crisis. It is nice to think that as soon as the framework changed, people would notice, because surely if it the ground moved, that would be obvious! But it is not true. Nothing about the deductive-ness or in-framework-ness of mathematics changes this. It is the same for psychology, the replication crisis is a crisis because people do not notice these results are wrong, and then rely on them in clinical guidelines and affect people's lives for generations. It is possible to stave off replication crises by using a machine, because mathematics is self-contained and you can use a computer to show that you proved something using only these specific assumptions and they can believe you without reading and understanding it fully. But this doesn't mean mathematics cannot ever have a replication crisis. I think your arguments here are best summarised as confusing "replication crisis" with "any published results being inconsistent with previous results". Your idea of "inconsistent" is also strange, the article considers this to mean mistakes in proofs, whereas you mean new mathematics that changes the field by discovery. You're just talking about something completely foreign from the concept of a replication crisis.
deleted 5y ago
[deleted]
cubefox 5y ago
This doesn't make sense. This is actually addressed in the piece. If inductive reasoning is irrelevant for mathematics, why do then mathematicians have excellent intuitions about what is true even before they manage to prove it? The quote from the essay: "[D]espite the fact that error-correction is really hard, publishing actually false results was quite rare because “people’s intuition about what’s true is mysteriously really good.” Because we mostly only try to prove true things, our conclusions are right even when our proofs are wrong." Deduction can't explain this surprising reliability of mathematical intuition, only induction can. The question the author tries to answer is why the intuitions of mathematicians seem to be more reliable than those of social scientists.
- scoofy 5y ago
  I mean, again, this is a very messy thing to talk about explicitly because it's playing fast and loose with language. I think the fact that our brains are computing/pattern-recognition machines is the obvious reason computing (deducing) is 'easier'. However, human minds are not actually mysteriously really good at math. Our brains are very clearly broken at some types of mathematical problems. It's just something acedemics tell themselves. When it comes to gambling, large numbers, power laws, etc. The number of times I've had to explain to my friends and family how exponential growth "will appear" with the spread of covid is testament to the fact that our brains don't actually have that math built in, probably because it wasn't particularly important for evolutionary survival and reproduction. The entire field of behavior economics studies this delta specifically. I'd say the deductiveness of a problem, itself, is what makes people have the intuitions. It's vastly easier to use deduction that it is to do induction... because induction is by ultimately unknowable and effectively solipsistic.
  - walleeee 5y ago
    Humans in general may be prone to all sorts of erroneous mathematical reasoning but the speed and consistency with which mathematical concepts reproduce in the minds of mathematicians can justifiably be called mysterious imo, at least until we better understand human language faculties
JetAlone 5y ago
The results of the Milgram and Stanford Prison Experiments (the amount of harm caused by studying obedience and harm to humans) were instrumental in giving us the very ethics codes that make them unfalsifiable. That makes them an important facet of the field of psychology's history and worth teaching for that alone. If you believe that ethics code is appropriate, then you believe those studies are valid and don't need to be run again to re-verified or falsified. If you really don't trust the ethics code which was produced in reaction to the results of these 20th century experiments, then the authentic thing to do would be to re-run the experiments clandestinely and disseminate the results anonymously.
- zozbot234 5y ago
  > If you believe that ethics code is appropriate, then you believe those studies are valid and don't need to be run again to re-verified or falsified. Why? The experiments would still be unethical, even if they led to the opposite result.
  - jstanley 5y ago
    Why? If the opposite result is "some people are prisoners, some are guards, and they all sit around and have a jolly old time" then what's unethical about running that experiment?
- scoofy 5y ago
  The ethical codes are extremely important, that's not my point. I'm saying we are taught the results are correct, when, due to the impossibility of reproduction we can't know whether the results are correct or incorrect. What should be discussed in the Milgram experiment is "we can't know the results of the experiment because one of the subjects killed themselves due to their participation. Try to consider the ethical implications, don't design experiments that might lead to psychological trauma, etc." The idea that the results being true is why we shouldn't do the experiment is wrong to do, is nonsense, because we can't know whether the results are true because can't replicate the experiment. Yet, it's results are published as a pop-science book: https://en.wikipedia.org/wiki/Obedience_to_Authority:_An_Experimental_View https://en.wikipedia.org/wiki/Obedience_to_Authority:_An_Exp... Instead, these two studies if their findings, not the ethics, are discussed at all, are a perfect illustration of the problems that Karl Popper discusses regarding falsifiability.
topaz0 5y ago
You may not have read very far into the post, because I think it deals quite well with these issues, if not in the same terms.
walleeee 5y ago
Why should the Milgram authority experiment not be taught? If nothing else it's suggestive and might inspire new studies. The fact that the original design wouldn't pass a modern IRB doesn't mean the phenomenon is forever closed to investigation
- scoofy 5y ago
  Yes, of course, the Milgram experiment can be taught as a tragedy and a lesson in experimental ethics. But it's "correct results" are almost always taught, even being published as a pop-science book, when they cannot be know because the experiment is too ethically problematic to be replicated in a modern setting. Ironically, the "correct results" are often cited as the reason for the tragic suicide of one of the participants.
  - walleeee 5y ago
    I'm just having trouble with "almost always" Any psychologist worth their salt knows better than to claim any interpretation of experimental results as unassailable Pop science is another story (literally) but the profs I've had the privilege of studying under spent a lot of time warning us against the shoddy reasoning you describe Psychology has a lot of bullshit in its orbit but the discipline as taught in quality departments is just that, disciplined
    - scoofy 5y ago
      I don't mean to impugn the study of psychology in the slightest. Only that i find it maddening that obviously unfalsifiable results are taught as though they can teach us anything about human behavior. I hope you are right and I am wrong, I was taught these experiments this way at a good school, i can only hope that is more uncommon than i think it is.
      - walleeee 5y ago
        Fair enough- and agreed. I hope so too.
bawolff 5y ago
If you replace math with (experimental) physics, you get an inductive framework, yet physics doesn't have the same problems psychology has.
- peteradio 5y ago
  Physics has calculable observation biases baked in. At least you can create experiments where you knowingly limit those biases based on some serious fundamentals. All other fields have so much more to worry about, it really isn't fair.
- majormajor 5y ago
  Does the behavior of a particle depend in part on every other experience the particle has had in the past? Physics has things that we can't figure out, and things that have to be observed in the aggregate, but it's much easier to perform multiple experiments with the same base conditions. If we could clone the participants of a psychology experiment prior to the experiment, with their memories and personalities intact, we could possibly narrow the gap. Or maybe not! But that would be a fascinating finding! ;)
- tonyarkles 5y ago
  See https://en.wikipedia.org/wiki/Oil_drop_experiment#Fraud_allegations https://en.wikipedia.org/wiki/Oil_drop_experiment#Fraud_alle... and the following section. Physics isn’t immune to it.
  - peteradio 5y ago
    The fraud allegations are bogus (according to wikipedia). "Millikan's experiment as an example of psychological effects in scientific methodology" is in itself a psychological study. Has this wikipedia source been peer reviewed and replicated? This is quibbling about fractions of a percent of the secondary measurement, the primary result being quite obvious quantization of electric charge. And! Ultimately the charge measurement result converged on the result known today within 10 years of his original experiment. That's what immunity looks like.
  - 5y ago
kalimanzaro 5y ago
By your dichotomy, Physics seems closer to Psychology than math. Why aren't there replication crises in Physics? My answer is that there always has been-- it's just that they get resolved quick. The problem with Psych is that it is SLOW
- raverbashing 5y ago
  Because it is "trivially" easier to do a better experiment in Physics (money aside, of course) Psychology will always have ethic, sample size and cultural/time/space constraints. Your "psychology experiment" done with Mechanical Turk is more an exercise on sampling bias than anything else And they keep running into "replication failures" because they fail to account for that and try to chase the illusion that it is possible to have a perfect controlled environment in psych. They can't, and they're only fooling themselves they can.
- gpt5 5y ago
  No - both you and the OP miss the real problem. The problem with Psych is that you can't really experiment on humans in a meaningful way. Animal studies for example don't suffer the same replication problems as psych.
  - kalimanzaro 5y ago
    Well perhaps if animal studies were primarily concerned with investigating the interaction of reason and emotion in animals there might just well be :)
  - bildung 5y ago
    > The problem with Psych is that you can't really experiment on humans in a meaningful way While absolutely valid this isn't the only problem: One other complication is that conclusions from experimental studies often don't apply to the real world, i.e. people behave differently in experimental settings. Thus in any case you can either have good control over confounding varables but very limited scope (experimental studies) or bad control over the setting but the ability to observe real live behavior (field studies). But it gets worse: Two people with identical behaviour can have quite different motivations and thus identical observable behaviour can result in quite different consequences. I.E. 5 hours a day playing games can be both healthy behavour or an addiction/a means of suppressing thoughts or memories, depending on the person. So you'll always have that massive confounder called the mind, in every setting. Which means you have to inspect the mind itself, which only works indirectly, i.e. by interacting with the person, asking questions, trying to find out the motivation behind a particular action. And the person often doesn't really know, because a particular decision probably wasn't chosen consciously.
rocqua 5y ago
It doesn't matter much beyond semantics. Math papers aren't experiments so they can't replicate. However, math papers can still be wrong! Especially the actual proofs can be wrong. The question is, how many wrong proofs are out there, and how many of them have false conclusions. If we accept that most published proofs have errors, why isn't that a horrifying failure of Mathematics that shakes fundamental trust in its systems. That is what the article is about. This hypothetical shaking of fundamental trust would be highly analogous to the replication crisis. And there are more parallel arguments. So to me the premise makes sense, and I see little problem with some semantic wiggle room for sake of analogy.
throwawayffffas 5y ago
I think the author makes the distinction in the beginning. > In experimental sciences, the experiment is the “real work” and the paper is just a description of it. But in math, the paper, itself, is the “real work”. In other words, there is no replication crisis in math because there is no replication to be done. There is no experiment to be replicated, just work to be checked for correctness.
inimino 5y ago
> the now unfalsifiable Milgram experiment Total nonsense. Experiments are never falsifiable, what does that even mean? Hypotheses are falsifiable. Did any hypothesis of Milgram's suddenly become unfalsifiable one day in the last 50 years because the psychology research community found a new moral compass? Did the foundations of what is knowable by science shift during the night? "Now unfalsifiable" implies change. So what do you think changed, exactly? You seem really hung up on a strict Popperian falsificationism that you have misunderstood. In any case, the article mentions that you replicate a proof by reading it and checking the steps for yourself, so I feel your criticism about deductive reasoning is misplaced. It's implied that soft science papers require experiment to reproduce because they are inductive, just because the author doesn't spell this out for you doesn't mean they missed the distinction...
- tomerv 5y ago
  > Did the foundations of what is knowable by science shift during the night? Yes! The moment the approval committee decides that certain experiments are no longer allowed, that makes some results no longer reachable by science, or at least not in a reasonable manner.
  - inimino 5y ago
    How far "science" has fallen, if this is how people now define the word.
disgruntledphd2 5y ago
> the idea that the now unfalsifiable Milgram experiment is still taught in every university psychology dept should be outrageous I agree with this point in general, but the Milgram studies have been replicated many, many times so potentially not the best example. The Stanford Prison Study, on the other hand...
veilrap 5y ago
If you're actually interested in this topic, you should read "What is this thing called science?" by Alan Chalmers. One big take away from the book is that there are many things that are "factual" and "scientific" but are not "falsifiable". It's been an important part of modern scientific advancement that falsification is a good, but not perfect way of learning about how the universe works. For example, macro-evolution is basically unfalsifiable, and yet it is generally regarded as good science.

by scoofy 5y ago

netizen-936824 5y ago

I don't believe I'm familiar with an Ingram experiment, any chance you could expand on this? I was also unable to find something with a quick internet search, however Milgram was suggested to me.

scoofy 5y ago
Milgram experiment, sorry... must have got hit by autocorrect there: https://en.wikipedia.org/wiki/Milgram_experiment https://en.wikipedia.org/wiki/Milgram_experiment You can add the Stanford Prison Experiment (Zimbardo) to the list of, now unfalsifiable, experiments that inexplicably are regularly taught in university settings when they cannot possibly provide any useful data to the sciences: https://en.wikipedia.org/wiki/Stanford_prison_experiment https://en.wikipedia.org/wiki/Stanford_prison_experiment The idea that many if not most universities create educational programs based on these very obvious problematic studies, leads me to believe that the scientific community can be as guilty of info-tainment bias as the general public. I only wish that I'd been able to continue in academia so that i could have at least a small voice in changing this problematic dynamic we live with. They literally made a movie about the Zimbardo experiment in 2015. It couldn't be more obvious that people care more that it be true than whether it might be false: https://en.wikipedia.org/wiki/The_Stanford_Prison_Experiment_(film) https://en.wikipedia.org/wiki/The_Stanford_Prison_Experiment...
- zzzeek 5y ago
  what do you mean when you say "now unfalsifiable", is that because the experiments themselves are too unethical to be repeated?
  - netizen-936824 5y ago
    That's correct, however students are being told about the manipulation these days. Especially concerning the prison experiment.
  - mannerheim 5y ago
    The Zimbardo experiment, at least, has actually been falsified, so I don't think it's right to call it unfalsifiable.

stavros 5y ago

Exactly, I can't understand what the author was thinking. If a paper shows that 2+2=4, what would the replication problem be? You write the paper out again and 2+2 turns out to equal 5? "The results can't be replicated" is different from "the logic here is wrong". So different that this article starts from an entirely invalid premise.

bo1024 5y ago
From the post: > More seriously, it’s reasonably well-known among mathematicians that published math papers are full of errors.[1] I think the author's point is that actually the distinction is less clear: many math papers can't be (easily) replicated, yet people in math aren't too worried. [1] https://twitter.com/benskuhn/status/1419281164951556097 https://twitter.com/benskuhn/status/1419281164951556097
- stavros 5y ago
  But there's a difference between "can't get the results a second time" and "didn't even get the results the first time". There isn't even so much a notion of "results", just "logically, Y follows from X". If that's wrong, you don't need to run the experiment again (there is no experiment to run), the logic is just flawed.
  - topaz0 5y ago
    Right -- you need to run the argument again.
  - bo1024 5y ago
    I don’t think it’s quite that simple. A math paper generally says 3 kinds of things: X is true, this is the high level idea of how we can prove X is true, and this is the detailed proof that X is true. As the post mentions, usually when there’s a mistake, X is still true.
    - ConceptJunkie 5y ago

jcranberry 5y ago

I think you're wrong on some of these points, and the author was trying to make these points but was not explicit. I remember reading an article in the intelligencer by a mathematician who essentially said there are certain conjectures where if they were to be proved false rather than be thrown into a sea of uncertainty, mathematicians i would quickly move to investigate a readjustment of basic axioms rather than accept that those conjectures are incorrect. Then there are fields of mathematics around selecting different axioms. Investigating the ramifications of whether you take the undecidable "continuum hypothesis" as true or false. And then theres model theory and such. Presumably they study models of interest and not arbitrary ones. You're mostly correct that the methodology is mostly deductive but the point is that what we choose to use isn't arbitrary because there are things in math which are more important than axioms as they are things believed to be "real".

scoofy 5y ago
I don’t mean "arbitrary" in the sense that it doesn’t matter to the engineering of a building. I mean "arbitrary" in the sense that unprovableness within the framework. Axioms are by definition arbitrary in that sense. edit: I'll try to expand on this. Math is weird because it's a sort-of in-framework study. We use math when we do physics, and we use physics when we do engineering. If the physical world ever started disagreeing with our physics, then we need new physics. If the math ever started showing inconsistent results in our physics, then we would need new math. None of that is to say that Neutonian Physics, say, is "wrong in the sense that it's internally inconsistent", only that "it's wrong in the sense that it is not an accurate mapping of a framework to the world we find ourselves in." The first type of wrongness is the type of wrongness we concern ourselves with in deductive studies, the second type, the mapping-error, is inductive, and requires meta-analysis, and is prone to (Hume's) problem of induction, which means it's ultimately unknowable (i.e. Popper).
- cormacrelf 5y ago
  Here is where you're wrong: > You can't have a replication crisis inside of a deductive framework without changing the framework. You may change the framework, but the replication crisis would be that people don't notice it's changed. A proof can get so huge and complex that it would take a lifetime of study to understand it. One of the article's good points was that "you can replicate a math paper by reading it", but if you cannot read a proof, you cannot replicate it. If nobody can understand it, nobody can replicate it. There would be a big crisis if people started trusting huge papers blindly and using their results without stating them as assumptions. Mathematicians are largely not doing that, so there isn't really a crisis. But they might! And you would not notice that the framework had become inconsistent for a while. Therein would lie the crisis. It is nice to think that as soon as the framework changed, people would notice, because surely if it the ground moved, that would be obvious! But it is not true. Nothing about the deductive-ness or in-framework-ness of mathematics changes this. It is the same for psychology, the replication crisis is a crisis because people do not notice these results are wrong, and then rely on them in clinical guidelines and affect people's lives for generations. It is possible to stave off replication crises by using a machine, because mathematics is self-contained and you can use a computer to show that you proved something using only these specific assumptions and they can believe you without reading and understanding it fully. But this doesn't mean mathematics cannot ever have a replication crisis. I think your arguments here are best summarised as confusing "replication crisis" with "any published results being inconsistent with previous results". Your idea of "inconsistent" is also strange, the article considers this to mean mistakes in proofs, whereas you mean new mathematics that changes the field by discovery. You're just talking about something completely foreign from the concept of a replication crisis.

deleted 5y ago

[deleted]

cubefox 5y ago

This doesn't make sense. This is actually addressed in the piece. If inductive reasoning is irrelevant for mathematics, why do then mathematicians have excellent intuitions about what is true even before they manage to prove it? The quote from the essay: "[D]espite the fact that error-correction is really hard, publishing actually false results was quite rare because “people’s intuition about what’s true is mysteriously really good.” Because we mostly only try to prove true things, our conclusions are right even when our proofs are wrong." Deduction can't explain this surprising reliability of mathematical intuition, only induction can. The question the author tries to answer is why the intuitions of mathematicians seem to be more reliable than those of social scientists.

scoofy 5y ago
I mean, again, this is a very messy thing to talk about explicitly because it's playing fast and loose with language. I think the fact that our brains are computing/pattern-recognition machines is the obvious reason computing (deducing) is 'easier'. However, human minds are not actually mysteriously really good at math. Our brains are very clearly broken at some types of mathematical problems. It's just something acedemics tell themselves. When it comes to gambling, large numbers, power laws, etc. The number of times I've had to explain to my friends and family how exponential growth "will appear" with the spread of covid is testament to the fact that our brains don't actually have that math built in, probably because it wasn't particularly important for evolutionary survival and reproduction. The entire field of behavior economics studies this delta specifically. I'd say the deductiveness of a problem, itself, is what makes people have the intuitions. It's vastly easier to use deduction that it is to do induction... because induction is by ultimately unknowable and effectively solipsistic.
- walleeee 5y ago
  Humans in general may be prone to all sorts of erroneous mathematical reasoning but the speed and consistency with which mathematical concepts reproduce in the minds of mathematicians can justifiably be called mysterious imo, at least until we better understand human language faculties

JetAlone 5y ago

The results of the Milgram and Stanford Prison Experiments (the amount of harm caused by studying obedience and harm to humans) were instrumental in giving us the very ethics codes that make them unfalsifiable. That makes them an important facet of the field of psychology's history and worth teaching for that alone. If you believe that ethics code is appropriate, then you believe those studies are valid and don't need to be run again to re-verified or falsified. If you really don't trust the ethics code which was produced in reaction to the results of these 20th century experiments, then the authentic thing to do would be to re-run the experiments clandestinely and disseminate the results anonymously.

zozbot234 5y ago
> If you believe that ethics code is appropriate, then you believe those studies are valid and don't need to be run again to re-verified or falsified. Why? The experiments would still be unethical, even if they led to the opposite result.
- jstanley 5y ago
  Why? If the opposite result is "some people are prisoners, some are guards, and they all sit around and have a jolly old time" then what's unethical about running that experiment?
scoofy 5y ago
The ethical codes are extremely important, that's not my point. I'm saying we are taught the results are correct, when, due to the impossibility of reproduction we can't know whether the results are correct or incorrect. What should be discussed in the Milgram experiment is "we can't know the results of the experiment because one of the subjects killed themselves due to their participation. Try to consider the ethical implications, don't design experiments that might lead to psychological trauma, etc." The idea that the results being true is why we shouldn't do the experiment is wrong to do, is nonsense, because we can't know whether the results are true because can't replicate the experiment. Yet, it's results are published as a pop-science book: https://en.wikipedia.org/wiki/Obedience_to_Authority:_An_Experimental_View https://en.wikipedia.org/wiki/Obedience_to_Authority:_An_Exp... Instead, these two studies if their findings, not the ethics, are discussed at all, are a perfect illustration of the problems that Karl Popper discusses regarding falsifiability.

topaz0 5y ago

You may not have read very far into the post, because I think it deals quite well with these issues, if not in the same terms.

walleeee 5y ago

Why should the Milgram authority experiment not be taught? If nothing else it's suggestive and might inspire new studies. The fact that the original design wouldn't pass a modern IRB doesn't mean the phenomenon is forever closed to investigation

scoofy 5y ago
Yes, of course, the Milgram experiment can be taught as a tragedy and a lesson in experimental ethics. But it's "correct results" are almost always taught, even being published as a pop-science book, when they cannot be know because the experiment is too ethically problematic to be replicated in a modern setting. Ironically, the "correct results" are often cited as the reason for the tragic suicide of one of the participants.
- walleeee 5y ago
  I'm just having trouble with "almost always" Any psychologist worth their salt knows better than to claim any interpretation of experimental results as unassailable Pop science is another story (literally) but the profs I've had the privilege of studying under spent a lot of time warning us against the shoddy reasoning you describe Psychology has a lot of bullshit in its orbit but the discipline as taught in quality departments is just that, disciplined
  - scoofy 5y ago
    I don't mean to impugn the study of psychology in the slightest. Only that i find it maddening that obviously unfalsifiable results are taught as though they can teach us anything about human behavior. I hope you are right and I am wrong, I was taught these experiments this way at a good school, i can only hope that is more uncommon than i think it is.
    - walleeee 5y ago
      Fair enough- and agreed. I hope so too.

bawolff 5y ago

If you replace math with (experimental) physics, you get an inductive framework, yet physics doesn't have the same problems psychology has.

peteradio 5y ago
Physics has calculable observation biases baked in. At least you can create experiments where you knowingly limit those biases based on some serious fundamentals. All other fields have so much more to worry about, it really isn't fair.
majormajor 5y ago
Does the behavior of a particle depend in part on every other experience the particle has had in the past? Physics has things that we can't figure out, and things that have to be observed in the aggregate, but it's much easier to perform multiple experiments with the same base conditions. If we could clone the participants of a psychology experiment prior to the experiment, with their memories and personalities intact, we could possibly narrow the gap. Or maybe not! But that would be a fascinating finding! ;)
tonyarkles 5y ago
See https://en.wikipedia.org/wiki/Oil_drop_experiment#Fraud_allegations https://en.wikipedia.org/wiki/Oil_drop_experiment#Fraud_alle... and the following section. Physics isn’t immune to it.
- peteradio 5y ago
  The fraud allegations are bogus (according to wikipedia). "Millikan's experiment as an example of psychological effects in scientific methodology" is in itself a psychological study. Has this wikipedia source been peer reviewed and replicated? This is quibbling about fractions of a percent of the secondary measurement, the primary result being quite obvious quantization of electric charge. And! Ultimately the charge measurement result converged on the result known today within 10 years of his original experiment. That's what immunity looks like.
- 5y ago

kalimanzaro 5y ago

By your dichotomy, Physics seems closer to Psychology than math. Why aren't there replication crises in Physics? My answer is that there always has been-- it's just that they get resolved quick. The problem with Psych is that it is SLOW

raverbashing 5y ago
Because it is "trivially" easier to do a better experiment in Physics (money aside, of course) Psychology will always have ethic, sample size and cultural/time/space constraints. Your "psychology experiment" done with Mechanical Turk is more an exercise on sampling bias than anything else And they keep running into "replication failures" because they fail to account for that and try to chase the illusion that it is possible to have a perfect controlled environment in psych. They can't, and they're only fooling themselves they can.
gpt5 5y ago
No - both you and the OP miss the real problem. The problem with Psych is that you can't really experiment on humans in a meaningful way. Animal studies for example don't suffer the same replication problems as psych.
- kalimanzaro 5y ago
  Well perhaps if animal studies were primarily concerned with investigating the interaction of reason and emotion in animals there might just well be :)
- bildung 5y ago
  > The problem with Psych is that you can't really experiment on humans in a meaningful way While absolutely valid this isn't the only problem: One other complication is that conclusions from experimental studies often don't apply to the real world, i.e. people behave differently in experimental settings. Thus in any case you can either have good control over confounding varables but very limited scope (experimental studies) or bad control over the setting but the ability to observe real live behavior (field studies). But it gets worse: Two people with identical behaviour can have quite different motivations and thus identical observable behaviour can result in quite different consequences. I.E. 5 hours a day playing games can be both healthy behavour or an addiction/a means of suppressing thoughts or memories, depending on the person. So you'll always have that massive confounder called the mind, in every setting. Which means you have to inspect the mind itself, which only works indirectly, i.e. by interacting with the person, asking questions, trying to find out the motivation behind a particular action. And the person often doesn't really know, because a particular decision probably wasn't chosen consciously.

rocqua 5y ago

It doesn't matter much beyond semantics. Math papers aren't experiments so they can't replicate. However, math papers can still be wrong! Especially the actual proofs can be wrong. The question is, how many wrong proofs are out there, and how many of them have false conclusions. If we accept that most published proofs have errors, why isn't that a horrifying failure of Mathematics that shakes fundamental trust in its systems. That is what the article is about. This hypothetical shaking of fundamental trust would be highly analogous to the replication crisis. And there are more parallel arguments. So to me the premise makes sense, and I see little problem with some semantic wiggle room for sake of analogy.

throwawayffffas 5y ago

I think the author makes the distinction in the beginning. > In experimental sciences, the experiment is the “real work” and the paper is just a description of it. But in math, the paper, itself, is the “real work”. In other words, there is no replication crisis in math because there is no replication to be done. There is no experiment to be replicated, just work to be checked for correctness.

inimino 5y ago

> the now unfalsifiable Milgram experiment Total nonsense. Experiments are never falsifiable, what does that even mean? Hypotheses are falsifiable. Did any hypothesis of Milgram's suddenly become unfalsifiable one day in the last 50 years because the psychology research community found a new moral compass? Did the foundations of what is knowable by science shift during the night? "Now unfalsifiable" implies change. So what do you think changed, exactly? You seem really hung up on a strict Popperian falsificationism that you have misunderstood. In any case, the article mentions that you replicate a proof by reading it and checking the steps for yourself, so I feel your criticism about deductive reasoning is misplaced. It's implied that soft science papers require experiment to reproduce because they are inductive, just because the author doesn't spell this out for you doesn't mean they missed the distinction...

tomerv 5y ago
> Did the foundations of what is knowable by science shift during the night? Yes! The moment the approval committee decides that certain experiments are no longer allowed, that makes some results no longer reachable by science, or at least not in a reasonable manner.
- inimino 5y ago
  How far "science" has fallen, if this is how people now define the word.

disgruntledphd2 5y ago

> the idea that the now unfalsifiable Milgram experiment is still taught in every university psychology dept should be outrageous I agree with this point in general, but the Milgram studies have been replicated many, many times so potentially not the best example. The Stanford Prison Study, on the other hand...

veilrap 5y ago

If you're actually interested in this topic, you should read "What is this thing called science?" by Alan Chalmers. One big take away from the book is that there are many things that are "factual" and "scientific" but are not "falsifiable". It's been an important part of modern scientific advancement that falsification is a good, but not perfect way of learning about how the universe works. For example, macro-evolution is basically unfalsifiable, and yet it is generally regarded as good science.