3 ms·

Is it possible that small bugs or assumptions in a root paper could cascade through referencing papers leading to wildly inaccurate outcomes 5 or 6 papers down

by GEBBL 3y ago

Is it possible that small bugs or assumptions in a root paper could cascade through referencing papers leading to wildly inaccurate outcomes 5 or 6 papers down the line?

pfdietz 3y ago
There was an entire school of mathematicians in Italy that went off the rails with incorrect results. https://en.wikipedia.org/wiki/Italian_school_of_algebraic_geometry https://en.wikipedia.org/wiki/Italian_school_of_algebraic_ge...
gridentio 3y ago
https://proofwiki.org/wiki/False_Statement_implies_Every_Statement https://proofwiki.org/wiki/False_Statement_implies_Every_Sta...
thfuran 3y ago
I work in medical software and a few years ago fixed a bug that was an incorrect parameter value in a model used for computing diagnostic criteria that had proliferated throughout the literature after seemingly being incorrectly transcribed in one influential paper. The difference was relatively small, but it did make results somewhat worse.
isaacfrond 3y ago
More likely, wildly inaccurate outcomes will cause a re-examination of the cited theorems, which will probably flush out the bug. By the way it is not clear to me, if the theorem was false or if only proof was wrong.
- eigenket 3y ago
  The theorem was mostly correct. As stated it was false, but it was true for n >= 8. If you change some not very interesting constants it becomes true for all n. All you need change is the constants for n < 8.
lmm 3y ago
In theory yes. In practice mathematicians tend to have a good instinct for which things are true (although not always - some false theorems stood for decades if not centuries) and will avoid looking into ideas that don't pass the sniff test. Plus if you keep building on the consequences of a false result then you'll likely eventually reach a contradiction, which might inspire you to spot the bug in the original result.
- pbhjpbhj 3y ago
  If mathematicians have "a good instinct" does that suggest that their brains can somehow apply an informal proof? I wonder if they are good, or if it's selection/confirmation/other biases that lead is to think say. Also, aren't surprising results the most interesting!?
  - eigenket 3y ago
    In an informal sense disproving things is easier than proving things. For example theorems often say thing like "all objects of type Y have property X", its very difficult to work out even where to start proving such a statement unless you're an expert in Y and X, but to disprove it all you have to do is find some example Y which doesn't have property X. If you've worked with Y things for a while you probably have a bunch of "pet" Y objects you can think of and if someone proves something you kinda automatically check to see what their proof says about the ones you're familiar with.
    - Ar-Curunir 3y ago
      > In an informal sense disproving things is easier than proving things. Note that this is not true in general, and depends on the type of theorem. The idea is that while it’s easy to show why a particular proof is incorrect, it’s much more difficult to show that every proof is incorrect. Formally, this idea is captured by CoNP, which is believed to be different from NP and hence a strict superset of P.
      - drdeca 3y ago
eigenket 3y ago
Yes! I work in quantum information theory and this recently happened in a subfield called resource theories. The generalised quantum Stein's lemma [1][2] is (or was) a very powerful result that was used for over 10 years to prove things in this subfield. However last year it was noticed that there was an error in the proof of this lemma, and a whole bunch of results based on it weren't valid [3][4]. One of the authors of the paper where they wrote about the error gave a talk at the conference QIP 2023 this year, and there is a video of that talk available here [5]. Bartosz is a good speaker and I recommend watching the talk if you're interested, if you go to about 10 minutes, 30 seconds in the talk he discusses the consequences of this result now being not known to be true. [1] Published paper https://link.springer.com/article/10.1007/s00220-010-1005-z https://link.springer.com/article/10.1007/s00220-010-1005-z [2] Arvix version: https://arxiv.org/abs/0904.0281 https://arxiv.org/abs/0904.0281 [3] Published version: https://quantum-journal.org/papers/q-2023-09-07-1103/ https://quantum-journal.org/papers/q-2023-09-07-1103/ [4] Arxiv version: https://arxiv.org/abs/2205.02813 https://arxiv.org/abs/2205.02813 [5] Youtube link: https://www.youtube.com/watch?app=desktop&v=2Xyodvh6DSY https://www.youtube.com/watch?app=desktop&v=2Xyodvh6DSY
zmgsabst 3y ago
This is arguably what caused the push for formalism in mathematics: Multiple papers on calculus claimed results about continuity and derivatives, but we’re using subtly different definitions. The conflict between those results, and the counter-examples to demonstrate the difference, led to mathematicians building the modern machinery around proofs. > The Weierstrass function has historically served the role of a pathological function, being the first published example (1872) specifically concocted to challenge the notion that every continuous function is differentiable except on a set of isolated points. Weierstrass's demonstration that continuity did not imply almost-everywhere differentiability upended mathematics, overturning several proofs that relied on geometric intuition and vague definitions of smoothness. https://en.wikipedia.org/wiki/Weierstrass_function https://en.wikipedia.org/wiki/Weierstrass_function

by GEBBL 3y ago

Is it possible that small bugs or assumptions in a root paper could cascade through referencing papers leading to wildly inaccurate outcomes 5 or 6 papers down the line?

pfdietz 3y ago

There was an entire school of mathematicians in Italy that went off the rails with incorrect results. https://en.wikipedia.org/wiki/Italian_school_of_algebraic_geometry https://en.wikipedia.org/wiki/Italian_school_of_algebraic_ge...

gridentio 3y ago

https://proofwiki.org/wiki/False_Statement_implies_Every_Statement https://proofwiki.org/wiki/False_Statement_implies_Every_Sta...

thfuran 3y ago

I work in medical software and a few years ago fixed a bug that was an incorrect parameter value in a model used for computing diagnostic criteria that had proliferated throughout the literature after seemingly being incorrectly transcribed in one influential paper. The difference was relatively small, but it did make results somewhat worse.

isaacfrond 3y ago

More likely, wildly inaccurate outcomes will cause a re-examination of the cited theorems, which will probably flush out the bug. By the way it is not clear to me, if the theorem was false or if only proof was wrong.

eigenket 3y ago
The theorem was mostly correct. As stated it was false, but it was true for n >= 8. If you change some not very interesting constants it becomes true for all n. All you need change is the constants for n < 8.

lmm 3y ago

In theory yes. In practice mathematicians tend to have a good instinct for which things are true (although not always - some false theorems stood for decades if not centuries) and will avoid looking into ideas that don't pass the sniff test. Plus if you keep building on the consequences of a false result then you'll likely eventually reach a contradiction, which might inspire you to spot the bug in the original result.

pbhjpbhj 3y ago
If mathematicians have "a good instinct" does that suggest that their brains can somehow apply an informal proof? I wonder if they are good, or if it's selection/confirmation/other biases that lead is to think say. Also, aren't surprising results the most interesting!?
- eigenket 3y ago
  In an informal sense disproving things is easier than proving things. For example theorems often say thing like "all objects of type Y have property X", its very difficult to work out even where to start proving such a statement unless you're an expert in Y and X, but to disprove it all you have to do is find some example Y which doesn't have property X. If you've worked with Y things for a while you probably have a bunch of "pet" Y objects you can think of and if someone proves something you kinda automatically check to see what their proof says about the ones you're familiar with.
  - Ar-Curunir 3y ago
    > In an informal sense disproving things is easier than proving things. Note that this is not true in general, and depends on the type of theorem. The idea is that while it’s easy to show why a particular proof is incorrect, it’s much more difficult to show that every proof is incorrect. Formally, this idea is captured by CoNP, which is believed to be different from NP and hence a strict superset of P.
    - drdeca 3y ago

eigenket 3y ago

Yes! I work in quantum information theory and this recently happened in a subfield called resource theories. The generalised quantum Stein's lemma [1][2] is (or was) a very powerful result that was used for over 10 years to prove things in this subfield. However last year it was noticed that there was an error in the proof of this lemma, and a whole bunch of results based on it weren't valid [3][4]. One of the authors of the paper where they wrote about the error gave a talk at the conference QIP 2023 this year, and there is a video of that talk available here [5]. Bartosz is a good speaker and I recommend watching the talk if you're interested, if you go to about 10 minutes, 30 seconds in the talk he discusses the consequences of this result now being not known to be true. [1] Published paper https://link.springer.com/article/10.1007/s00220-010-1005-z https://link.springer.com/article/10.1007/s00220-010-1005-z [2] Arvix version: https://arxiv.org/abs/0904.0281 https://arxiv.org/abs/0904.0281 [3] Published version: https://quantum-journal.org/papers/q-2023-09-07-1103/ https://quantum-journal.org/papers/q-2023-09-07-1103/ [4] Arxiv version: https://arxiv.org/abs/2205.02813 https://arxiv.org/abs/2205.02813 [5] Youtube link: https://www.youtube.com/watch?app=desktop&v=2Xyodvh6DSY https://www.youtube.com/watch?app=desktop&v=2Xyodvh6DSY

zmgsabst 3y ago

This is arguably what caused the push for formalism in mathematics: Multiple papers on calculus claimed results about continuity and derivatives, but we’re using subtly different definitions. The conflict between those results, and the counter-examples to demonstrate the difference, led to mathematicians building the modern machinery around proofs. > The Weierstrass function has historically served the role of a pathological function, being the first published example (1872) specifically concocted to challenge the notion that every continuous function is differentiable except on a set of isolated points. Weierstrass's demonstration that continuity did not imply almost-everywhere differentiability upended mathematics, overturning several proofs that relied on geometric intuition and vague definitions of smoothness. https://en.wikipedia.org/wiki/Weierstrass_function https://en.wikipedia.org/wiki/Weierstrass_function