8 ms·

The praise from popular press and the promotion by the authors should be put into context of what mathematicians think of it. Two blog posts by a professor at

The praise from popular press and the promotion by the authors should be put into context of what mathematicians think of it. Two blog posts by a professor at U. Chicago, qualifying it of intellectual fraud: https://www.galoisrepresentations.com/2019/07/17/the-ramanujan-machine-is-an-intellectual-fraud https://www.galoisrepresentations.com/2019/07/17/the-ramanuj... https://www.galoisrepresentations.com/2019/07/07/en-passant-v/ https://www.galoisrepresentations.com/2019/07/07/en-passant-...

Daniel51 6y ago
Irrelevant.
- Daniel51 6y ago
  The Nature paper presents several new conjectures related to the Catalan constant, pi^2, and zeta(3) (Apéry's constant): http://www.ramanujanmachine.com/wp-content/uploads/2020/06/catalan.pdf http://www.ramanujanmachine.com/wp-content/uploads/2020/06/c... http://www.ramanujanmachine.com/wp-content/uploads/2020/06/pi_square.pdf http://www.ramanujanmachine.com/wp-content/uploads/2020/06/p... http://www.ramanujanmachine.com/wp-content/uploads/2020/06/zeta3.pdf http://www.ramanujanmachine.com/wp-content/uploads/2020/06/z... The main criticism of the blog is "that the program has not yet generated anything new", but the post does not refer to these results. So it seems that this blog post is currently irrelevant and out-dated compared to the Nature publication.
QuesnayJr 6y ago
This is shocking stuff. I encourage everyone to read these two links.
dealforager 6y ago
Man, this is an example of how difficult it is to know what is BS or not if you're not an expert on the subject. On one hand, this article was published in Nature, which I thought was trustworthy. On the other, there's this comment on a social media platform that links to a blog that also seems legit. No wonder misinformation spreads so fast. Even after reading both, I don't know what to make of it. The reaction and comments here just confuse me more.
- TeMPOraL 6y ago
  This phenomenon has a name: epistemic learned helplessness. https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/ https://slatestarcodex.com/2019/06/03/repost-epistemic-learn...
- QuesnayJr 6y ago
  Nature in particular seems vulnerable to the academic equivalent of click-bait articles. I think the top journals within a specific field are more reliable.
- PartiallyTyped 6y ago
  Nature has published some very questionable papers in AI/ML that are filled with malpractices. Another bogus paper that comes to mind was predicting earthquakes with a deep(read huge) neural network that appears to have information leakage and was fuelled with the hype of DL when a simple logistic regression (i.e. single neuron) could perform just as well [1,2,3]. [1] https://www.reddit.com/r/MachineLearning/comments/c4ylga/d_misuse_of_deep_learning_in_nature_journals/ https://www.reddit.com/r/MachineLearning/comments/c4ylga/d_m... [2] https://www.reddit.com/r/MachineLearning/comments/c8zf14/d_was_this_quake_ai_a_little_too_artificial/ https://www.reddit.com/r/MachineLearning/comments/c8zf14/d_w... [3] https://www.nature.com/articles/s41586-019-1582-8 https://www.nature.com/articles/s41586-019-1582-8 / https://arxiv.org/pdf/1904.01983.pdf https://arxiv.org/pdf/1904.01983.pdf
  - lumost 6y ago
    This is a frighteningly common practice in DL research. Baselines are rarely taken with resect to alternate techniques, largely due to publication bias. On one hand papers about DL applications are of interest to the DL community, and useful to see if there is promise in the technique. On the other hand, they may not be particularly useful to industry, or to forwarding broader research goals.
    - throwawaygh 6y ago
      A good rule of thumb is to be slightly more suspicious of "DL for X" unless X was part of the AI/ML umbrella in the 2000s. If no one was publishing about X in AAAI/NIPS/ICML before 2013 or so then there's a pretty good chance that "DL for X" is ignoring 30+ years of work on X. This is becoming less true if one of the paper's senior author comes from the field where "X" is traditionally studied. Another good rule of thumb is that physicists writing DL papers about "DL for X" where X is not physics are especially terrible about arrogantly ignoring 30+ years of deeply related research. I don't quite understand why, but there's an epidemic of physicists dabbling in CS/AI and hyping it way the hell up.
mxcrossb 6y ago
> Well … OK I guess? But, pretty much exactly as pointed out last time, not only is the proof only one line, but the nature of the proof makes clear exactly how unoriginal this is to mathematics This is what I was wondering about while reading the article. If the AI only generates formula for which proofs involve only a few trivial steps back to something that is known, then it doesn’t feel useful. But I feel like the question “what makes a good conjecture?” in its own right makes for a very interesting discussion.
- bawolff 6y ago
  Wouldn't a good conjecture be anything that's interesting if true. General bonus points for if intuitively it seems like it should be obviously true (or false) but yet is hard to prove or if proving it is true would allow you to prove lots of other interesting statements.
  - alisonkisk 6y ago
    Sure. Define "interesting". What's interesting to me is probably in a standard textbook already.
    - bawolff 6y ago
      If it is in a standard textbook, than its almost certainly interesting (although probably not a conjecture unless its a pretty advanced textbook)
- prof-dr-ir 6y ago
  Mathematical physicist Robbert Dijkgraaf has got you covered: https://www.quantamagazine.org/the-subtle-art-of-the-mathematical-conjecture-20190507/ https://www.quantamagazine.org/the-subtle-art-of-the-mathema...
0b01 6y ago
I agree with the article you linked. Mathematical knowledge is about compression. Most if not all of these formulae are just specializations of known formulae. So the value of this approach is questionable. Generating these forms can possibly be done in a much simpler way.
- agumonkey 6y ago
  who else holds the compression view ?
boriselec 6y ago
I understand his frustration. But calling it a fraud is a little bit too much.
deleted 6y ago
[deleted]
Radim 6y ago
Ah yes, the good old "SV culture disrupts X! Revolution at 8 o'clock!" There's an arms race: * People are evolving memetic resistance to the incessant BS, ads and bombastic headlines. * The SV/startup culture is evolving to inject authenticity to overcome people's BS defenses, convince them they need a change. Honestly, do you still get excited when you read "AI solves X!"? Probably another huckster peddling empty air, cutting corners, externalizing costs. The whole game is tired, and people are taking note. Not everything that exists requires a radical change.
j7ake 6y ago
I think one interesting lesson from this nice qualification here is that at the moment ML methods to learn mathematics may look trivial from a professional mathematician (ie the results are unoriginal or trivial) but perhaps the target audience of this method may be for non professional mathematicians or students training to be mathematicians. I could still see this ML tool as a way to automate the work of some more “trivial” (from the POV of an expert) mathematics, although not the work of professional mathematicians. The knowledge gap in mathematics between professional mathematicians and non professionals is vast, and this tool could narrow the gap. I would bet the majority of readers of nature would not be able to point out that the outputs of the ML tool were trivial. So there is need to narrow this gap.
- deleted 6y ago
  [deleted]
- alisonkisk 6y ago
  Simple results in almost any specialist field would stump most readers of Nature. That's not a reason to publish in Nature.
  - deleted 6y ago
    [deleted]
catgary 6y ago
Look at the comments: > The paper is amazingly bad. None of the authors are mathematicians as far as I can see. I think the word “new” appears 50+ times in the paper. Looks like they updated the paper to include your observation from last time about the Gauss continued fraction without mentioning the source (the authors admit here they read your blog: http://www.ramanujanmachine.com/idea/some-well-known-results/ http://www.ramanujanmachine.com/idea/some-well-known-results...). Classy! Just some light plagiarism/academic misconduct!
David147 6y ago
Looking at his post, the main criticism is "that the program has not yet generated anything new", but the post does not refer to the actual results (like formulas for Catalan's and Apery's constants).
- deleted 6y ago
  [deleted]
yharris 6y ago
Full disclosure- I am one of the authors of the paper. Note that the blog you're citing was written a year and a half ago. It refers to a select few conjectures, and naturally has no references to the developments in the past year and half (which were the main reason the paper got published). Furthermore, the author of the blog didn't respond to multiple emails we sent him, attempting to discuss the actual mathematics. So basically the vast majority of the criticism here, is based on a single, outdated blog, by a professor (respected as he may be) who has not revisited the issues and new results since first posting the blog, and has not given any mathematical argument as to why the results shown in the paper (the actual updated paper that was published) are supposably unimportant. Would appreciate your opinions on the matter.
- _8091149529 6y ago
  Not the person you're replying to, but I admit to characterizing your paper as "garbage" in another comment thread. Since you're inviting discourse, which I greatly appreciate, I'm compelled to reply. 1) To anyone who's studied algebra, it is clear that identities of the form LHS = RHS can be obtained by a nested application of transformations and substitutions in a consistent manner. 2) Of course, arriving at a new, insightful result often involves taking mundane steps. However, in this case, the new mathematical discoveries based on the output tableaus of your algorithm are hypothetical. Whereas the manuscript (and the authors) have already pocketed one of the premium accolades in sciences in the form of a Nature publication. 3) To drive the point above home, do you think the resulting mathematical insights themselves, without riding on the "AI" novelty aspect, would clear the bar for a Nature (or similar high-impact) publication? To be clear, I'm not a mathematican, but I believe the answer would be no. Contrast this with another AI/ML advance published in Nature quite recently: AlphaGo. Note how the gist of their paper, superhuman performance in Go, is a self-standing achievement that merely makes use of machine learning techniques.
  - throwawaygh 6y ago
    "garbage" and "fraud" are really strong words. I would give the actual work behind this paper a "strong accept" if the claims were properly scoped, perhaps with a weak/borderline score on "significance/impact" since I'm not really sure why anyone cares about discovering discovering these sorts of identities. Probably a Conditional Accept in its current form because of the mismatch between actual results + reasonable expectation of potential vs. what's claimed. So, "over-hyped" and "claims wildly out of line with actual results" are definitely more than fair statements. "Fraud" or "garbage" are way too strong. Re: Nature, I don't really understand it or care. I can say that in my own input to hiring committees I tend to treat Nature papers in CS/Math as red flags unless they're consolidations of a bunch of other work published in top sub-field journals/conferences. For some reason Nature really loves these "automated discovery of random mathematical facts" type of papers. I don't understand it. I tend to assume it's click-through-rate-driven editorial decision making.

Daniel51 6y ago

Irrelevant.

Daniel51 6y ago
The Nature paper presents several new conjectures related to the Catalan constant, pi^2, and zeta(3) (Apéry's constant): http://www.ramanujanmachine.com/wp-content/uploads/2020/06/catalan.pdf http://www.ramanujanmachine.com/wp-content/uploads/2020/06/c... http://www.ramanujanmachine.com/wp-content/uploads/2020/06/pi_square.pdf http://www.ramanujanmachine.com/wp-content/uploads/2020/06/p... http://www.ramanujanmachine.com/wp-content/uploads/2020/06/zeta3.pdf http://www.ramanujanmachine.com/wp-content/uploads/2020/06/z... The main criticism of the blog is "that the program has not yet generated anything new", but the post does not refer to these results. So it seems that this blog post is currently irrelevant and out-dated compared to the Nature publication.

QuesnayJr 6y ago

This is shocking stuff. I encourage everyone to read these two links.

dealforager 6y ago

Man, this is an example of how difficult it is to know what is BS or not if you're not an expert on the subject. On one hand, this article was published in Nature, which I thought was trustworthy. On the other, there's this comment on a social media platform that links to a blog that also seems legit. No wonder misinformation spreads so fast. Even after reading both, I don't know what to make of it. The reaction and comments here just confuse me more.

TeMPOraL 6y ago
This phenomenon has a name: epistemic learned helplessness. https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/ https://slatestarcodex.com/2019/06/03/repost-epistemic-learn...
QuesnayJr 6y ago
Nature in particular seems vulnerable to the academic equivalent of click-bait articles. I think the top journals within a specific field are more reliable.
PartiallyTyped 6y ago
Nature has published some very questionable papers in AI/ML that are filled with malpractices. Another bogus paper that comes to mind was predicting earthquakes with a deep(read huge) neural network that appears to have information leakage and was fuelled with the hype of DL when a simple logistic regression (i.e. single neuron) could perform just as well [1,2,3]. [1] https://www.reddit.com/r/MachineLearning/comments/c4ylga/d_misuse_of_deep_learning_in_nature_journals/ https://www.reddit.com/r/MachineLearning/comments/c4ylga/d_m... [2] https://www.reddit.com/r/MachineLearning/comments/c8zf14/d_was_this_quake_ai_a_little_too_artificial/ https://www.reddit.com/r/MachineLearning/comments/c8zf14/d_w... [3] https://www.nature.com/articles/s41586-019-1582-8 https://www.nature.com/articles/s41586-019-1582-8 / https://arxiv.org/pdf/1904.01983.pdf https://arxiv.org/pdf/1904.01983.pdf
- lumost 6y ago
  This is a frighteningly common practice in DL research. Baselines are rarely taken with resect to alternate techniques, largely due to publication bias. On one hand papers about DL applications are of interest to the DL community, and useful to see if there is promise in the technique. On the other hand, they may not be particularly useful to industry, or to forwarding broader research goals.
  - throwawaygh 6y ago
    A good rule of thumb is to be slightly more suspicious of "DL for X" unless X was part of the AI/ML umbrella in the 2000s. If no one was publishing about X in AAAI/NIPS/ICML before 2013 or so then there's a pretty good chance that "DL for X" is ignoring 30+ years of work on X. This is becoming less true if one of the paper's senior author comes from the field where "X" is traditionally studied. Another good rule of thumb is that physicists writing DL papers about "DL for X" where X is not physics are especially terrible about arrogantly ignoring 30+ years of deeply related research. I don't quite understand why, but there's an epidemic of physicists dabbling in CS/AI and hyping it way the hell up.

mxcrossb 6y ago

> Well … OK I guess? But, pretty much exactly as pointed out last time, not only is the proof only one line, but the nature of the proof makes clear exactly how unoriginal this is to mathematics This is what I was wondering about while reading the article. If the AI only generates formula for which proofs involve only a few trivial steps back to something that is known, then it doesn’t feel useful. But I feel like the question “what makes a good conjecture?” in its own right makes for a very interesting discussion.

bawolff 6y ago
Wouldn't a good conjecture be anything that's interesting if true. General bonus points for if intuitively it seems like it should be obviously true (or false) but yet is hard to prove or if proving it is true would allow you to prove lots of other interesting statements.
- alisonkisk 6y ago
  Sure. Define "interesting". What's interesting to me is probably in a standard textbook already.
  - bawolff 6y ago
    If it is in a standard textbook, than its almost certainly interesting (although probably not a conjecture unless its a pretty advanced textbook)
prof-dr-ir 6y ago
Mathematical physicist Robbert Dijkgraaf has got you covered: https://www.quantamagazine.org/the-subtle-art-of-the-mathematical-conjecture-20190507/ https://www.quantamagazine.org/the-subtle-art-of-the-mathema...

0b01 6y ago

I agree with the article you linked. Mathematical knowledge is about compression. Most if not all of these formulae are just specializations of known formulae. So the value of this approach is questionable. Generating these forms can possibly be done in a much simpler way.

agumonkey 6y ago
who else holds the compression view ?

boriselec 6y ago

I understand his frustration. But calling it a fraud is a little bit too much.

deleted 6y ago

[deleted]

Radim 6y ago

Ah yes, the good old "SV culture disrupts X! Revolution at 8 o'clock!" There's an arms race: * People are evolving memetic resistance to the incessant BS, ads and bombastic headlines. * The SV/startup culture is evolving to inject authenticity to overcome people's BS defenses, convince them they need a change. Honestly, do you still get excited when you read "AI solves X!"? Probably another huckster peddling empty air, cutting corners, externalizing costs. The whole game is tired, and people are taking note. Not everything that exists requires a radical change.

j7ake 6y ago

I think one interesting lesson from this nice qualification here is that at the moment ML methods to learn mathematics may look trivial from a professional mathematician (ie the results are unoriginal or trivial) but perhaps the target audience of this method may be for non professional mathematicians or students training to be mathematicians. I could still see this ML tool as a way to automate the work of some more “trivial” (from the POV of an expert) mathematics, although not the work of professional mathematicians. The knowledge gap in mathematics between professional mathematicians and non professionals is vast, and this tool could narrow the gap. I would bet the majority of readers of nature would not be able to point out that the outputs of the ML tool were trivial. So there is need to narrow this gap.

deleted 6y ago
[deleted]
alisonkisk 6y ago
Simple results in almost any specialist field would stump most readers of Nature. That's not a reason to publish in Nature.
- deleted 6y ago
  [deleted]

catgary 6y ago

Look at the comments: > The paper is amazingly bad. None of the authors are mathematicians as far as I can see. I think the word “new” appears 50+ times in the paper. Looks like they updated the paper to include your observation from last time about the Gauss continued fraction without mentioning the source (the authors admit here they read your blog: http://www.ramanujanmachine.com/idea/some-well-known-results/ http://www.ramanujanmachine.com/idea/some-well-known-results...). Classy! Just some light plagiarism/academic misconduct!

David147 6y ago

Looking at his post, the main criticism is "that the program has not yet generated anything new", but the post does not refer to the actual results (like formulas for Catalan's and Apery's constants).

deleted 6y ago
[deleted]

yharris 6y ago

Full disclosure- I am one of the authors of the paper. Note that the blog you're citing was written a year and a half ago. It refers to a select few conjectures, and naturally has no references to the developments in the past year and half (which were the main reason the paper got published). Furthermore, the author of the blog didn't respond to multiple emails we sent him, attempting to discuss the actual mathematics. So basically the vast majority of the criticism here, is based on a single, outdated blog, by a professor (respected as he may be) who has not revisited the issues and new results since first posting the blog, and has not given any mathematical argument as to why the results shown in the paper (the actual updated paper that was published) are supposably unimportant. Would appreciate your opinions on the matter.

_8091149529 6y ago
Not the person you're replying to, but I admit to characterizing your paper as "garbage" in another comment thread. Since you're inviting discourse, which I greatly appreciate, I'm compelled to reply. 1) To anyone who's studied algebra, it is clear that identities of the form LHS = RHS can be obtained by a nested application of transformations and substitutions in a consistent manner. 2) Of course, arriving at a new, insightful result often involves taking mundane steps. However, in this case, the new mathematical discoveries based on the output tableaus of your algorithm are hypothetical. Whereas the manuscript (and the authors) have already pocketed one of the premium accolades in sciences in the form of a Nature publication. 3) To drive the point above home, do you think the resulting mathematical insights themselves, without riding on the "AI" novelty aspect, would clear the bar for a Nature (or similar high-impact) publication? To be clear, I'm not a mathematican, but I believe the answer would be no. Contrast this with another AI/ML advance published in Nature quite recently: AlphaGo. Note how the gist of their paper, superhuman performance in Go, is a self-standing achievement that merely makes use of machine learning techniques.
- throwawaygh 6y ago
  "garbage" and "fraud" are really strong words. I would give the actual work behind this paper a "strong accept" if the claims were properly scoped, perhaps with a weak/borderline score on "significance/impact" since I'm not really sure why anyone cares about discovering discovering these sorts of identities. Probably a Conditional Accept in its current form because of the mismatch between actual results + reasonable expectation of potential vs. what's claimed. So, "over-hyped" and "claims wildly out of line with actual results" are definitely more than fair statements. "Fraud" or "garbage" are way too strong. Re: Nature, I don't really understand it or care. I can say that in my own input to hiring committees I tend to treat Nature papers in CS/Math as red flags unless they're consolidations of a bunch of other work published in top sub-field journals/conferences. For some reason Nature really loves these "automated discovery of random mathematical facts" type of papers. I don't understand it. I tend to assume it's click-through-rate-driven editorial decision making.