5 ms·
The results look almost too good to be true. There could be a bias in the way the expressions are generated; there is no guarantee that this is in any way repre
by andrepd 7y ago
The results look almost too good to be true. There could be a bias in the way the expressions are generated; there is no guarantee that this is in any way representative of real problems, and I don't think this is addressed in the paper. A good thing to do, in my opinion, is compile a corpus of real-world, occurring problems, and then measure performance on that. Then you would know how it performs on some objectively useful metric, not on a potentially biased sample.
- knzhou 7y agoMathematica is at a disadvantage since it worries about getting everything perfectly right, i.e. it cares about branch cuts and domains. I can cook up plenty of examples engineered to have these issues, which I could do (naively) but which Mathematica would choke on (when trying to solve it properly). All of the examples that Mathematica couldn't solve seem to be of this form. If you actually want it to stand a chance, you have to put in appropriate Assumptions, not just toss it in. That said, I'm still extremely surprised this works that well. I would be very curious to test this interactively.
- amelius 7y agoYou could take a 2-step approach, where you first take the "fuzzy" approach, and then later try to correct for possible oversights (which is just simple bookkeeping).
- mlevental 7y agohow exactly would you do that? how would you know when the fuzzy search has failed?
- amelius 7y agoYou plug the result back into the equation, then simplify the equation until you get a tautology. Of course, the "simplify" step can be difficult, but it's usually a lot simpler than solving equations.