4 ms·

The trouble with SymPy is it's, well, buggy. I tried it a few years ago, and as soon as I got serious, I quite quickly ran into problems that I reported, some o

The trouble with SymPy is it's, well, buggy. I tried it a few years ago, and as soon as I got serious, I quite quickly ran into problems that I reported, some of which I now see they apparently still haven't gotten around to addressing. [1] [2] Symbolic math is hard; they have my sympathies. I don't think I could do better. But as long as bugs like these exist, it's going to be hard to convince people to switch away from better tools like Mathematica. [1] https://github.com/sympy/sympy/issues/12561 https://github.com/sympy/sympy/issues/12561 [2] https://github.com/sympy/sympy/issues/12562 https://github.com/sympy/sympy/issues/12562

BeetleB 6y ago
Yikes those are bad. As a non-user interested in it, I must confess that seeing those issues (especially the 2nd one) makes me not want to use it! I do get that it is a hard problem, but I would then recommend they not have an option for "real" if they can't get it right. Users will expect it to work (unless the docs point out it is unreliable).
- mehrdadn 6y ago
  Yeah. For what it's worth (not sure if it makes you feel better or worse...) I've found similarly bad bugs in SciPy too.
  - BeetleB 6y ago
    Curious what those were.
    - mehrdadn 6y ago
      Here's the one I can find (might not have reported/written others so I don't recall them): https://github.com/scipy/scipy/issues/7332 https://github.com/scipy/scipy/issues/7332 I can't tell if I'm just exceptionally good at making bugs explode in my face or something, but these are so simple that it boggles my mind when I'm the first person to report them...
      - BeetleB 6y ago
        That one seems a bit more reasonable, though. When doing numerical computation (as opposed to symbolic), you should never assume the algorithm "just works". You need to know the quirks of the underlying method, etc. And I believe most root finding numerical algorithms will be sensitive to the initial guess. Surprised they haven't added the warning and closed the bug.
        mehrdadn 6y ago
        Not at all. I've worked on numerical stuff myself; unlike the symbolic bugs, I have little sympathy for these particular numerical errors. It would be far more reasonable if it was convoluted composition of functions, or some kind of bizarre numerical edge case. However: (a) Univariate quadratics are extremely well-understood and extremely important. For example, the convergences of optimization algorithms (this is root-finding, and there are some differences, but they're not unrelated, and this is beside my point) are often proven on quadratics first, and then they're applied to other functions. (b) (x - 1)^2 - 1 is just about the simplest quadratic you can possibly imagine that isn't outright lacking in other terms (like x^2), and quadratics are pretty much the simplest nonlinear functions. (c) No matter how good or bad your algorithm is, it is absolutely trivial to do a few quick tests afterward to sanity-check the result, and to return some kind of error if it looks wrong. This isn't some kind of quibbling over a solver that has an error of 0.0001 or something. We're feeding in the most basic quadratic you can imagine, and the solver is telling you with confidence that the solution is 1.01 when in fact it's supposed to be 2. It's just inexcusable. At the very least (and even this would honestly still be woefully lacking), it should be able to do the dumbest thing possible, which is to plug that back in, and maybe plug in a couple close numbers (maybe +/- some epsilon) and see if results change sign or land anywhere near zero. In reality there are all kinds of tests they could and should be doing to check things like concavity and switch to better algorithms, but that's kind of moot when they're not doing the most basic check. Edit: Actually I could probably say something similar about one of the SymPy bugs too (they really shouldn't have trouble telling if 0.66 + 0.56I is a real number), but one key difference is at least their problem is in filtering out correct solutions, not in returning completely wrong solutions with full confidence.
- deleted 6y ago
  [deleted]
- ogogmad 6y ago
  > I would then recommend they not have an option for "real" if they can't get it right. Users will expect it to work (unless the docs point out it is unreliable). You should be suspicious of using floats and expecting the solver to detect if a solution is real. Consider the polynomial $x^2 + C$ for $C \approx 0$. Are all of its roots real or complex? The problem is that even a very small change in C, maybe caused by rounding errors, can easily change a root from real to not real. When solving polynomials, either use complex numbers throughout or insist on exact solutions and don't use floats.
  - mehrdadn 6y ago
    > You should be suspicious of using floats and expecting the solver to detect if a solution is real. Consider the polynomial $x^2 + C$ for $C \approx 0$. Are all of its roots real or complex? The problem is that even a very small change in C, maybe caused by rounding errors, can easily change a root from real to not real. C \approx 0? The imaginary parts in the first bug are nowhere close to zero.
    - ogogmad 6y ago
      I was actually responding to the following claim: > I would then recommend they not have an option for "real" if they can't get it right. Users will expect it to work (unless the docs point out it is unreliable). It can't work reliably if the coefficients of the polynomial consist of floats. At least not in general. This point is a more fundamental one than your example. Also, regarding your particular example: When solving cubics using the cubic formula, it is often the case that you will end up computing with complex numbers even if the result ends up real. If you introduce floats into this, it's impossible to guarantee that the imaginary parts will cancel out completely. Instead, they may cancel up to 10^-21 or something like that, which is precisely what happens in your example.
      - mehrdadn 6y ago
        You're missing my point entirely. They don't need to solve this by keeping the variable real at every step or anything like that. What they need to do is to ensure the final solutions are real. Which means they need to solve it in the complex domain entirely and then filter out non-real solutions at the end, where they have the final magnitudes of the real and imaginary components. The only place that would run into trouble is where the imaginary components are small but nonzero, where you'd just expect it to use some tolerance threshold as the cutoff, but that obviously isn't the case in the first example (hence my comment). There's no reason that the algorithm must fail on univariate real polynomials if it can't work equally well on arbitrary functions with arbitrary domains. It's trivial for a symbolic engine to recognize a univariate polynomial. And practically speaking it's the job of a computer algebra system to recognize different well-known classes of functions and treat them as well as it can. What I suspect likely happened here is they've been just so busy dealing with the more general problems (which are far more difficult) that they haven't gotten around to implementing many better methods for the special cases (like polynomials). Which I sympathize with, as I'd have virtually no clue how to solve some of the more general problems to begin with.
- leephillips 6y ago
  I don’t think there is a general-purpose symbolic math program that doesn’t have some bugs. Mathematica was getting some definite integrals wrong for some years. Anyone who uses these knows to check the results.
bee_rider 6y ago
Have you tried out SageMath? It seemed to work pretty well when I tried it out, but I never got serious really. My needs for symbolic math programming are pretty limited.
- mehrdadn 6y ago
  I haven't, though I did download it. I was trying to work on an algorithm rather than solve specific problem instances, so I ended up just working around it by using different problem instances that didn't result in these errors.

BeetleB 6y ago

Yikes those are bad. As a non-user interested in it, I must confess that seeing those issues (especially the 2nd one) makes me not want to use it! I do get that it is a hard problem, but I would then recommend they not have an option for "real" if they can't get it right. Users will expect it to work (unless the docs point out it is unreliable).

mehrdadn 6y ago
Yeah. For what it's worth (not sure if it makes you feel better or worse...) I've found similarly bad bugs in SciPy too.
- BeetleB 6y ago
  Curious what those were.
  - mehrdadn 6y ago
    Here's the one I can find (might not have reported/written others so I don't recall them): https://github.com/scipy/scipy/issues/7332 https://github.com/scipy/scipy/issues/7332 I can't tell if I'm just exceptionally good at making bugs explode in my face or something, but these are so simple that it boggles my mind when I'm the first person to report them...
    - BeetleB 6y ago
      That one seems a bit more reasonable, though. When doing numerical computation (as opposed to symbolic), you should never assume the algorithm "just works". You need to know the quirks of the underlying method, etc. And I believe most root finding numerical algorithms will be sensitive to the initial guess. Surprised they haven't added the warning and closed the bug.
      - mehrdadn 6y ago
        Not at all. I've worked on numerical stuff myself; unlike the symbolic bugs, I have little sympathy for these particular numerical errors. It would be far more reasonable if it was convoluted composition of functions, or some kind of bizarre numerical edge case. However: (a) Univariate quadratics are extremely well-understood and extremely important. For example, the convergences of optimization algorithms (this is root-finding, and there are some differences, but they're not unrelated, and this is beside my point) are often proven on quadratics first, and then they're applied to other functions. (b) (x - 1)^2 - 1 is just about the simplest quadratic you can possibly imagine that isn't outright lacking in other terms (like x^2), and quadratics are pretty much the simplest nonlinear functions. (c) No matter how good or bad your algorithm is, it is absolutely trivial to do a few quick tests afterward to sanity-check the result, and to return some kind of error if it looks wrong. This isn't some kind of quibbling over a solver that has an error of 0.0001 or something. We're feeding in the most basic quadratic you can imagine, and the solver is telling you with confidence that the solution is 1.01 when in fact it's supposed to be 2. It's just inexcusable. At the very least (and even this would honestly still be woefully lacking), it should be able to do the dumbest thing possible, which is to plug that back in, and maybe plug in a couple close numbers (maybe +/- some epsilon) and see if results change sign or land anywhere near zero. In reality there are all kinds of tests they could and should be doing to check things like concavity and switch to better algorithms, but that's kind of moot when they're not doing the most basic check. Edit: Actually I could probably say something similar about one of the SymPy bugs too (they really shouldn't have trouble telling if 0.66 + 0.56I is a real number), but one key difference is at least their problem is in filtering out correct solutions, not in returning completely wrong solutions with full confidence.
deleted 6y ago
[deleted]
ogogmad 6y ago
> I would then recommend they not have an option for "real" if they can't get it right. Users will expect it to work (unless the docs point out it is unreliable). You should be suspicious of using floats and expecting the solver to detect if a solution is real. Consider the polynomial $x^2 + C$ for $C \approx 0$. Are all of its roots real or complex? The problem is that even a very small change in C, maybe caused by rounding errors, can easily change a root from real to not real. When solving polynomials, either use complex numbers throughout or insist on exact solutions and don't use floats.
- mehrdadn 6y ago
  > You should be suspicious of using floats and expecting the solver to detect if a solution is real. Consider the polynomial $x^2 + C$ for $C \approx 0$. Are all of its roots real or complex? The problem is that even a very small change in C, maybe caused by rounding errors, can easily change a root from real to not real. C \approx 0? The imaginary parts in the first bug are nowhere close to zero.
  - ogogmad 6y ago
    I was actually responding to the following claim: > I would then recommend they not have an option for "real" if they can't get it right. Users will expect it to work (unless the docs point out it is unreliable). It can't work reliably if the coefficients of the polynomial consist of floats. At least not in general. This point is a more fundamental one than your example. Also, regarding your particular example: When solving cubics using the cubic formula, it is often the case that you will end up computing with complex numbers even if the result ends up real. If you introduce floats into this, it's impossible to guarantee that the imaginary parts will cancel out completely. Instead, they may cancel up to 10^-21 or something like that, which is precisely what happens in your example.
    - mehrdadn 6y ago
      You're missing my point entirely. They don't need to solve this by keeping the variable real at every step or anything like that. What they need to do is to ensure the final solutions are real. Which means they need to solve it in the complex domain entirely and then filter out non-real solutions at the end, where they have the final magnitudes of the real and imaginary components. The only place that would run into trouble is where the imaginary components are small but nonzero, where you'd just expect it to use some tolerance threshold as the cutoff, but that obviously isn't the case in the first example (hence my comment). There's no reason that the algorithm must fail on univariate real polynomials if it can't work equally well on arbitrary functions with arbitrary domains. It's trivial for a symbolic engine to recognize a univariate polynomial. And practically speaking it's the job of a computer algebra system to recognize different well-known classes of functions and treat them as well as it can. What I suspect likely happened here is they've been just so busy dealing with the more general problems (which are far more difficult) that they haven't gotten around to implementing many better methods for the special cases (like polynomials). Which I sympathize with, as I'd have virtually no clue how to solve some of the more general problems to begin with.
leephillips 6y ago
I don’t think there is a general-purpose symbolic math program that doesn’t have some bugs. Mathematica was getting some definite integrals wrong for some years. Anyone who uses these knows to check the results.

bee_rider 6y ago

Have you tried out SageMath? It seemed to work pretty well when I tried it out, but I never got serious really. My needs for symbolic math programming are pretty limited.

mehrdadn 6y ago
I haven't, though I did download it. I was trying to work on an algorithm rather than solve specific problem instances, so I ended up just working around it by using different problem instances that didn't result in these errors.