5 ms·

I think the interesting stuff starts on page 2: With the help of Mathematica, one of us found some counterexamples to our conjectures. Fortunately, another one

I think the interesting stuff starts on page 2: With the help of Mathematica, one of us found some counterexamples to our conjectures. Fortunately, another one of us was using Maple and, when checking those supposed counterexamples, found that they were not counterexamples at all. After revising our algorithms from scratch, we concluded that either the computations performed with Mathematica or the computations performed with Maple had to be wrong. Things started to become clear when the colleague using Mathematica also found some “counterexamples” to the above-mentioned result of Karlin and Szego for the case in (1) and, even more dramatically, his algorithm yielded different outputs given the same inputs. Our conclusion was that Mathematica was computing incorrectly.

angersock 12y ago
And because it's all closed-source, you'll never know how it was finding the wrong answers! I think that the use of closed-source software in any public research or scientific undertaking should probably be verboten.
- hyperbovine 12y ago
  That would be Sage, but (and I say this as somebody who uses Sage wherever possible) it's light years behind Mathematica and always will be. Do you want to be the guy in charge of implementing all of the incredibly complicated functionality in Mathematica? Or even better, writing quality documentation for all of it, which Mathematica has? There is a lot of software out there for which the intersection of the people who possess the technical knowledge to write it correctly, and the people who are willing to do so for free, is empty. (In the cases where it's not, we call those "graduate students" :)
  - AnimalMuppet 12y ago
    Great, but if Mathematica has also not written it correctly, then what? The fact that Mathematica has it and Sage doesn't is irrelevant, if Mathematica has it wrong.
  - octonion 12y ago
    I just used Sage to find all sets of 1 quadrillion positive integers that have the same product and sum. Feel free to try that in Mathematica.
    - jessaustin 12y ago
      Is this a finite set? If so, what's the simplest way to show this?
      - octonion 12y ago
        I came up with my own algorithm and found several speedups, but I haven't written an expanded article yet. I've implemented it in Julia, but Julia's built-in factoring is (unfortunately) far slower than Sage's. A short preliminary article: http://angrystatistician.blogspot.com/2014/08/learn-this-one-weird-trick-and-easily.html http://angrystatistician.blogspot.com/2014/08/learn-this-one... Here's the Sage code: https://github.com/octonion/puzzles/tree/master/product-sum https://github.com/octonion/puzzles/tree/master/product-sum I'd be curious to see the speed of the corresponding Mathematica or Maple code.
        kgwgk 12y ago
        The Mathematica code below is roughly comparable to your vanilla version. Using a 2013 iMac (3.2 GHz Intel Core i5) I get the following timings for Sage/Mathematica (Mathematica gets more competitive for larger problems): 10^8 22s 65s 10^9 225s 635s 10^11 14h20 16h05 However, I found that Sage seems to calculate divisors faster than Mathematica for large numbers. The following are the times to sum the divisors of one million numbers starting at 10^8, 10^9, 10^10, 10^11: Sage: 18s, 21s, 24s, 27s Mathematica: 12s, 14s, 28s, 49s Solutions[pSum_, pProd_] := With[{term = 1 + pSumpProd}, With[{div = Divisors[term]}, Map[Function[x, If[Mod[x + 1, pProd] == 0, (1 + {x, term/x})/pProd, ## &[]]], Take[div, Ceiling[Length[div]/2]]]]] WithNonOneValues[n_, nval_] := Last[Reap[If[nval == 0, Sow /@ Solutions[n - 2, 1], Module[{val = Array[2 &, nval], idx = 1}, While[idx <= nval, With[{pSum = Plus @@ val + n - nval - 2, pProd = Times @@ val}, If[pSum + 2val[[1]] < pProd*val[[1]]^2, If[(++idx) <= nval, val[[Range[idx]]] = val[[idx]] + 1], Map[Function[x, If[Min[x] >= val[[1]], Sow[Flatten[{x, val}], ## &[]]]], Solutions[pSum, pProd]]; idx = 1; val[[1]] = val[[1]] + 1]]]]]]] AllSolutions[n_] := Flatten[Map[Function[x, WithNonOneValues[n, x]], Range[0, Log2[n]]], 2]
  - jasongrout 12y ago
    At least for this research, Sage may have been better. This code follows the random matrix example to show that Sage gives at least consistent results in one of the cases they pointed out. Here's the public SageMathCloud worksheet: https://cloud.sagemath.com/projects/49a2531d-9d02-42c9-9db6-f9551fbfa59e/files/2014-10-24-212837.sagews https://cloud.sagemath.com/projects/49a2531d-9d02-42c9-9db6-... I wish they would have supplied those matrices they printed out in machine-readable format so we could easily put those in Sage too.
    - williamstein 12y ago
      Here's another worksheet with some symbolic integrals from the article: https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/ams-article-mathematica-bugs.sagews https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-... I wrote the det code in Sage that you're calling there (though in Sage-6.4 it'll likely be replaced by a call to FLINT). It computes det(A) in a very interesting way, which is asymptotically massively faster than Mathematica. To compute det(A), choose a random vector v and solve Ax = v using a p-adic lifting algorithm (the one in https://cs.uwaterloo.ca/~astorjoh/iml.html https://cs.uwaterloo.ca/~astorjoh/iml.html). One can prove --using Cramer's rule--that the lcm of the denominators of the entries of x will then be a divisor d of det(A), and with high probability one expects that det(A)/d is a tiny integer. One can then provably (using the Hadamard bound) find det(A) by working modulo a few additional primes and using the Chinese Remainder theorem.
  - judk 12y ago
    Magma gets massive funding from Simons Foundation. It is a shame to have all that money donated to nonfree software program, really.
    - octonion 12y ago
      I couldn't agree more. A terribly wasted opportunity.
  - EdwardDiego 12y ago
    It doesn't have to be free (as in beer) to have source code available. But it does require some out of the box thinking from the vendor - I deal with one licensed product, and they won't give us source code for the JDBC drivers. It's a JDBC driver! It's not doing anything revolutionary, it has nothing to do with the core competitive advantage of your product, ffs.
- GregBuchholz 12y ago
  I suppose that you'll also need to use only open-source operating systems. And motherboards, memory, storage, CPUs, and GPUs. And even then you'll want to audit the supply chain for the computer manufacturer, to make sure that there are no counterfeit open source components being used. Of course that doesn't guarantee that there are no bugs in the implementation of any of the above. Or that you don't have a faulty component. So to develop confidence, you'll have to test against multiple independent implementations. Which you need to do anyway, even in the case that one link in the chain is closed.
  - e12e 12y ago
    If the software is open source, and you test both on ARM, Intel and... Sparc? You should at least be able to get a "majority vote", and be able to narrow down errors in hardware, by running some software tests. Yes, if you're doing work that it would be very high-value to break (say you're the NSA, and China might have an interest in undermining your crypto work) -- that might not be enough. But we know from experience that one can often work around bugs in hardware in software (and/or microcode). So I wouldn't say "just" using open source software on top of proprietary/opaque harware gains you nothing. Also, it's been a long while since using an open source operating system was a controversial choice -- in fact, I think it'd be hard to find a proprietary operating system that both supports good CAS, and runs on at least three different architectures...
deleted 12y ago
[deleted]
- monochr 12y ago
  It shouldn't be. Maple is just as much of a black box as Mathematica. That it didn't screw up on this one task is no guarantee it won't screw up others when you aren't looking. Worse since you can't check the source code you will never know why it failed in any specific case without reverse engineering it. In fact they say as much towards the end of the paper: "Both Mathematica and Maple return zero as the answer to this calculation. Yet this cannot be correct, because the integrand is clearly positive and nonzero in the indicated region."
  - fdej 12y ago
    Maple isn't quite as much of a black box as Mathematica. Many of the library functions in Maple are written in Maple and you can easily read their source code.
igravious 12y ago
My MA thesis, in part, dealt with this topic: why open-source _should_ be used for mathematical software. Because science and the scientific method proceeds via the necessity of verification and because a lot of science is mathematics and because a lot of mathematics is nowadays done on a computer we need to be able to inspect these mathematical black boxes. If you don't mind me promoting my own work, here it is: https://jyx.jyu.fi/dspace/handle/123456789/38495# https://jyx.jyu.fi/dspace/handle/123456789/38495#

angersock 12y ago

And because it's all closed-source, you'll never know how it was finding the wrong answers! I think that the use of closed-source software in any public research or scientific undertaking should probably be verboten.

hyperbovine 12y ago
That would be Sage, but (and I say this as somebody who uses Sage wherever possible) it's light years behind Mathematica and always will be. Do you want to be the guy in charge of implementing all of the incredibly complicated functionality in Mathematica? Or even better, writing quality documentation for all of it, which Mathematica has? There is a lot of software out there for which the intersection of the people who possess the technical knowledge to write it correctly, and the people who are willing to do so for free, is empty. (In the cases where it's not, we call those "graduate students" :)
- AnimalMuppet 12y ago
  Great, but if Mathematica has also not written it correctly, then what? The fact that Mathematica has it and Sage doesn't is irrelevant, if Mathematica has it wrong.
- octonion 12y ago
  I just used Sage to find all sets of 1 quadrillion positive integers that have the same product and sum. Feel free to try that in Mathematica.
  - jessaustin 12y ago
    Is this a finite set? If so, what's the simplest way to show this?
    - octonion 12y ago
      I came up with my own algorithm and found several speedups, but I haven't written an expanded article yet. I've implemented it in Julia, but Julia's built-in factoring is (unfortunately) far slower than Sage's. A short preliminary article: http://angrystatistician.blogspot.com/2014/08/learn-this-one-weird-trick-and-easily.html http://angrystatistician.blogspot.com/2014/08/learn-this-one... Here's the Sage code: https://github.com/octonion/puzzles/tree/master/product-sum https://github.com/octonion/puzzles/tree/master/product-sum I'd be curious to see the speed of the corresponding Mathematica or Maple code.
      - kgwgk 12y ago
        The Mathematica code below is roughly comparable to your vanilla version. Using a 2013 iMac (3.2 GHz Intel Core i5) I get the following timings for Sage/Mathematica (Mathematica gets more competitive for larger problems): 10^8 22s 65s 10^9 225s 635s 10^11 14h20 16h05 However, I found that Sage seems to calculate divisors faster than Mathematica for large numbers. The following are the times to sum the divisors of one million numbers starting at 10^8, 10^9, 10^10, 10^11: Sage: 18s, 21s, 24s, 27s Mathematica: 12s, 14s, 28s, 49s Solutions[pSum_, pProd_] := With[{term = 1 + pSumpProd}, With[{div = Divisors[term]}, Map[Function[x, If[Mod[x + 1, pProd] == 0, (1 + {x, term/x})/pProd, ## &[]]], Take[div, Ceiling[Length[div]/2]]]]] WithNonOneValues[n_, nval_] := Last[Reap[If[nval == 0, Sow /@ Solutions[n - 2, 1], Module[{val = Array[2 &, nval], idx = 1}, While[idx <= nval, With[{pSum = Plus @@ val + n - nval - 2, pProd = Times @@ val}, If[pSum + 2val[[1]] < pProd*val[[1]]^2, If[(++idx) <= nval, val[[Range[idx]]] = val[[idx]] + 1], Map[Function[x, If[Min[x] >= val[[1]], Sow[Flatten[{x, val}], ## &[]]]], Solutions[pSum, pProd]]; idx = 1; val[[1]] = val[[1]] + 1]]]]]]] AllSolutions[n_] := Flatten[Map[Function[x, WithNonOneValues[n, x]], Range[0, Log2[n]]], 2]
- jasongrout 12y ago
  At least for this research, Sage may have been better. This code follows the random matrix example to show that Sage gives at least consistent results in one of the cases they pointed out. Here's the public SageMathCloud worksheet: https://cloud.sagemath.com/projects/49a2531d-9d02-42c9-9db6-f9551fbfa59e/files/2014-10-24-212837.sagews https://cloud.sagemath.com/projects/49a2531d-9d02-42c9-9db6-... I wish they would have supplied those matrices they printed out in machine-readable format so we could easily put those in Sage too.
  - williamstein 12y ago
    Here's another worksheet with some symbolic integrals from the article: https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/ams-article-mathematica-bugs.sagews https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-... I wrote the det code in Sage that you're calling there (though in Sage-6.4 it'll likely be replaced by a call to FLINT). It computes det(A) in a very interesting way, which is asymptotically massively faster than Mathematica. To compute det(A), choose a random vector v and solve Ax = v using a p-adic lifting algorithm (the one in https://cs.uwaterloo.ca/~astorjoh/iml.html https://cs.uwaterloo.ca/~astorjoh/iml.html). One can prove --using Cramer's rule--that the lcm of the denominators of the entries of x will then be a divisor d of det(A), and with high probability one expects that det(A)/d is a tiny integer. One can then provably (using the Hadamard bound) find det(A) by working modulo a few additional primes and using the Chinese Remainder theorem.
- judk 12y ago
  Magma gets massive funding from Simons Foundation. It is a shame to have all that money donated to nonfree software program, really.
  - octonion 12y ago
    I couldn't agree more. A terribly wasted opportunity.
- EdwardDiego 12y ago
  It doesn't have to be free (as in beer) to have source code available. But it does require some out of the box thinking from the vendor - I deal with one licensed product, and they won't give us source code for the JDBC drivers. It's a JDBC driver! It's not doing anything revolutionary, it has nothing to do with the core competitive advantage of your product, ffs.
GregBuchholz 12y ago
I suppose that you'll also need to use only open-source operating systems. And motherboards, memory, storage, CPUs, and GPUs. And even then you'll want to audit the supply chain for the computer manufacturer, to make sure that there are no counterfeit open source components being used. Of course that doesn't guarantee that there are no bugs in the implementation of any of the above. Or that you don't have a faulty component. So to develop confidence, you'll have to test against multiple independent implementations. Which you need to do anyway, even in the case that one link in the chain is closed.
- e12e 12y ago
  If the software is open source, and you test both on ARM, Intel and... Sparc? You should at least be able to get a "majority vote", and be able to narrow down errors in hardware, by running some software tests. Yes, if you're doing work that it would be very high-value to break (say you're the NSA, and China might have an interest in undermining your crypto work) -- that might not be enough. But we know from experience that one can often work around bugs in hardware in software (and/or microcode). So I wouldn't say "just" using open source software on top of proprietary/opaque harware gains you nothing. Also, it's been a long while since using an open source operating system was a controversial choice -- in fact, I think it'd be hard to find a proprietary operating system that both supports good CAS, and runs on at least three different architectures...

deleted 12y ago

[deleted]

monochr 12y ago
It shouldn't be. Maple is just as much of a black box as Mathematica. That it didn't screw up on this one task is no guarantee it won't screw up others when you aren't looking. Worse since you can't check the source code you will never know why it failed in any specific case without reverse engineering it. In fact they say as much towards the end of the paper: "Both Mathematica and Maple return zero as the answer to this calculation. Yet this cannot be correct, because the integrand is clearly positive and nonzero in the indicated region."
- fdej 12y ago
  Maple isn't quite as much of a black box as Mathematica. Many of the library functions in Maple are written in Maple and you can easily read their source code.

igravious 12y ago

My MA thesis, in part, dealt with this topic: why open-source _should_ be used for mathematical software. Because science and the scientific method proceeds via the necessity of verification and because a lot of science is mathematics and because a lot of mathematics is nowadays done on a computer we need to be able to inspect these mathematical black boxes. If you don't mind me promoting my own work, here it is: https://jyx.jyu.fi/dspace/handle/123456789/38495# https://jyx.jyu.fi/dspace/handle/123456789/38495#