4 ms·

I'm a little confused as to what distinction is being made here between "provable" and "unprovable" factoring algorithms. Is it just that "provable" ones may o

I'm a little confused as to what distinction is being made here between "provable" and "unprovable" factoring algorithms. Is it just that "provable" ones may output the correct result all the time, while "unprovable" ones are probabilistic and output the correct result with high probability? (E.g., roughly the distinction between P or ZPP on the one hand and BPP on the other hand, if these hypothetically ran in polynomial time and we were talking about decision problems.) Or is something even weaker meant by "unprovable", like retuning possibly incorrect results based not on a random input but on the number to be factored? I assume it's the former -- if it were the latter I'd say such things are not really factoring algorithms, practically useful as they might be -- but the way it's worded isn't exactly clear.

ColinWright 12y ago
The following is general, and may be wrong in some details. I look forward to someone more knowledgeable than I coming along to fix it to be totally correct, but here are some of the ideas. The unprovable algorithm usually use randomness and/or rely for their correctness on unproved hypotheses, ... As an example, some algorithms rely on the Riemann Hypothesis for their correctness. ... yet are observed to be the fastest in practice for numbers of substantial size. The Pollard Rho algorithm is not a fast one, running in expected time proportional to the square root of the factor found, which is usually (but not always) the smallest factor. however, this effectively relies on the birthday problem, and is only a heuristic, not proven. It's also not guaranteed to factor the number at all. The provable algorithms are usually deterministic, but their key feature is that their correctness is unconditional. And their run time is proven without using unproven hypotheses.
- pbsd 12y ago
  Nitpick: Pollard's rho is unproven not because it relies on the birthday paradox, but because there's no way to prove that f(x) = x^2 + a behaves like a random walk. It may not because, e.g, the length of the cycle may be the same for every prime divisor of the number being factored. Similarly, the complexity of the quadratic sieve is heuristic because there is no known way to prove that (sqrt(n) + i)^2 - n results in the same distribution of smooth integers than that of a random sequence of the same size. Dixon's algorithm [1] turns the quadratic sieve provable (but much slower in practice) by sampling randomly instead of sequentially. Sieving is thus no longer possible, but we get a subexponential running time of L_n[1/2, 2 sqrt(2)]. The hyperelliptic curve factorization algorithm [2, 3] is another probabilistic but provable algorithm with a more efficient but not provable counterpart, the elliptic curve method [4]. The blog post only seems to care about deterministic factoring algorithms, though. There are indeed no known deterministic factorization algorithms with better than exponential complexity. [1] http://www.ams.org/journals/mcom/1981-36-153/S0025-5718-1981-0595059-1/home.html http://www.ams.org/journals/mcom/1981-36-153/S0025-5718-1981... [2] https://math.dartmouth.edu/~carlp/hyperI.pdf https://math.dartmouth.edu/~carlp/hyperI.pdf [3] https://math.dartmouth.edu/~carlp/PDF/paper129.pdf https://math.dartmouth.edu/~carlp/PDF/paper129.pdf [4] https://openaccess.leidenuniv.nl/bitstream/handle/1887/3826/346_086.pdf?sequence=1 https://openaccess.leidenuniv.nl/bitstream/handle/1887/3826/...
  - ColinWright 12y ago
    Perfect - thank you.
  - Sniffnoy 12y ago
    Huh -- I had always assumed when people talked about e.g. GNFS being the fastest known, they meant proven. I had no idea about this, thank you.
algorias 12y ago
Well, in math some things are true but not provable unless our formal system is inconsistent. This is one of Gödel's incompleteness theorems. It's therefore not far fetched that some algorithms might in fact always work correctly, but at the same time a proof for their correctness doesn't exist. You can never be sure (literally!) that you're not just confusing it with an algorithm that's wrong but happens to work on all problem instances you've thrown at it so far. IIUC, when the article says unprovable, what it actually means is unproven. To correctly describe an algorithm as unprovable, you would have to have a proof of its unprovability, which would be quite something.
- thaumasiotes 12y ago
  There's no obstacle in principle to proving that something can't be proved. As far as I understand things, the continuum hypothesis has been shown to be unprovable within ZFC (since ZFC is consistent with both the truth and the falsity of the continuum hypothesis, obviously it is unable to prove the hypothesis true or false), and theoretical computer science produces results of the form "to prove X, the following kinds of approaches cannot work".
  - JadeNB 12y ago
    "ZFC is consistent with …" must, of course, be replaced by "If ZFC is consistent, then so are …". In fact, singling out ZFC gives you another example of this kind of result: if ZF is consistent, then so are ZF+C and ZF-C.
    - thaumasiotes 12y ago
      That all depends whether you're comfortable saying a contradiction is "consistent" with an unrelated proposition; I don't see much of a problem with it. Contradictions, being impossible, are logically consistent with everything else. I guess unprovable propositions would switch over to being provable, though.
      - JadeNB 12y ago
deleted 12y ago
[deleted]

ColinWright 12y ago

The following is general, and may be wrong in some details. I look forward to someone more knowledgeable than I coming along to fix it to be totally correct, but here are some of the ideas. The unprovable algorithm usually use randomness and/or rely for their correctness on unproved hypotheses, ... As an example, some algorithms rely on the Riemann Hypothesis for their correctness. ... yet are observed to be the fastest in practice for numbers of substantial size. The Pollard Rho algorithm is not a fast one, running in expected time proportional to the square root of the factor found, which is usually (but not always) the smallest factor. however, this effectively relies on the birthday problem, and is only a heuristic, not proven. It's also not guaranteed to factor the number at all. The provable algorithms are usually deterministic, but their key feature is that their correctness is unconditional. And their run time is proven without using unproven hypotheses.

pbsd 12y ago
Nitpick: Pollard's rho is unproven not because it relies on the birthday paradox, but because there's no way to prove that f(x) = x^2 + a behaves like a random walk. It may not because, e.g, the length of the cycle may be the same for every prime divisor of the number being factored. Similarly, the complexity of the quadratic sieve is heuristic because there is no known way to prove that (sqrt(n) + i)^2 - n results in the same distribution of smooth integers than that of a random sequence of the same size. Dixon's algorithm [1] turns the quadratic sieve provable (but much slower in practice) by sampling randomly instead of sequentially. Sieving is thus no longer possible, but we get a subexponential running time of L_n[1/2, 2 sqrt(2)]. The hyperelliptic curve factorization algorithm [2, 3] is another probabilistic but provable algorithm with a more efficient but not provable counterpart, the elliptic curve method [4]. The blog post only seems to care about deterministic factoring algorithms, though. There are indeed no known deterministic factorization algorithms with better than exponential complexity. [1] http://www.ams.org/journals/mcom/1981-36-153/S0025-5718-1981-0595059-1/home.html http://www.ams.org/journals/mcom/1981-36-153/S0025-5718-1981... [2] https://math.dartmouth.edu/~carlp/hyperI.pdf https://math.dartmouth.edu/~carlp/hyperI.pdf [3] https://math.dartmouth.edu/~carlp/PDF/paper129.pdf https://math.dartmouth.edu/~carlp/PDF/paper129.pdf [4] https://openaccess.leidenuniv.nl/bitstream/handle/1887/3826/346_086.pdf?sequence=1 https://openaccess.leidenuniv.nl/bitstream/handle/1887/3826/...
- ColinWright 12y ago
  Perfect - thank you.
- Sniffnoy 12y ago
  Huh -- I had always assumed when people talked about e.g. GNFS being the fastest known, they meant proven. I had no idea about this, thank you.

algorias 12y ago

Well, in math some things are true but not provable unless our formal system is inconsistent. This is one of Gödel's incompleteness theorems. It's therefore not far fetched that some algorithms might in fact always work correctly, but at the same time a proof for their correctness doesn't exist. You can never be sure (literally!) that you're not just confusing it with an algorithm that's wrong but happens to work on all problem instances you've thrown at it so far. IIUC, when the article says unprovable, what it actually means is unproven. To correctly describe an algorithm as unprovable, you would have to have a proof of its unprovability, which would be quite something.

thaumasiotes 12y ago
There's no obstacle in principle to proving that something can't be proved. As far as I understand things, the continuum hypothesis has been shown to be unprovable within ZFC (since ZFC is consistent with both the truth and the falsity of the continuum hypothesis, obviously it is unable to prove the hypothesis true or false), and theoretical computer science produces results of the form "to prove X, the following kinds of approaches cannot work".
- JadeNB 12y ago
  "ZFC is consistent with …" must, of course, be replaced by "If ZFC is consistent, then so are …". In fact, singling out ZFC gives you another example of this kind of result: if ZF is consistent, then so are ZF+C and ZF-C.
  - thaumasiotes 12y ago
    That all depends whether you're comfortable saying a contradiction is "consistent" with an unrelated proposition; I don't see much of a problem with it. Contradictions, being impossible, are logically consistent with everything else. I guess unprovable propositions would switch over to being provable, though.
    - JadeNB 12y ago

deleted 12y ago

[deleted]