4 ms·

While there are those in the AI community who believe scaling laws will unearth such programs, they are deluded. There are problems which scale faster than our

While there are those in the AI community who believe scaling laws will unearth such programs, they are deluded. There are problems which scale faster than our computational resources allow, even if hardware scaling continues unabated for millennia to come. The space of all programs in 10TB is far too large to count, even if we could harness galactic computation. Even within a much smaller search space, there are valid programs which cannot be found by gradient descent. Let BT(n) be the number of distinct binary trees less than or equal to height n. This number scales according to the following recurrence relation: BT(n+1)=(BT(n)+2)²−1 Consider the space of all binary trees of height 20 - there fewer atoms in the visible universe. And this is just laying out bits on a hard drive. There are other functions (e.g. Busy beaver and friends) which scale even faster. The space of valid programs in 10TB is too large to enumerate, never mind evaluate. In case anyone here is interested in learning more about program synthesis, there is a new workshop at NeurIPS 2021 which explores some of these topics. You can check it out here: https://aiplans.github.io/ https://aiplans.github.io/

supperburg 5y ago
Yeah
twofornone 5y ago
You're not wrong, but the numbers are quite different if you have a strong search heuristic. And that is where neural nets excel. See AlphaGo and derivatives. So this isn't just some random gradient descent exploring the full space; a properly designed and trained neural net will effectively cordon off vast swaths of the search space, and suddenly the problem is probably many orders of magnitude more tractable. That's really why deep learning shines in technical applications. Solution search heuristics were until recently solely within the domain of biological neural networks; now we have created technology which is capable of extracting superior heuristics over the course of learning. And it's already paying off in industrial science, despite the cries of naysayers, though the applications are still in infancy.
- bmc7505 5y ago
  There is a vast chasm of computational complexity between Chess, Go, and protein folding, to program induction. Unlike problems where the configuration space grows exponentially, the space of valid programs is at least super-exponential and depending on the language family, (e.g. context free, context sensitive, recursively enumerable), can often be undecidable. Furthermore, many language induction problems do not have optimal substructure or overlapping subproblems, two important prerequisites for reinforcement learning to work. In these settings, gradient-based heuristics will only get you so far. If you are interested in learning more about the limits of gradient descent, you should look into the TerpreT problem [1]. There are surprisingly tiny Boolean circuits which can be found using constraint solving, but we have not yet been able to learn despite the success of reinforcement learning in other domains. I'm not saying that program induction is impossible, but it is extremely hard even for relative "simple" languages like source code. [1]: https://arxiv.org/pdf/1608.04428.pdf https://arxiv.org/pdf/1608.04428.pdf
  - bhntr3 5y ago
    Thank you. I just finished my master's thesis in program synthesis/induction. You're explaining this better than I could.
  - twofornone 5y ago
    So what do you think is different about human learning that allows us to learn to find solutions in such large spaces?
    - bmc7505 5y ago
      There are two answers to your question. One is, humans have simply not found very good solutions in these spaces. We have designed algorithms which barely work on denumerable sets, but are unsuitable in more general spaces. There are many decision problems which are known to be undecidable, i.e., there exists no algorithm that decides first order logic propositions. There are fragments which are decidable, and essentially all useful algorithms in CS fall into this category. The second answer is that we are exploiting statistical regularities in the data which make these problems effectively regular or context-free in practice. When someone asks you to solve a new programming problem, you are applying some heuristics that have worked on similar problems in the past. Given a truly novel problem, you can either make some assumptions to reduce it into a more tractable form (e.g. 3-SAT), or design some clever search heuristic, but without any prior examples of programs or a distribution over probable inputs, you can do no better than naïve search.
Veedrac 5y ago
Like most anti-AGI arguments, this one disproves humans, and is therefore wrong.
- bruce343434 5y ago
  Do humans fit in 10TB? Do humans fit in our computational model at all? Neurons aren't simple callback closures or matrices, and neurons don't even form the full picture.
  - supperburg 5y ago
    A lot of the brain is for unrelated things. Imagine how many neurons you eliminate by excluding the cerebellum or the brain stem. And if you carved away the brain until only the intellect remained, distilling its digital equivalent could reduce its size even more. And if it were just the algorithm, I think 10TB would be enough.
    - cantrevealname 5y ago
      >> Do humans fit in 10TB? > If it were just the algorithm, I think 10TB would be enough Just thinking about it makes me shiver. I had been assuming that AGI or strong AI needed enormous advances in hardware like quantum computing or numerous iterations of Moore's Law. Would it be correct to say that the right 10TB bit pattern might give us strong AI on today's commodity hardware -- at least in theory?
      - supperburg 5y ago
        It’s such a fascinating question. Given some domestic computer, what is the most intelligent program it might be able to run in something like real-time? My intuition is that domestic computers could achieve a sparse intellect but maybe not a rich sensory relationship with the physical world. With every year we expand our vast collective computational resources. It’s like a powder keg. It’s sitting dormant just waiting for a good program to be stumbled upon. I hope I’m not alive when that happens
      - 5y ago
codeisawesome 5y ago
Somewhat off topic, and I ask out of genuine curiosity - when did the “this computation will take sifting through more combinations than there are visible atoms in the universe” meme emerge? I’ve never quite understood why that’s an important metric for considering processing. Does it have an actual impact on the computability of something, or is it just a visualisation to help human minds scale the grasp of something?
- bmc7505 5y ago
  We don't have the words to describe how quickly these nonlinear recurrence relations grow. Many program synthesis problems are essentially super-exponential. Look at the Ackermann function, which describes the complexity of many decidable algorithms as you increase problem size -- it's a different kind of scaling altogether.
- rank0 5y ago
  It’s a pretty useful concept when you consider cryptography. The idea that enumeration is mathematically infeasible for some set is the basis for many cryptographic functions. But basically yeah it’s just an illustration of the difficulty. Obviously we’ll never have a supercomputer’s worth of computing power for every atom.
- kragen 5y ago
  I prefer "sifting through more combinations than there are elementary particles in the universe times the universe's age in picoseconds"; it's a fairly reliable indicator that exhaustive search is not a viable way to solve the problem, not just today but, with classical computing, ever, because it's very unlikely that we can make a working computer out of less than a single elementary particle, try more than one guess per picosecond (per computer), or wait longer than the current age of the universe for an answer. I've seen such calculations at least since the 01990s. It might turn out to be wrong (for example, elementary particles or black holes might have exploitable structure, a picosecond is pretty long compared to the Planck time, or closed timelike curves in spacetime might allow you to spend an unboundedly long time computing something), but at the very least it's a strong suggestion that exhaustive search will not be fruitful. For more immediate purposes I prefer the dollar cost of carrying out the computation with currently available hardware. Quantum computing, when it becomes practical, will give you only a quadratic speedup, as far as we know. So the relevant problem size then expands to the square of the number above.

supperburg 5y ago

Yeah

twofornone 5y ago

You're not wrong, but the numbers are quite different if you have a strong search heuristic. And that is where neural nets excel. See AlphaGo and derivatives. So this isn't just some random gradient descent exploring the full space; a properly designed and trained neural net will effectively cordon off vast swaths of the search space, and suddenly the problem is probably many orders of magnitude more tractable. That's really why deep learning shines in technical applications. Solution search heuristics were until recently solely within the domain of biological neural networks; now we have created technology which is capable of extracting superior heuristics over the course of learning. And it's already paying off in industrial science, despite the cries of naysayers, though the applications are still in infancy.

bmc7505 5y ago
There is a vast chasm of computational complexity between Chess, Go, and protein folding, to program induction. Unlike problems where the configuration space grows exponentially, the space of valid programs is at least super-exponential and depending on the language family, (e.g. context free, context sensitive, recursively enumerable), can often be undecidable. Furthermore, many language induction problems do not have optimal substructure or overlapping subproblems, two important prerequisites for reinforcement learning to work. In these settings, gradient-based heuristics will only get you so far. If you are interested in learning more about the limits of gradient descent, you should look into the TerpreT problem [1]. There are surprisingly tiny Boolean circuits which can be found using constraint solving, but we have not yet been able to learn despite the success of reinforcement learning in other domains. I'm not saying that program induction is impossible, but it is extremely hard even for relative "simple" languages like source code. [1]: https://arxiv.org/pdf/1608.04428.pdf https://arxiv.org/pdf/1608.04428.pdf
- bhntr3 5y ago
  Thank you. I just finished my master's thesis in program synthesis/induction. You're explaining this better than I could.
- twofornone 5y ago
  So what do you think is different about human learning that allows us to learn to find solutions in such large spaces?
  - bmc7505 5y ago
    There are two answers to your question. One is, humans have simply not found very good solutions in these spaces. We have designed algorithms which barely work on denumerable sets, but are unsuitable in more general spaces. There are many decision problems which are known to be undecidable, i.e., there exists no algorithm that decides first order logic propositions. There are fragments which are decidable, and essentially all useful algorithms in CS fall into this category. The second answer is that we are exploiting statistical regularities in the data which make these problems effectively regular or context-free in practice. When someone asks you to solve a new programming problem, you are applying some heuristics that have worked on similar problems in the past. Given a truly novel problem, you can either make some assumptions to reduce it into a more tractable form (e.g. 3-SAT), or design some clever search heuristic, but without any prior examples of programs or a distribution over probable inputs, you can do no better than naïve search.

Veedrac 5y ago

Like most anti-AGI arguments, this one disproves humans, and is therefore wrong.

bruce343434 5y ago
Do humans fit in 10TB? Do humans fit in our computational model at all? Neurons aren't simple callback closures or matrices, and neurons don't even form the full picture.
- supperburg 5y ago
  A lot of the brain is for unrelated things. Imagine how many neurons you eliminate by excluding the cerebellum or the brain stem. And if you carved away the brain until only the intellect remained, distilling its digital equivalent could reduce its size even more. And if it were just the algorithm, I think 10TB would be enough.
  - cantrevealname 5y ago
    >> Do humans fit in 10TB? > If it were just the algorithm, I think 10TB would be enough Just thinking about it makes me shiver. I had been assuming that AGI or strong AI needed enormous advances in hardware like quantum computing or numerous iterations of Moore's Law. Would it be correct to say that the right 10TB bit pattern might give us strong AI on today's commodity hardware -- at least in theory?
    - supperburg 5y ago
      It’s such a fascinating question. Given some domestic computer, what is the most intelligent program it might be able to run in something like real-time? My intuition is that domestic computers could achieve a sparse intellect but maybe not a rich sensory relationship with the physical world. With every year we expand our vast collective computational resources. It’s like a powder keg. It’s sitting dormant just waiting for a good program to be stumbled upon. I hope I’m not alive when that happens
    - 5y ago

codeisawesome 5y ago

Somewhat off topic, and I ask out of genuine curiosity - when did the “this computation will take sifting through more combinations than there are visible atoms in the universe” meme emerge? I’ve never quite understood why that’s an important metric for considering processing. Does it have an actual impact on the computability of something, or is it just a visualisation to help human minds scale the grasp of something?

bmc7505 5y ago
We don't have the words to describe how quickly these nonlinear recurrence relations grow. Many program synthesis problems are essentially super-exponential. Look at the Ackermann function, which describes the complexity of many decidable algorithms as you increase problem size -- it's a different kind of scaling altogether.
rank0 5y ago
It’s a pretty useful concept when you consider cryptography. The idea that enumeration is mathematically infeasible for some set is the basis for many cryptographic functions. But basically yeah it’s just an illustration of the difficulty. Obviously we’ll never have a supercomputer’s worth of computing power for every atom.
kragen 5y ago
I prefer "sifting through more combinations than there are elementary particles in the universe times the universe's age in picoseconds"; it's a fairly reliable indicator that exhaustive search is not a viable way to solve the problem, not just today but, with classical computing, ever, because it's very unlikely that we can make a working computer out of less than a single elementary particle, try more than one guess per picosecond (per computer), or wait longer than the current age of the universe for an answer. I've seen such calculations at least since the 01990s. It might turn out to be wrong (for example, elementary particles or black holes might have exploitable structure, a picosecond is pretty long compared to the Planck time, or closed timelike curves in spacetime might allow you to spend an unboundedly long time computing something), but at the very least it's a strong suggestion that exhaustive search will not be fruitful. For more immediate purposes I prefer the dollar cost of carrying out the computation with currently available hardware. Quantum computing, when it becomes practical, will give you only a quadratic speedup, as far as we know. So the relevant problem size then expands to the square of the number above.