3 ms·

While interesting, this research is not on the path to "true" artificial intelligence. It may solve playing a majority of video games given enough effort, but t

While interesting, this research is not on the path to "true" artificial intelligence. It may solve playing a majority of video games given enough effort, but that is not the author's stated goal. Why do I think this? The representations of the various algorithms are not unified. The answer to the question "what algorithm lies halfway between JinJerry and YOLOBOT" is difficult to answer. This is because they are discrete solvers who's implementations cannot be seamlessly blended. It is only at the level of their decision outputs that they share a common language. The strength of natural intelligence derive partly from the fact that all strategies are implemented using the same components. Strategies can be described as sparse activation patterns of a neural substrate. Any strategy can be added to, subtracted from, or otherwise combined with another because the activations have a common representation language. The choice is never "either/or" for natural intelligence but "how much of which?" This problem is also found in the meta-selection problem. The decision tree used to choose a component algorithm (even if applied repeatedly throughout the course of a game) doesn't allow for strategic blending, and itself is not implemented in a language common to the strategies. In contrast, biological strategies are selected by competition between signals from many lower level systems and top down control and predictions. Ultimately a decision such as "fight or flight" comes down to how strongly a sparse set of neurons if firing and which motor paths are suppressed and which are activated. Because both the strategies and the evaluation of those strategies are implemented in a common substrate, you can blend, adapt, compare and update all aspects of the system. This is crucial to the adaptability and speed of natural intelligence. Edit: I should note I am in full agreement with the author that simulated environments (of which video games are an example) are going to be essential for the creation of "true" AI. My comments relate specifically to the line of inquiry in the papers described.

derefr 10y ago
Why do you think human cognitive processes don't resemble "discrete solvers who's implementations cannot be seamlessly blended"? Humans often get 'caught half-way' between a fight-or-flight response and a calm, analytical solution to a problem, and end up doing nothing. There's no strategy-blending there. Humans can learn blended strategies by practice, but by doing so they're neuroplastically creating a single new blended-strategy strategy (which can poke at subgraphs of its component strategies' synapses), rather than simultaneously activating two strategies. You can learn to play a game that combines Tetris and a platformer (e.g. Wario's Woods)—and you'll become skilled at this faster if you've played both Tetris and platformers before—but learning this skill won't help you at all in becoming better at multitasking two separate, concurrently-running games of Tetris and a platformer. The synthesized strategy is its own concrete node, not an enhancement of the original nodes to enable efficient blending. --- ETA: I suppose that it might be that "poking at subgraphs" thing that you're getting at, though? When a human learns a game, it gets better at playing all games, because there are individual "subroutines" that can be retried and, if successful, reused to accelerate the learning of other games. Adding this feature to a GGP model wouldn't require "blending" (i.e. fluent instantaneous synthesis of strategy-components), but it would achieve a similar gain in performance. Programming the GGP algorithm to factor the decision-algorithm implementations it generates into hierarchies of small, modular black-box implementations, rather than large, monolithic black-box implementations, would get you a gradually-building "library of functions" that the GGP algorithm could attempt to "slot in" in new algorithms as it's synthesizing them.
iaw 10y ago
I agree with your premise that the answer to "what algorithm lies halfway between JinJerry and YOLOBOT" is difficult to ascertain but I have heard of zero effort at trying to answer such a question. In the end I could see a scenario where all strategy algorithms are ran in parallel with some understanding of the number of turns required for each strategy choice to provide payoff. That net can then weigh the choices and choose the relevant strategy algorithm for a period of time until it deems changing strategy algorithms is appropriate. Things can become even blurrier if the net can choose components from multiple algorithms (depending on the game and the strategy). There are a few whole directions I'd want to go with this work if I were pursuing it.

derefr 10y ago

Why do you think human cognitive processes don't resemble "discrete solvers who's implementations cannot be seamlessly blended"? Humans often get 'caught half-way' between a fight-or-flight response and a calm, analytical solution to a problem, and end up doing nothing. There's no strategy-blending there. Humans can learn blended strategies by practice, but by doing so they're neuroplastically creating a single new blended-strategy strategy (which can poke at subgraphs of its component strategies' synapses), rather than simultaneously activating two strategies. You can learn to play a game that combines Tetris and a platformer (e.g. Wario's Woods)—and you'll become skilled at this faster if you've played both Tetris and platformers before—but learning this skill won't help you at all in becoming better at multitasking two separate, concurrently-running games of Tetris and a platformer. The synthesized strategy is its own concrete node, not an enhancement of the original nodes to enable efficient blending. --- ETA: I suppose that it might be that "poking at subgraphs" thing that you're getting at, though? When a human learns a game, it gets better at playing all games, because there are individual "subroutines" that can be retried and, if successful, reused to accelerate the learning of other games. Adding this feature to a GGP model wouldn't require "blending" (i.e. fluent instantaneous synthesis of strategy-components), but it would achieve a similar gain in performance. Programming the GGP algorithm to factor the decision-algorithm implementations it generates into hierarchies of small, modular black-box implementations, rather than large, monolithic black-box implementations, would get you a gradually-building "library of functions" that the GGP algorithm could attempt to "slot in" in new algorithms as it's synthesizing them.

iaw 10y ago

I agree with your premise that the answer to "what algorithm lies halfway between JinJerry and YOLOBOT" is difficult to ascertain but I have heard of zero effort at trying to answer such a question. In the end I could see a scenario where all strategy algorithms are ran in parallel with some understanding of the number of turns required for each strategy choice to provide payoff. That net can then weigh the choices and choose the relevant strategy algorithm for a period of time until it deems changing strategy algorithms is appropriate. Things can become even blurrier if the net can choose components from multiple algorithms (depending on the game and the strategy). There are a few whole directions I'd want to go with this work if I were pursuing it.