5 ms·

Ask HN: Can we adapt AlphaZero's self-play technique for better human learning?

• stenecdote 9y ago
  It only now occurs to me that the line of thinking I follow here is sub-consciously inspired by section 3 of this Marvin Minsky talk (https://web.media.mit.edu/~minsky/papers/TuringLecture/TuringLecture.html https://web.media.mit.edu/~minsky/papers/TuringLecture/Turin...). If you're at all interested in the intersection of learning and computer science, I highly recommend taking a look.
• conistonwater 9y ago
  Don't humans already do this, in a way? Instead of playing against yourself, you take somebody stronger and play them. You only need on the order of a 100 games of chess against a decent opposition, with some verbal explanations, to reach amateur level. Per-game, this is much more efficient than AlphaZero, which requires millions of games as well as tons of computing power. Surely the main reason AlphaZero uses that particular technique is that nobody can figure out something better? You'd really want it to copy learning techniques from humans (especially learning from many fewer examples), not the other way around.
  • stenecdote 9y ago
    Thanks for being the first to reply. I was worried I'd just get upvotes and no replies! I think you have two separate points, one with which I agree and one with which I disagree. First, I agree (and other commentators about AlphaZero seem to as well) that human learning "algorithms" still beat AlphaZero's on per-game ROI. On the other hand, I disagree that AlphaZero's self-play is no more interesting than a human playing someone better and learning from them. AlphaGo, AlphaZero's predecessor, followed a strategy more like what you described, learning from a large corpus of existing expert chess matches. AlphaZero, on the other hand, requires no training beyond an encoding of the basic rules of chess that it can understand. From there, it bootstraps its understanding of chess without input from experts. This is the piece I find most interesting, see as potentially useful for the future of human learning, and believe differs from practice with an expert teacher. And so I wonder, can we design learning environments where the learner bootstraps their own understanding from a limited input without continuous feedback from an expert or teacher?
    • Cookingboy 9y ago
      > And so I wonder, can we design learning environments where the learner bootstraps their own understanding from a limited input without continuous feedback from an expert or teacher? Why would you remove continuous feedback from expert or teacher? Would that make human learning "faster" and more "efficient"? That approach works for AI because unlike human, AI remembers every single data point with 100% accuracy and can iterate repeatedly without fatigue. It also does not suffer from issues such as boredom and it doesn't require motivation either. By the way, human already learn from experience by bootstraping their own understanding, teachers and experts exist to fast track the beginning phase so a kid doesn't have to play ten thousand games just to reach beginner skill level.
      • stenecdote 9y ago
        > By the way, human already learn from experience by bootstraping their own understanding, teachers and experts exist to fast track the beginning phase so a kid doesn't have to play ten thousand games just to reach beginner skill level. Yeah, I came part of the way to this realization in my reply to your other message.
    • dragonwriter 9y ago
      > And so I wonder, can we design learning environments where the learner bootstraps their own understanding from a limited input without continuous feedback from an expert or teacher? Yes, and we do it all the time. We can just do a lot better with continuous feedback. (And AI probably could, too, if experts that could communicate fast enough not to be a huge drag on the AIs training cycles were available. But since with current technology once we've trained an AI of the type we can make today, we can replicate it, that's not really important; if ever developed AIs that depend on reconfigurable hardware without trivially extractable state, that may change.)
    • olau 9y ago
      I think this is actually how everyone learns! You can't put information into people. You can present it to them, but they need to teach themselves, so to speak. If you have what I think is a good schooling system, it will recognize and emphasize the self-teach aspect - students are encouraged to figure things out on their own. For instance, where I studied CS most of the time was allocated to doing semester projects where we'd be a small self-organized team of 3-7 students working on something with very little external input. You can find similar ideas for schools, e.g. Sudbury schools. I think the Waldorf school has some aspects of it too. I'm sending my children to such a school.
• Cookingboy 9y ago
  Are we sure AlphaZero has better learning efficiency than human? Sure, it reached peak skill after 4 hours of learning, but how many games did it play during those 4 hours? How many moves did it memorize perfectly and analyzed? Are those numbers even achievable by a human in one's lifetime? Even with AlphaZero's efficiency, it still evaluates 80000 moves per second, which is by far more moves than a human grandmaster evaluates in an entire game. If we cut AlphaZero's "processing power" to that of a human, can it still beat a top level human player, let alone other AIs? To me it seems like there is still a long way to go to improve in this space.
  • stenecdote 9y ago
    I agree that AlphaZero's per-game learning efficiency is much shorter than a human's (as mentioned in my other reply). The part that interested me more was the fact that it bootstrapped its learning from the basic rules of each game. Now that I think about it though, one might argue that human learning in a given discipline starts as isolated with feedback only coming from the outside world. This is what we typically call research. But the magic of our education system, when it works, is that we compress the output of this slow process into a faster one and feed it to learners, allowing them to build understanding of knowledge which originally took generations to discover. Riffing off Matt Might's illustrated depiction of a PhD (http://matt.might.net/articles/phd-school-in-pictures/ http://matt.might.net/articles/phd-school-in-pictures/), expanding the circle of knowledge is exponentially slower than getting close to the edge.
• infinity0 9y ago
  AlphaZero plays games with (1) perfect information and (2) well-defined winning conditions. Neither of these hold for most human-learning scenarios. I can imagine that a healthy dose of probability theory (and probably more advanced stuff I don't know about[1]) might improve (1), but (2) is going to keep computer scientists and philosophers and ethicists arguing for quite a long time. :) [1] get the joke, eh? eh? eh?
  • abecedarius 9y ago
    They essentially hold for math, which is a pretty big deal.
    • infinity0 9y ago
      What are the winning conditions for math?
      • ansible 9y ago
        A smaller proof using fewer axioms or other proofs than the current state-of-the-art. Discovering new and "interesting" proofs. Don't ask me to define "interesting" in this context.
        • infinity0 9y ago
          It's exactly my original point that these goals are not well-defined in early 21st century maths.
        • seanwilson 9y ago
          In the field of formal/machine proofs, nobody really cares about the length of the proofs because part of the point is the proofs are checked by the computer back to the basic axioms so you can trust the proofs are correct. Being able to discover long and ugly proofs to difficult theorems or coming up with new theorems would have endless applications.
  • paulcole 9y ago
• jdoliner 9y ago
  Short answer: No, there's nothing new here that can inform better human learning. Longer answer: The concept of self-play isn't new in any sense. All chess players use this technique to some degree. None use only this technique. The advantage of self play is that there's no risk of accidentally picking up someone else's an incorrect assumption. Since you're deriving everything from scratch. Some people take this to extremes, there's a math professor who doesn't read any math papers so that he's deriving everything from first principles and not "contaminating his mind" it works quite well for him but unfortunately I'm blanking on his name. However, commitment to this technique removes one of the major advantages that humans have which is their ability to communicate knowledge amongst themselves in a compact, abstract way with language. Humans also have a pretty good way to mitigate the faulty assumption risk: skepticism. We can reevaluate our assumptions, and, if we deem it necessary, excise them from our mental model. AlphaZero could in theory do the same thing, the reality for AlphaZero though is that there's not much point, it has no use for the sum total of human knowledge on chess, it's capable of recreating that and much more in a few hours. If there is something to be learned from AlphaZero's training it's that you should always be skeptical of your assumptions, that's not anything new, but it's always worth reiterating. It's pretty obviously not feasible to take this to the extremes of AlphaZero though, humans need other humans to learn. Even the math professor who doesn't read papers needed a lot of interfacing with other humans to learn to get to the point where he could derive things from first principles.
  • markussss 9y ago
    >unfortunately I'm blanking on his name John Nash (supposedly) had this mindset? Is that who you're thinking about?
    • jdoliner 9y ago
      That wasn't who I had in mind, but thanks for sharing that example. I think the guy I'm thinking of is at Cornell and still alive. He also might actually be in CS instead of Math. I tried googling it but, unfortunately, "math professor who doesn't read papers" didn't come up with any results.
• ThrustVectoring 9y ago
  Human brain architecture already does "self-play" during REM sleep. So yeah, but the implementation details are "get more sleep" rather than some sort of novel technique.
• forgot-my-pw 9y ago
  I don't think we can learn much from how an engine learns, but we certainly can learn from its results. For example, there's this interesting discussion: https://www.reddit.com/r/chess/comments/7ibzq4/stockfish_vs_alphazero_jerrys_analysis_of/dqy4yzc/ https://www.reddit.com/r/chess/comments/7ibzq4/stockfish_vs_... Because Alphazero did not learn from human games, it looks at the different pieces without attaching values like we do. It has no problems sacrificing a higher "valued" piece for the sake of its strategy.
  • irascible 9y ago
    Yeah. Our human chess heuristics are biased toward strategies that fit within a human working memory. Alphagos working memory is orders of magnitude larger.
• deleted 9y ago
  [deleted]
• eutropia 9y ago
  Remember that AlphaZero played 44 million games of chess, whereas your average professional chess player has played somewhere on the order of 10,000-100,000. Self-play works, but rather slowly.
  • pixl97 9y ago
    How many years did it take the professional to play 100,000 games? How many minutes did it take AGZ to play 44M? It sounds like self play is rather fast to me.
• egypturnash 9y ago
  I would submit that we already have an example of self-play being used as part of a strategy to learn chess: chess problems. Something like "Here's a board position. It looks utterly hopeless but the problem says "Black to mate in 7 moves". How can you get there from here without relying on White making any beginner's mistakes?" is pretty much self-play.
• canadaduane 9y ago
  My impression of Plato is he channeled different people/characters in his writing in order to create adversarial conditions in which he could improve his rhetoric. Perhaps this is similar to AlphaZero's technique?
• donnaglenn 9y ago
  lumineux cream Treating your skin appearance with the right cosmetic or beauty products is the key to maintain a healthy glow on your entire face. However, yourentireskin struggles with several external or atmosphere damage over time. Not only this, hormonal changes are also highly responsible for altering your skincomplexion and requirements. When you were in teenage, the only thing you had to do focus on reducing blemishes. But, once you reach your 30, your one and only priority is to maintain your skin remain soft and hydrated to avoid stubborn wrinkles. http://www.wellness786.com/lumineux-cream/ http://www.wellness786.com/lumineux-cream/ https://www.youtube.com/watch?v=VNx6M_k2IFY https://www.youtube.com/watch?v=VNx6M_k2IFY
• ararar 9y ago
  I think there is a possibility of applying machine learning to teaching humans in the sense of continuous, algorithmic tuning/personalization of lesson plans/teaching strategies to accelerate human learning ... as a teacher's aide in other words.