18 ms·

AlphaGo documentary (2020) [video]

249 points by rdli 5y ago

notanog 5y ago
Its starwars for AI starters.
DantesKite 5y ago
It really is a brilliant documentary, even if you have no interest in artificial intelligence or Go.
vavooom 5y ago
This documentary really helped me understand and appreciate Go way more. Started my new found love of the game. Wish more people in the US played!
- therein 5y ago
  There are quite a lot of us actually. See you on online-go.com.
plasma 5y ago
This is a great watch, since then there has been AlphaGo Zero which surpassed the AI you see play Lee, in 3 days. https://deepmind.com/blog/article/alphago-zero-starting-scratch https://deepmind.com/blog/article/alphago-zero-starting-scra...
- some_cut 5y ago
  There has also been AlphaZero [1], a generalized version of AlphaGo which also has been trained to play Chess and Shogi (all learning from only the rules), and MuZero [2], which is a further generalization which can also play Atari games and does not even use the rules of the game when doing tree search - it has to learn a model of the rules instead. [1] https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go https://deepmind.com/blog/article/alphazero-shedding-new-lig... [2] https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules https://deepmind.com/blog/article/muzero-mastering-go-chess-...
platz 5y ago
feel free to add/play me https://online-go.com/player/1021081/ https://online-go.com/player/1021081/ ~8k
platz 5y ago
Michael Redmond's Go TV (was U.S. commentator for the official alphago games) https://www.youtube.com/channel/UCRJyagla1B5cxIfR4i2LdgA/videos?sort=p https://www.youtube.com/channel/UCRJyagla1B5cxIfR4i2LdgA/vid... He has some nice playlists: - how to play go (for beginners) https://www.youtube.com/watch?v=KTWujSwL2bQ&list=PLW5_cMTm0wvamCNX7qNoUqbXxeHt9n67i https://www.youtube.com/watch?v=KTWujSwL2bQ&list=PLW5_cMTm0w... - basic openings (joseki) https://www.youtube.com/playlist?list=PLW5_cMTm0wvZOTchMWZagFDHuUX0cEk4I https://www.youtube.com/playlist?list=PLW5_cMTm0wvZOTchMWZag... - joseki made popular by AlphaGo/AIs https://www.youtube.com/playlist?list=PLW5_cMTm0wvZU5pQhmQFwXh-ojU1mQIg3 https://www.youtube.com/playlist?list=PLW5_cMTm0wvZU5pQhmQFw... also of interest, Go Pro Yeonwoo (korean go professional) https://www.youtube.com/user/goingceo/videos https://www.youtube.com/user/goingceo/videos
- spindle 5y ago
  I'm so pleased to see this upvoted. Michael Redmond is unspeakably wonderful. He's a top professional player, is fantastic at explaining everything to a kyu player, and prepares his videos really dilligently.
  - falcrist 5y ago
    Isn't he also the only American to reach the professional grade of 9-Dan?
- yesenadam 5y ago
  Ahh I wish I'd watched the first few in that beginners playlist before watching the movie! Thanks for that.
sillysaurusx 5y ago
Highly recommend the Deep Blue documentary too. https://www.youtube.com/watch?v=HwF229U2ba8&ab_channel=FredrikKnudsen https://www.youtube.com/watch?v=HwF229U2ba8&ab_channel=Fredr... It's easy to forget that none of this was guaranteed to work. Nowadays it feels inevitable that Chess and Go would fall to computers, but in the moment it was quite a different experience.
mNovak 5y ago
This film in large part inspired me to start playing Go
- tkgally 5y ago
  It's a great film and a great game. I learned how to play about forty years ago, when I was studying math in graduate school in the United States. I liked go partly because I felt refreshed after playing it, whether I won or lost, while chess had given me headaches. I moved to Japan a few years later, and for a while I played fairly regularly at go clubs in Kabukicho and Takadanobaba in Tokyo. The shot early in the film of some old guys playing go reminded me of those places. They were seedy and smoky, but you could get a game any time of the day or night. Many of the older players seemed practically to live there. Once I got matched against someone who I found out later was a highly ranked professional. At first he seemed interested in playing against a foreigner, but he started looking bored after about my fifth move. I stopped playing after my kids were born, and I haven't sat at a board—or played a game against a human online—in thirty years. But now that I’m approaching retirement myself, and I have a grandchild who will soon be old enough to learn, I’m thinking that maybe I should take it up again.
dstala 5y ago
Narration, BGM - all good. Even someone with absolute no interest in AI will get goosebumps. (Man) vs (Man with Machine)
esturk 5y ago
I'm curious to ask what's next for this series of AI at Deepmind? Is there other more challenging problems they are tackling at the moment? I read they have already master StarCraft 2 even. Is it the case that they have stopped this series of AI and going all in on protein folding at the moment?
- d13 5y ago
  I’d like to see AI take on a really difficult game like Root.
- shmageggy 5y ago
  This blog post mentions testing on Atari and being applied to chemistry and quantum physics https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules https://deepmind.com/blog/article/muzero-mastering-go-chess-...
  - westurner 5y ago
    AlphaFold 2 solved the CASP protein folding problem that AFAIU e.g. Folding@home et. al have been churning at for awhile FWIU. From November 2020: https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology https://deepmind.com/blog/article/alphafold-a-solution-to-a-... https://en.wikipedia.org/wiki/AlphaFold#SARS-CoV-2 https://en.wikipedia.org/wiki/AlphaFold#SARS-CoV-2 : > AlphaFold has been used to a predict structures of proteins of SARS-CoV-2, the causative agent of COVID-19 [...] The team acknowledged that though these protein structures might not be the subject of ongoing therapeutical research efforts, they will add to the community's understanding of the SARS-CoV-2 virus.[74] Specifically, AlphaFold 2's prediction of the structure of the ORF3a protein was very similar to the structure determined by researchers at University of California, Berkeley using cryo-electron microscopy. This specific protein is believed to assist the virus in breaking out of the host cell once it replicates. This protein is also believed to play a role in triggering the inflammatory response to the infection (... Berkeley ALS and SLAC beamlines ... S309 & Sotrovimab: https://scitechdaily.com/inescapable-covid-19-antibody-discovery-neutralizes-all-known-sars-cov-2-strains/ https://scitechdaily.com/inescapable-covid-19-antibody-disco... ) Is there yet an open implementation of AlphaFold 2? edit: https://github.com/search?q=alphafold https://github.com/search?q=alphafold ... https://github.com/deepmind/alphafold https://github.com/deepmind/alphafold How do I reframe this problem in terms of fundamental algorithmic complexity classes (and thus the Quantum Algorithm Zoo thing that might optimize the currently fundamentally algorithmically computationally hard part of the hot loop that is the cost driver in this implementation)? To cite in full from the MuZero blog post from December 2020: https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules https://deepmind.com/blog/article/muzero-mastering-go-chess-... : > Researchers have tried to tackle this major challenge in AI by using two main approaches: lookahead search or model-based planning. > Systems that use lookahead search, such as AlphaZero, have achieved remarkable success in classic games such as checkers, chess and poker, but rely on being given knowledge of their environment’s dynamics, such as the rules of the game or an accurate simulator. This makes it difficult to apply them to messy real world problems, which are typically complex and hard to distill into simple rules. > Model-based systems aim to address this issue by learning an accurate model of an environment’s dynamics, and then using it to plan. However, the complexity of modelling every aspect of an environment has meant these algorithms are unable to compete in visually rich domains, such as Atari. Until now, the best results on Atari are from model-free systems, such as DQN, R2D2 and Agent57. As the name suggests, model-free algorithms do not use a learned model and instead estimate what is the best action to take next. > MuZero uses a different approach to overcome the limitations of previous approaches. Instead of trying to model the entire environment, MuZero just models aspects that are important to the agent’s decision-making process. After all, knowing an umbrella will keep you dry is more useful to know than modelling the pattern of raindrops in the air. > Specifically, MuZero models three elements of the environment that are critical to planning: > * The value: how good is the current position? > * The policy: which action is the best to take? > * The reward: how good was the last action? > These are all learned using a deep neural network and are all that is needed for MuZero to understand what happens when it takes a certain action and to plan accordingly. > Illustration of how Monte Carlo Tree Search can be used to plan with the MuZero neural networks. Starting at the current position in the game (schematic Go board at the top of the animation), MuZero uses the representation function (h) to map from the observation to an embedding used by the neural network (s0). Using the dynamics function (g) and the prediction function (f), MuZero can then consider possible future sequences of actions (a), and choose the best action. > MuZero uses the experience it collects when interacting with the environment to train its neural network. This experience includes both observations and rewards from the environment, as well as the results of searches performed when deciding on the best action. > During training, the model is unrolled alongside the collected experience, at each step predicting the previously saved information: the value function v predicts the sum of observed rewards (u), the policy estimate (p) predicts the previous search outcome (π), the reward estimate r predicts the last observed reward (u). This approach comes with another major benefit: MuZero can repeatedly use its learned model to improve its planning, rather than collecting new data from the environment. For example, in tests on the Atari suite, this variant - known as MuZero Reanalyze - used the learned model 90% of the time to re-plan what should have been done in past episodes. FWIU, from what's going on over there: AlphaGo => AlphaGo {Fan, Lee, Master, Zero} => AlphaGoZero => AlphaZero => MuZero AlphaGo: https://en.wikipedia.org/wiki/AlphaGo_Zero https://en.wikipedia.org/wiki/AlphaGo_Zero AlphaZero: https://en.wikipedia.org/wiki/AlphaZero https://en.wikipedia.org/wiki/AlphaZero MuZero: https://en.wikipedia.org/wiki/MuZero https://en.wikipedia.org/wiki/MuZero AlphaFold {1,2}: https://en.wikipedia.org/wiki/AlphaFold https://en.wikipedia.org/wiki/AlphaFold IIRC, there is not an official implementation of e.g. AlphaZero or MuZero with e.g. openai/gym (and openai/retro) for comparing reinforcement learning algorithms? https://github.com/openai/gym https://github.com/openai/gym What are the benchmarks for Applied RL? From https://news.ycombinator.com/item?id=28499001 https://news.ycombinator.com/item?id=28499001 : > AFAIU, while there are DLTs that cost CPU, RAM, and Data storage between points in spacetime, none yet incentivize energy efficiency by varying costs depending upon whether the instructions execute on a FPGA, ASIC, CPU, GPU, TPU, or QPU? [...] > To be 200% green - to put a 200% green footer with search-discoverable RDFa on your site - I think you need PPAs and all directly sourced clean energy. > (Energy efficiency is very relevant to ML/AI/AGI, because while it may be the case that the dumb universal function approximator will eventually find a better solution, "just leave it on all night/month/K12+postdoc" in parallel is a very expensive proposition with no apparent oracle; and then to ethically filter solutions still costs at least one human)
CalChris 5y ago
I don't play go but the AlphaZero chess games are quite beautiful. Agadmator has a description of the weird unhuman like machine continuations, disgusting engine lines. They're correct but incomputable by a human and just look weird. AlphaZero had some beautiful lines that looked human but slightly counterintuitive. They were the sort of moves that humans could learn from. Are the AlphaGo games similar?
- mattbillenstein 5y ago
  Agreed, I think ChessNetwork analyzed several of the published games as well - and some of the Leela games - they're so much more interesting than most of what I see out of Stockfish...
- vmilner 5y ago
  Yes, very much so - for example it used to be almost axiomatic (told in your second or third go lesson) that placing a white stone at the 3-3 point when black had played at 4-4 was very bad, but this video shows that AlphaGo has radically changed this view: https://www.youtube.com/watch?v=2khNnE5Q3GM https://www.youtube.com/watch?v=2khNnE5Q3GM
- AboveTheGame 5y ago
  ///UNPOPULAR OPINION WARNING/// No, they’re for inept idiots who are obsessed with mathematically solved abstractions of reality. My god, to someone who sees chaos maps in reality the patterns these pathetic programming abstractions offer is a limitation of /our current society/ — we are slaves to wealth and process over progress and intuition. Nothing more need be said, I am always throwing Pearls before swine.
- AboveTheGame 5y ago
  You are also NOT “that guy” Fuck this gay simulation lmao Hahahahaha get pressed on y’all soft ass morons
- roenxi 5y ago
  Go is a bit different from Chess, there are so many options that you can't really have a "disgusting engine line", because either sequences are forced and humans can find them too eventually or the number of lines the move effects is so vast that we would never realise how clever it is. Computer Go is obviously making calculations that a human couldn't, but as an observer it just looks like not making any mistakes. There is the occasional spectacular attack or defence, but humans do that too from time to time - human players often see or try moves that others wouldn't. In complicated situations computers will do a better job at assessing what the impacts are. In Go, to win a player needs to consistently make moves that are on average 0.0025 points better than their opponent for 300 moves. It is hard to detect the genius if a neural net decides it is going to win the game that way. The games are interesting to study, they are all masterworks. But the strength of a Go player is in the consistency rather than individual flashes of insight.
bumbledraven 5y ago
(2020)
- Kiro 5y ago
  The movie is from 2017.
bumbledraven 5y ago
"7% Documentary: Behind the scenes of Fine Art AI - 纪录片《7%》：揭秘人工智能“绝艺”夺冠幕后腾讯网 - English subtitles" (https://v.qq.com/x/page/r0025m06t5o.html https://v.qq.com/x/page/r0025m06t5o.html) is a neat 2018 documentary about Fine Art, a world-class Go AI created by Chinese developers at Tencent. The documentary features extensive commentary by the main programmers, Ma Bo and Tang Shanmin, as well as the project lead Liu Yongsheng. This bit from 4:02 stuck out to me: INTERVIEWER: What level do you think Fine Art has reached? MA BO: About the same level AlphaGo had when playing against Li Shiqi last year. INTERVIEWER: Then wouldn't you regard what you do as a redundant work? MA BO: How should I say this? Its like, when China made the atom bomb after America. Was that redundant work?
- DSingularity 5y ago
  The Chinese have a level of nationalism that many nations lack.
  - duttaditya18 5y ago
    That is a good thing.
    - pathseeker 5y ago
      no it's not "identification with one's own nation and support for its interests, especially to the exclusion or detriment of the interests of other nations."
    - worrycue 5y ago
      I feel it just artificially and pointlessly divides us and create needless "us vs them" situations. Nationalism has interesting "properties". It carries with it the implicit assumption that people miles away in the same nation that you have never met share some of the same beliefs and values you do. It somehow justify you taking credit for the accomplishments of others in the same nation even though you haven't done anything to contribute to said accomplishment.
manigandham 5y ago
One of the best films I've ever seen. Brilliant storytelling about man vs machine at the last frontier of our minds.
Kiro 5y ago
Why did the mods change the title and put an incorrect year in it? The movie is from 2017.
- throwaway81523 5y ago
  Thanks for that. I have seen it in that case. I saw 2020 and thought wow, there is a new documentary, I'll watch it when I get a chance. Mods can you fix this?
- ZephyrBlu 5y ago
  The upload date is 2020.
- rdli 5y ago
  I originally watched this on Amazon Prime, but didn’t realize it was now for free on YouTube, which is why I submitted. I suppose I could have put “AlphaGo Documentary on YouTube” or some such ... (original poster)
  - TchoBeer 5y ago
    I could've sworn I saw it for free on YouTube in 2018 or so. Am I just remembering wrong?
    - Kiro 5y ago
      I also saw it for free back in the day.
  - Kiro 5y ago
    Your original submission and title were fine. It's the addition of (2020) I dislike which I presume was added by the mods. https://hackernewstitles.netlify.app/ https://hackernewstitles.netlify.app/
29athrowaway 5y ago
The best way to learn the rules of the game, as well as basic skills, is an app called badukpop. https://badukpop.com/ https://badukpop.com/ Once you've learned enough you can try playing via OGS, http://online-go.com http://online-go.com Don't expect to win right away.
sytelus 5y ago
More recent but not full length: AlphaFold https://www.youtube.com/watch?v=gg7WjuFs8F4&t https://www.youtube.com/watch?v=gg7WjuFs8F4&t
punnerud 5y ago
Collective > individual. Do I need to say more?
mypastself 5y ago
Thanks for this, an enjoyable documentary I remember seeing on Netflix a few years ago but never getting around to it. I understand it was made with a broader audience than HN in mind, but I wish they extended the runtime to cover the technical aspects in greater depth. As it is, it’s a quality sports movie, although I do find it amusing that in a match between a self-made young man and a cutting-edge piece of technology developed by one of world’s most powerful companies, the latter was initially presented as the underdog.
yesenadam 5y ago
Lee Sedol was rated #4 at the start of 2016, the year the match was played. 3 years later, aged 36, > Lee announced his retirement from professional play, stating that he could never be the top overall player of Go due to the increasing dominance of AI. Lee referred to them as being "an entity that cannot be defeated" https://en.wikipedia.org/wiki/Lee_Sedol https://en.wikipedia.org/wiki/Lee_Sedol His rating seems to have crashed and burned from soon after the match in 2016 until his retirement in 2019.. https://www.goratings.org/en/history/ https://www.goratings.org/en/history/ The #1 rated player is now Shin Jin-seo. > In January 2019, Shin was defeated by South Korean Go program HanDol. The program defeated the top five South Korean go players. HanDol has been compared to AlphaGo, but is considered to be weaker. https://en.wikipedia.org/wiki/Shin_Jin-seo https://en.wikipedia.org/wiki/Shin_Jin-seo

249 points by rdli 5y ago

notanog 5y ago

Its starwars for AI starters.

DantesKite 5y ago

It really is a brilliant documentary, even if you have no interest in artificial intelligence or Go.

vavooom 5y ago

This documentary really helped me understand and appreciate Go way more. Started my new found love of the game. Wish more people in the US played!

therein 5y ago
There are quite a lot of us actually. See you on online-go.com.

plasma 5y ago

This is a great watch, since then there has been AlphaGo Zero which surpassed the AI you see play Lee, in 3 days. https://deepmind.com/blog/article/alphago-zero-starting-scratch https://deepmind.com/blog/article/alphago-zero-starting-scra...

some_cut 5y ago
There has also been AlphaZero [1], a generalized version of AlphaGo which also has been trained to play Chess and Shogi (all learning from only the rules), and MuZero [2], which is a further generalization which can also play Atari games and does not even use the rules of the game when doing tree search - it has to learn a model of the rules instead. [1] https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go https://deepmind.com/blog/article/alphazero-shedding-new-lig... [2] https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules https://deepmind.com/blog/article/muzero-mastering-go-chess-...

platz 5y ago

feel free to add/play me https://online-go.com/player/1021081/ https://online-go.com/player/1021081/ ~8k

platz 5y ago

Michael Redmond's Go TV (was U.S. commentator for the official alphago games) https://www.youtube.com/channel/UCRJyagla1B5cxIfR4i2LdgA/videos?sort=p https://www.youtube.com/channel/UCRJyagla1B5cxIfR4i2LdgA/vid... He has some nice playlists: - how to play go (for beginners) https://www.youtube.com/watch?v=KTWujSwL2bQ&list=PLW5_cMTm0wvamCNX7qNoUqbXxeHt9n67i https://www.youtube.com/watch?v=KTWujSwL2bQ&list=PLW5_cMTm0w... - basic openings (joseki) https://www.youtube.com/playlist?list=PLW5_cMTm0wvZOTchMWZagFDHuUX0cEk4I https://www.youtube.com/playlist?list=PLW5_cMTm0wvZOTchMWZag... - joseki made popular by AlphaGo/AIs https://www.youtube.com/playlist?list=PLW5_cMTm0wvZU5pQhmQFwXh-ojU1mQIg3 https://www.youtube.com/playlist?list=PLW5_cMTm0wvZU5pQhmQFw... also of interest, Go Pro Yeonwoo (korean go professional) https://www.youtube.com/user/goingceo/videos https://www.youtube.com/user/goingceo/videos

spindle 5y ago
I'm so pleased to see this upvoted. Michael Redmond is unspeakably wonderful. He's a top professional player, is fantastic at explaining everything to a kyu player, and prepares his videos really dilligently.
- falcrist 5y ago
  Isn't he also the only American to reach the professional grade of 9-Dan?
yesenadam 5y ago
Ahh I wish I'd watched the first few in that beginners playlist before watching the movie! Thanks for that.

sillysaurusx 5y ago

Highly recommend the Deep Blue documentary too. https://www.youtube.com/watch?v=HwF229U2ba8&ab_channel=FredrikKnudsen https://www.youtube.com/watch?v=HwF229U2ba8&ab_channel=Fredr... It's easy to forget that none of this was guaranteed to work. Nowadays it feels inevitable that Chess and Go would fall to computers, but in the moment it was quite a different experience.

mNovak 5y ago

This film in large part inspired me to start playing Go

tkgally 5y ago
It's a great film and a great game. I learned how to play about forty years ago, when I was studying math in graduate school in the United States. I liked go partly because I felt refreshed after playing it, whether I won or lost, while chess had given me headaches. I moved to Japan a few years later, and for a while I played fairly regularly at go clubs in Kabukicho and Takadanobaba in Tokyo. The shot early in the film of some old guys playing go reminded me of those places. They were seedy and smoky, but you could get a game any time of the day or night. Many of the older players seemed practically to live there. Once I got matched against someone who I found out later was a highly ranked professional. At first he seemed interested in playing against a foreigner, but he started looking bored after about my fifth move. I stopped playing after my kids were born, and I haven't sat at a board—or played a game against a human online—in thirty years. But now that I’m approaching retirement myself, and I have a grandchild who will soon be old enough to learn, I’m thinking that maybe I should take it up again.

dstala 5y ago

Narration, BGM - all good. Even someone with absolute no interest in AI will get goosebumps. (Man) vs (Man with Machine)

esturk 5y ago

I'm curious to ask what's next for this series of AI at Deepmind? Is there other more challenging problems they are tackling at the moment? I read they have already master StarCraft 2 even. Is it the case that they have stopped this series of AI and going all in on protein folding at the moment?

d13 5y ago
I’d like to see AI take on a really difficult game like Root.
shmageggy 5y ago
This blog post mentions testing on Atari and being applied to chemistry and quantum physics https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules https://deepmind.com/blog/article/muzero-mastering-go-chess-...
- westurner 5y ago
  AlphaFold 2 solved the CASP protein folding problem that AFAIU e.g. Folding@home et. al have been churning at for awhile FWIU. From November 2020: https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology https://deepmind.com/blog/article/alphafold-a-solution-to-a-... https://en.wikipedia.org/wiki/AlphaFold#SARS-CoV-2 https://en.wikipedia.org/wiki/AlphaFold#SARS-CoV-2 : > AlphaFold has been used to a predict structures of proteins of SARS-CoV-2, the causative agent of COVID-19 [...] The team acknowledged that though these protein structures might not be the subject of ongoing therapeutical research efforts, they will add to the community's understanding of the SARS-CoV-2 virus.[74] Specifically, AlphaFold 2's prediction of the structure of the ORF3a protein was very similar to the structure determined by researchers at University of California, Berkeley using cryo-electron microscopy. This specific protein is believed to assist the virus in breaking out of the host cell once it replicates. This protein is also believed to play a role in triggering the inflammatory response to the infection (... Berkeley ALS and SLAC beamlines ... S309 & Sotrovimab: https://scitechdaily.com/inescapable-covid-19-antibody-discovery-neutralizes-all-known-sars-cov-2-strains/ https://scitechdaily.com/inescapable-covid-19-antibody-disco... ) Is there yet an open implementation of AlphaFold 2? edit: https://github.com/search?q=alphafold https://github.com/search?q=alphafold ... https://github.com/deepmind/alphafold https://github.com/deepmind/alphafold How do I reframe this problem in terms of fundamental algorithmic complexity classes (and thus the Quantum Algorithm Zoo thing that might optimize the currently fundamentally algorithmically computationally hard part of the hot loop that is the cost driver in this implementation)? To cite in full from the MuZero blog post from December 2020: https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules https://deepmind.com/blog/article/muzero-mastering-go-chess-... : > Researchers have tried to tackle this major challenge in AI by using two main approaches: lookahead search or model-based planning. > Systems that use lookahead search, such as AlphaZero, have achieved remarkable success in classic games such as checkers, chess and poker, but rely on being given knowledge of their environment’s dynamics, such as the rules of the game or an accurate simulator. This makes it difficult to apply them to messy real world problems, which are typically complex and hard to distill into simple rules. > Model-based systems aim to address this issue by learning an accurate model of an environment’s dynamics, and then using it to plan. However, the complexity of modelling every aspect of an environment has meant these algorithms are unable to compete in visually rich domains, such as Atari. Until now, the best results on Atari are from model-free systems, such as DQN, R2D2 and Agent57. As the name suggests, model-free algorithms do not use a learned model and instead estimate what is the best action to take next. > MuZero uses a different approach to overcome the limitations of previous approaches. Instead of trying to model the entire environment, MuZero just models aspects that are important to the agent’s decision-making process. After all, knowing an umbrella will keep you dry is more useful to know than modelling the pattern of raindrops in the air. > Specifically, MuZero models three elements of the environment that are critical to planning: > * The value: how good is the current position? > * The policy: which action is the best to take? > * The reward: how good was the last action? > These are all learned using a deep neural network and are all that is needed for MuZero to understand what happens when it takes a certain action and to plan accordingly. > Illustration of how Monte Carlo Tree Search can be used to plan with the MuZero neural networks. Starting at the current position in the game (schematic Go board at the top of the animation), MuZero uses the representation function (h) to map from the observation to an embedding used by the neural network (s0). Using the dynamics function (g) and the prediction function (f), MuZero can then consider possible future sequences of actions (a), and choose the best action. > MuZero uses the experience it collects when interacting with the environment to train its neural network. This experience includes both observations and rewards from the environment, as well as the results of searches performed when deciding on the best action. > During training, the model is unrolled alongside the collected experience, at each step predicting the previously saved information: the value function v predicts the sum of observed rewards (u), the policy estimate (p) predicts the previous search outcome (π), the reward estimate r predicts the last observed reward (u). This approach comes with another major benefit: MuZero can repeatedly use its learned model to improve its planning, rather than collecting new data from the environment. For example, in tests on the Atari suite, this variant - known as MuZero Reanalyze - used the learned model 90% of the time to re-plan what should have been done in past episodes. FWIU, from what's going on over there: AlphaGo => AlphaGo {Fan, Lee, Master, Zero} => AlphaGoZero => AlphaZero => MuZero AlphaGo: https://en.wikipedia.org/wiki/AlphaGo_Zero https://en.wikipedia.org/wiki/AlphaGo_Zero AlphaZero: https://en.wikipedia.org/wiki/AlphaZero https://en.wikipedia.org/wiki/AlphaZero MuZero: https://en.wikipedia.org/wiki/MuZero https://en.wikipedia.org/wiki/MuZero AlphaFold {1,2}: https://en.wikipedia.org/wiki/AlphaFold https://en.wikipedia.org/wiki/AlphaFold IIRC, there is not an official implementation of e.g. AlphaZero or MuZero with e.g. openai/gym (and openai/retro) for comparing reinforcement learning algorithms? https://github.com/openai/gym https://github.com/openai/gym What are the benchmarks for Applied RL? From https://news.ycombinator.com/item?id=28499001 https://news.ycombinator.com/item?id=28499001 : > AFAIU, while there are DLTs that cost CPU, RAM, and Data storage between points in spacetime, none yet incentivize energy efficiency by varying costs depending upon whether the instructions execute on a FPGA, ASIC, CPU, GPU, TPU, or QPU? [...] > To be 200% green - to put a 200% green footer with search-discoverable RDFa on your site - I think you need PPAs and all directly sourced clean energy. > (Energy efficiency is very relevant to ML/AI/AGI, because while it may be the case that the dumb universal function approximator will eventually find a better solution, "just leave it on all night/month/K12+postdoc" in parallel is a very expensive proposition with no apparent oracle; and then to ethically filter solutions still costs at least one human)

CalChris 5y ago

I don't play go but the AlphaZero chess games are quite beautiful. Agadmator has a description of the weird unhuman like machine continuations, disgusting engine lines. They're correct but incomputable by a human and just look weird. AlphaZero had some beautiful lines that looked human but slightly counterintuitive. They were the sort of moves that humans could learn from. Are the AlphaGo games similar?

mattbillenstein 5y ago
Agreed, I think ChessNetwork analyzed several of the published games as well - and some of the Leela games - they're so much more interesting than most of what I see out of Stockfish...
vmilner 5y ago
Yes, very much so - for example it used to be almost axiomatic (told in your second or third go lesson) that placing a white stone at the 3-3 point when black had played at 4-4 was very bad, but this video shows that AlphaGo has radically changed this view: https://www.youtube.com/watch?v=2khNnE5Q3GM https://www.youtube.com/watch?v=2khNnE5Q3GM
AboveTheGame 5y ago
///UNPOPULAR OPINION WARNING/// No, they’re for inept idiots who are obsessed with mathematically solved abstractions of reality. My god, to someone who sees chaos maps in reality the patterns these pathetic programming abstractions offer is a limitation of /our current society/ — we are slaves to wealth and process over progress and intuition. Nothing more need be said, I am always throwing Pearls before swine.
AboveTheGame 5y ago
You are also NOT “that guy” Fuck this gay simulation lmao Hahahahaha get pressed on y’all soft ass morons
roenxi 5y ago
Go is a bit different from Chess, there are so many options that you can't really have a "disgusting engine line", because either sequences are forced and humans can find them too eventually or the number of lines the move effects is so vast that we would never realise how clever it is. Computer Go is obviously making calculations that a human couldn't, but as an observer it just looks like not making any mistakes. There is the occasional spectacular attack or defence, but humans do that too from time to time - human players often see or try moves that others wouldn't. In complicated situations computers will do a better job at assessing what the impacts are. In Go, to win a player needs to consistently make moves that are on average 0.0025 points better than their opponent for 300 moves. It is hard to detect the genius if a neural net decides it is going to win the game that way. The games are interesting to study, they are all masterworks. But the strength of a Go player is in the consistency rather than individual flashes of insight.

bumbledraven 5y ago

(2020)

Kiro 5y ago
The movie is from 2017.

bumbledraven 5y ago

"7% Documentary: Behind the scenes of Fine Art AI - 纪录片《7%》：揭秘人工智能“绝艺”夺冠幕后腾讯网 - English subtitles" (https://v.qq.com/x/page/r0025m06t5o.html https://v.qq.com/x/page/r0025m06t5o.html) is a neat 2018 documentary about Fine Art, a world-class Go AI created by Chinese developers at Tencent. The documentary features extensive commentary by the main programmers, Ma Bo and Tang Shanmin, as well as the project lead Liu Yongsheng. This bit from 4:02 stuck out to me: INTERVIEWER: What level do you think Fine Art has reached? MA BO: About the same level AlphaGo had when playing against Li Shiqi last year. INTERVIEWER: Then wouldn't you regard what you do as a redundant work? MA BO: How should I say this? Its like, when China made the atom bomb after America. Was that redundant work?

DSingularity 5y ago
The Chinese have a level of nationalism that many nations lack.
- duttaditya18 5y ago
  That is a good thing.
  - pathseeker 5y ago
    no it's not "identification with one's own nation and support for its interests, especially to the exclusion or detriment of the interests of other nations."
  - worrycue 5y ago
    I feel it just artificially and pointlessly divides us and create needless "us vs them" situations. Nationalism has interesting "properties". It carries with it the implicit assumption that people miles away in the same nation that you have never met share some of the same beliefs and values you do. It somehow justify you taking credit for the accomplishments of others in the same nation even though you haven't done anything to contribute to said accomplishment.

manigandham 5y ago

One of the best films I've ever seen. Brilliant storytelling about man vs machine at the last frontier of our minds.

Kiro 5y ago

Why did the mods change the title and put an incorrect year in it? The movie is from 2017.

throwaway81523 5y ago
Thanks for that. I have seen it in that case. I saw 2020 and thought wow, there is a new documentary, I'll watch it when I get a chance. Mods can you fix this?
ZephyrBlu 5y ago
The upload date is 2020.
rdli 5y ago
I originally watched this on Amazon Prime, but didn’t realize it was now for free on YouTube, which is why I submitted. I suppose I could have put “AlphaGo Documentary on YouTube” or some such ... (original poster)
- TchoBeer 5y ago
  I could've sworn I saw it for free on YouTube in 2018 or so. Am I just remembering wrong?
  - Kiro 5y ago
    I also saw it for free back in the day.
- Kiro 5y ago
  Your original submission and title were fine. It's the addition of (2020) I dislike which I presume was added by the mods. https://hackernewstitles.netlify.app/ https://hackernewstitles.netlify.app/

29athrowaway 5y ago

The best way to learn the rules of the game, as well as basic skills, is an app called badukpop. https://badukpop.com/ https://badukpop.com/ Once you've learned enough you can try playing via OGS, http://online-go.com http://online-go.com Don't expect to win right away.

sytelus 5y ago

More recent but not full length: AlphaFold https://www.youtube.com/watch?v=gg7WjuFs8F4&t https://www.youtube.com/watch?v=gg7WjuFs8F4&t

punnerud 5y ago

Collective > individual. Do I need to say more?

mypastself 5y ago

Thanks for this, an enjoyable documentary I remember seeing on Netflix a few years ago but never getting around to it. I understand it was made with a broader audience than HN in mind, but I wish they extended the runtime to cover the technical aspects in greater depth. As it is, it’s a quality sports movie, although I do find it amusing that in a match between a self-made young man and a cutting-edge piece of technology developed by one of world’s most powerful companies, the latter was initially presented as the underdog.

yesenadam 5y ago

Lee Sedol was rated #4 at the start of 2016, the year the match was played. 3 years later, aged 36, > Lee announced his retirement from professional play, stating that he could never be the top overall player of Go due to the increasing dominance of AI. Lee referred to them as being "an entity that cannot be defeated" https://en.wikipedia.org/wiki/Lee_Sedol https://en.wikipedia.org/wiki/Lee_Sedol His rating seems to have crashed and burned from soon after the match in 2016 until his retirement in 2019.. https://www.goratings.org/en/history/ https://www.goratings.org/en/history/ The #1 rated player is now Shin Jin-seo. > In January 2019, Shin was defeated by South Korean Go program HanDol. The program defeated the top five South Korean go players. HanDol has been compared to AlphaGo, but is considered to be weaker. https://en.wikipedia.org/wiki/Shin_Jin-seo https://en.wikipedia.org/wiki/Shin_Jin-seo