44 ms·
>> Because priors are hard to embed in a model. That's a limitation of neural networks, where inductive biases are very difficult to represent. Other approache
by YeGoblynQueenne 6y ago
>> Because priors are hard to embed in a model.
That's a limitation of neural networks, where inductive biases are very difficult to represent. Other approaches don't have any such problem, e.g. all the other approaches I listed above have well-defined, clean and tidy representations for inductive bias.
As to AlphaGo, this may come as a shock, but AlphaGo was not the first system to surpass humans in anything. It was the first system to surpass humans _in Go_ but e.g. the first computer system to beat a human grandmaster in chess was Deep Blue [1], in 1997, the first computer system to win a world championship against human players was the checkers (draughts) player Chinook [2], in 1990, the first computer system to outperform humans in medical diagnosis was MYCIN in the 1970's and so on. All those were systems that used strong inductive biases.
And of course, AlphaGo itself was limited by human priors- e.g. piece moves, checkerboard dimensions and structure were hard-coded into its architecture and a search for moves was performed by MCTS.
In any case, the lack of good enough priors is not a reason to not use any priors- it's a reason to look for better priors.
Edit: Has any end-to-end approach rediscovered an entire state-of-the-art architecture, like CNNs or LSTMs?
___________________
[1] https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer) https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)
[2] https://en.wikipedia.org/wiki/Chinook_(computer_program) https://en.wikipedia.org/wiki/Chinook_(computer_program)
[3] https://en.wikipedia.org/wiki/Mycin https://en.wikipedia.org/wiki/Mycin