3 ms·
As the article says: " If you're a mathematician the argument is not difficult to follow, but it's not so easy for most people. That's a pity, since the underl
by gizmo686 11y ago
As the article says:
" If you're a mathematician the argument is not difficult to follow, but it's not so easy for most people. That's a pity, since the underlying reasons for universality are simple and beautiful."
Indeed, as a mathematician, the universality of neural networks is obvious to me from their definition. However, this article is explicitly not aimed at mathematicians, and (as far as I can tell) does a good job of presenting the argument without requiring unnecessary math knowledge, which is something that we tend to be bad at doing.
Furthermore, the universality does not directly follow from the quote you provide, as you would still need to show that the hidden layer neurons form a basis. Infact, the article was about constructing such a basis in the neurons. The only thing that bringing in your quote would serve to do is make the article more confusing by unnecessarily introducing the concept of basis functions.
- dicroce 11y agoAs a non mathematician, this is non obvious to me. Thinking about it a bit (I haven't finished reading the article yet)... Since the size of the hidden layer isn't specified, I suppose you could have a hidden layer node for every possible input... So, of course any function is computable with a neural network. Really the magical thing here is finding the smallest set of nodes that computes the function...
- 49531 11y agoI've been toying with a NN trying to get it to play 2048 based on game data I recorded. I still have about 60% error rate, but I found that with 16 inputs (the tiles on the game), and 4 outputs (directions to move), it works best like a funnel. I currently have 2 hidden layers, of 12 and 8 and it's the best I've gotten so far.
- gizmo686 11y agoThat is essentially correct. The subtlety is that the hidden layer still has a finite (but arbitrarily large) number of nodes, while there are an infinite number of inputs. The solution to this is that you can keep adding nodes until the space between the nodes is close enough to the function you are modelling.