3 ms·
> l (feed forward network) won't do great. See my answer below, in the case of this problem a generic feed-forward network, even a simple one, will work. Not
by george3d6 7y ago
> l (feed forward network) won't do great.
See my answer below, in the case of this problem a generic feed-forward network, even a simple one, will work.
Not any ffn, but assuming you are using an efficient architecture search it will probably find one that works.
There's other numerical problems where this doesn't hold but that's another story.
- 6gvONxR4sf7o 7y agoYou demonstrated it for a reeeeeeeallly constrained version of the problem. Do you expect your solution would generalize to many lists? Because it would be easy to make a neural network that does, while your toy example (and larger generalizations) probably won't generalize super well. x_i = ith list element from list x y = sum(x_i * softmax(k * x)_i) This one parameter, arbitrarily wide network one will get arbitrarily close to the max function. This is a super toy version of why attention is so effective. It can pick stuff.