3 ms·
The softmax value y is a linear combination of the vectors you're attending over: y = a1v1 + a2v2 + ... + an*vn where a_i >= 0 and sum(a_i) = 1. Then y is a co
by sdenton4 2y ago
The softmax value y is a linear combination of the vectors you're attending over:
y = a1v1 + a2v2 + ... + an*vn
where a_i >= 0 and sum(a_i) = 1.
Then y is a convex combination of the v_i, and sits in the convex hull of the v_i.