3 ms·
> which is nonsense: the gate simply, directly, selects the experts. There's nothing probabilistic about it. Isn't G a learned probability?
by didibus 2mo ago
> which is nonsense: the gate simply, directly, selects the experts. There's nothing probabilistic about it.
Isn't G a learned probability?
- wren6991 2mo agoI don't think so. Probability of what? If you put something through a softmax the output is (trivially) a valid PMF. Does that matter? You're not sampling from it.