3 ms·

So softmax is e^x projection followed by l1 norm. Why is e^x projection useful?

• aesthesia 6mo ago
  It maps (-inf, inf) to (0, inf) in about as nice a way as you could expect (addition turns into multiplication). When you want to constrain a value to be positive, parameterizing it with exp is usually a good option.
  • ainch 6mo ago
    And importantly it's got nice properties like being differentiable and monotonic, unlike eg. taking |x|.