You would use one of the Generalized Extreme Value distributions. One of those, the Gumbel distribution, is actually commonly used in machine learning to sample

You would use one of the Generalized Extreme Value distributions. One of those, the Gumbel distribution, is actually commonly used in machine learning to sample from a softmax in a differentiable manner (by reparametrization), and should work for the maximum of normals (it's definitely one of the three). You may be interested in this theorem : https://en.m.wikipedia.org/wiki/Fisher%E2%80%93Tippett%E2%80%93Gnedenko_theorem https://en.m.wikipedia.org/wiki/Fisher%E2%80%93Tippett%E2%80...