2 ms·Almost no one uses ReLU anymore. It's usually GeLU for llms, and Swish for images.by p1esk 4y agoAlmost no one uses ReLU anymore. It's usually GeLU for llms, and Swish for images.