3 ms·In reality it’s probably not a RELU modern LLMs use GeLU or something more advanced.by andy_ppp 1y agoIn reality it’s probably not a RELU modern LLMs use GeLU or something more advanced.