3 ms·
From what I remember, larger LLMs like PaLM don't use biases for training stability, but smaller ones tend to still use them.
by tripplyons 2y ago
From what I remember, larger LLMs like PaLM don't use biases for training stability, but smaller ones tend to still use them.