2 ms·
> 1. No biases. Following LLaMA, PaLM, and others, we exclude all bias terms from our architecture in order to improve training stability. What does this mean?
by margorczynski 3y ago
> 1. No biases. Following LLaMA, PaLM, and others, we exclude all bias terms from our
architecture in order to improve training stability.
What does this mean? What is a "bias term"?
- polygamous_bat 3y agoThink of the term b in y = Wx+b. W is called weight, b is called bias.