5 ms·
Connections between LLM neurons also change during training.
by kadushka 11mo ago
Connections between LLM neurons also change during training.
- advisedwang 11mo agoa) "during training" is a huuuuge asterisk b) Do you have a citation for that? my understanding is that while some weights can go to zero and effectively be removed, no (actually used in prod) network architecture or training method allows arbitrary connections.