3 ms·
There was some paper about routing at training bio-knowledge into a particular region of the model, which you then can cutoff when serving. But you probably los
by dist-epoch 2mo ago
There was some paper about routing at training bio-knowledge into a particular region of the model, which you then can cutoff when serving. But you probably lose some efficiency since maybe you sized that region too small/too big.
- mapontosevenths 2mo agoThat's a very clever approach. Any idea about the papers title or authors? I'd love to look it up.
- dist-epoch 2mo agohttps://arxiv.org/abs/2410.04332 https://arxiv.org/abs/2410.04332 https://alignment.anthropic.com/2025/selective-gradient-masking/ https://alignment.anthropic.com/2025/selective-gradient-mask...