3 ms·
An important piece here is that there's still a training signal making it to the makes weights. See SimSiam for a similar example.
by sdenton4 3y ago
An important piece here is that there's still a training signal making it to the makes weights. See SimSiam for a similar example.
- lucidrains 3y agoindeed, simsiam is a great example of the effectiveness of using stop gradient