2 ms·
This is the assumption of a flat and smooth latent manifold. If either of those conditions are not met, then seeds can have a wild effect on the model output. T
by godelski 3y ago
This is the assumption of a flat and smooth latent manifold. If either of those conditions are not met, then seeds can have a wild effect on the model output. There's only bounds on the latent structure being locally smooth or locally flat btw. This is the underpinning of such works as the Lottery Hypothesis but there's a whole rabbit hole to dig down here and it is why Alpha Go did so poorly in round 2. This is an important part of the discussion of generalizability btw. I do not think your analogy works out and it is inconsistent with your prior comment, because you're right that anyone asking GPT is akin to anyone asking the same doctor. The second doctor is a second model (or potentially a second GPT checkpoint depending on how we want to consider divergence).