3 ms·

So there is an "attractor" that different initializations end up converging on?

• andy99 1y ago
  Different initialization converge to different places, e.g https://arxiv.org/abs/1912.02757 https://arxiv.org/abs/1912.02757 For LLMs (as with other models), many local optima appear to support roughly the same behavior. This is the idea of the problem being under-specified ie many more equations than unknowns so there are many ways to get the same result.