5 ms·

Wei et al 2022 might be a good starting point: https://gwern.net/doc/ai/scaling/emergence/index#wei-et-al-2022-section https://gwern.net/doc/ai/scaling/emergenc

• nologic01 3y ago
  Good reference. The sparsity of the data points (presumably due to the huge training costs) doesn't reveal too much at present but as the authors suggest the phenomenon might be reproduced in smaller models (which would make it easier to study in greater detail).