3 ms·

It's well past time to speak of emergent capabilities. That idea was debunked by this paper. It was given a Best Paper award and was one of the few oral present

• atleastoptimal 2y ago
  One paper attempting one hypothesis against one model doesn't completely debunk emergent capabilities. If you've seen one of the thousands of novel capabilities of GPT-4 level models you'd realize this.
• pizza 2y ago
  Eh, I don't think that's their position: > nothing in this paper should be interpreted as claiming that large language models cannot display emergent abilities; rather, our message is that some previously claimed emergent abilities appear to be mirages induced by researcher analyses imo their paper's position is - downstream task metric jumps in accuracy don't imply sudden changes in pretrain loss - you can cherry pick metrics to show the sudden appearance of emergent abilities - model scale is not enough to explain emergent abilities because you can scale up one model family and not get them Their follow-up paper [0] has an appendix that touches upon how: - it's not impossible for continuous metrics to undergo sudden increases of performance - therefore not all sudden increases of performance are a consequence of using discontinuous metrics - therefore emergent abilities could still actually exist [0] Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive? https://arxiv.org/abs/2406.04391 https://arxiv.org/abs/2406.04391