The scaling appears linear in relationship, though on some performance metrics it is super linear. See Emergent Abilities of Large Language Models https://arxi

The scaling appears linear in relationship, though on some performance metrics it is super linear. See Emergent Abilities of Large Language Models https://arxiv.org/abs/2206.07682 https://arxiv.org/abs/2206.07682 Scaling Laws for Generative Mixed-Modal Language Models https://arxiv.org/abs/2301.03728 https://arxiv.org/abs/2301.03728 All this demonstrates further the "bitter lesson" http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html that surprising ability emerges from just throwing more data and compute at the problem and is seen more fruitful an endeavor than trying to find a deeper/analytic solution to cognition (like Yann Lecun's research, who has basically become a meme now for how much he poopoos LLMs and insists what we're seeing are LLMs only pretending to reason).