3 ms·
There isn't a curve at all. It's discrete steps. Which is why so many people freak out when there's a step change, because they extrapolate the existence of a c
by mjr00 4y ago
There isn't a curve at all. It's discrete steps. Which is why so many people freak out when there's a step change, because they extrapolate the existence of a curve where there isn't one.
- consumer451 4y agoCome on, a curve can be drawn over the steps. I'm a maths dumb dumb but.. https://en.wikipedia.org/wiki/Interpolation https://en.wikipedia.org/wiki/Interpolation Here is ChatGPT's response fwiw. Prompt: is the curve of improvement in LLMs linear or more exponential? ChatGPT: The improvement in language models has generally followed an exponential trend. With the advent of deep learning and the increasing availability of large amounts of data, the performance of language models has improved dramatically in recent years, leading to breakthroughs in various NLP tasks. However, it is important to note that the improvement curve of language models is influenced by multiple factors, including the amount of training data, the size and complexity of the model, and the computational resources available. The improvement curve of language models may also become more linear as the state of the art approaches certain limits.
- consumer451 4y agoGot caught by edit time limits, mid-edit. > The improvement curve of language models may also become more linear as the state of the art approaches certain limits. This part of ChatGPT's response is the crux of my original question. It might be notable that I was able to focus and possibly learn from the response to my question via interaction with an LLM, and not via interaction with a human.