3 ms·

>GPT will have an S curve or Sigmoid function How can you tell the difference between "we're in the beginning of the curve" vs "we're at the end of it" vs "the

• gavinhoward 4y ago
  Several ways. First, diminishing returns. When people start talking about exponential amounts of resources required (data or compute) or the results are not growing as fast, you're approaching the top. Second, historical precedent. Self-driving cars have been around the corner for years now. I don't expect them anytime soon. Third, practical limits. These LLM's are trained on human text. Therefore, their output will only approach the best humans can do in the limit and in the best case. In the average case, they will only have average output, and their output will be the average of everything, making it flavorless. This is also how you know it's an S curve and not a hockey stick: it cannot be any better than the training data. Fourth, limits of the model. These LLM's predict the next sequences of words (or in the case of picture bots, they predict a sequence of pixels). That means they can only do things that have those inputs and require or accept those outputs. You're not going to get an LLM to drive a car, and you're not going to get a self-driving car to output a sequence of text. You may add an LLM to a self-driving car, but it cannot communicate with the driving code because they have mismatched inputs and outputs.
  • slig 4y ago
    Thank you!