3 ms·

Probably a lot considering people are pouring practically unlimited resources into their development and scaling?

• WJW 3y ago
  This in itself proves nothing. Consider how much resources have been poured in making rockets better over the last few decades and yet we still can barely make it to Mars. It is also quite possible that there are fundamental limits to scaling LLMs that we have not discovered yet. You won't find a way to let airplanes fly to the moon no matter how many resources you pour into it. To make it there, you need better methods (ie rockets in this case) that look nothing like the first few steps to get off the ground.
  • robryan 3y ago
    LLMs fairly quickly got to consuming a significant portion of all available data. I can't see them improving significantly just by throwing more data at the problem.