7 ms·
Seeing the other comments, it still leaves me the question if this is a limit of scale or a limit of technology? How much more can LLMs scale up over the next d
by km3r 3y ago
Seeing the other comments, it still leaves me the question if this is a limit of scale or a limit of technology? How much more can LLMs scale up over the next decade?
- ChatGTP 3y agoProbably a lot considering people are pouring practically unlimited resources into their development and scaling?
- WJW 3y agoThis in itself proves nothing. Consider how much resources have been poured in making rockets better over the last few decades and yet we still can barely make it to Mars. It is also quite possible that there are fundamental limits to scaling LLMs that we have not discovered yet. You won't find a way to let airplanes fly to the moon no matter how many resources you pour into it. To make it there, you need better methods (ie rockets in this case) that look nothing like the first few steps to get off the ground.
- robryan 3y agoLLMs fairly quickly got to consuming a significant portion of all available data. I can't see them improving significantly just by throwing more data at the problem.