There is nearly an unlimited amount of data. For example, feed real world video/sensory data into LLMs and they can learn how real world physical objects behave

There is nearly an unlimited amount of data. For example, feed real world video/sensory data into LLMs and they can learn how real world physical objects behave. Simulate physics with a physics engine and then feed that data into the LLM. The internet represented easy/low hanging fruit data. It doesn't seem like the transformer algorithm is limited to pure text training data.