I think the difficulty in using virtual environments for training purposes isn't in simulating the environment, but accurately simulating the physical responses

I think the difficulty in using virtual environments for training purposes isn't in simulating the environment, but accurately simulating the physical responses/limitations of the robotic hardware in that environment in a way that would reflect real engineered hardware (ie motor responses, signal latency, etc).