Mentioning the blind as evidence that there is no pre-training based on vision is broken in my opinion. It could very well be that humans do multimodal pretrai

Mentioning the blind as evidence that there is no pre-training based on vision is broken in my opinion. It could very well be that humans do multimodal pretraining (including vision), landing babies with a pre-trained brains that works even without vision. A machine implementation of this is pre-training a self-driving car on vision + lidar then doing real-time inference on vision only.