44 ms·
What seems clever to me is the super cool hack of getting sensor data while the video data is captured, then training a model on realtime video and backward tim
by vessenes 4y ago
What seems clever to me is the super cool hack of getting sensor data while the video data is captured, then training a model on realtime video and backward time-shifted sensor data; result is a network that can predict sensor data 30 seconds ahead of time with visual data. This is then used to feed the walking model. That's the sort of super 'dumb' thing ML folks can do now; that task would have taken many many person years of work before, now, I speculate thinking of the idea to testing a network would be under a week. Verry cool.
- fwlr 4y agoI often think of this as the corollary to the bitter lesson: it is enough just to break some particular relationship between two things, and then throw compute at it. In the process of trying to recover that relationship, the model learns about the underlying real relationship - sometimes it even learns about each thing itself.
- stormfather 4y ago> training a model on realtime video and backward time-shifted sensor data Do you mean forward time shifted sensor data? I.e. using sensor data 30 seconds from the future as a label, and real time video as input?
- vessenes 4y agoI think we're saying the same thing. Training pair at t0 is video_t0, sensor_t30. So we have shifted sensor data back 30 seconds, or put your way, we look to sensor data 30 seconds in the future.
- stormfather 4y agoOh yeah you're right, thank you. I think the way you phrased it makes the most sense now.