3 ms·

I've done something similar in beginning of 2017: https://openreview.net/pdf?id=SyEiHNKxx https://openreview.net/pdf?id=SyEiHNKxx I had the whole physics engin

by 317070 8y ago

I've done something similar in beginning of 2017: https://openreview.net/pdf?id=SyEiHNKxx https://openreview.net/pdf?id=SyEiHNKxx I had the whole physics engine + renderer differentiable, in Theano. I used it to do control problems though (so using the image to infer which actions to take), not for system identification (i.e. given the image, what are the parameters of the scene). But yeah, I could backpropagate through the renderer too: https://twitter.com/317070/status/821062814798331905 https://twitter.com/317070/status/821062814798331905 I think more research should go in this direction, backpropagating these physical systems will empower a lot of powerful optimization methods. There is no reason why a physics engine should be treated as a black box. We know the equations and they are differentiable. I'm glad there will now be dedicated code for the gradient of a full blown renderer :-)

modeless 8y ago
I saw this work when you published it. It looks really impressive! Have you continued to work on it? Theano is no longer being developed, so I guess you'd have to switch to something else. What are the next directions to take for this approach?
- 317070 8y ago
  I tried to submit it to two conferences, twice rejected. Had to leave academia after funding ran out (did get my Phd). Moved the manuscript to a journal and just last week got a really bad review (saying this had been done in the nineties already) and it will probably be rejected again. All of this is not encouraging my old research group to continue in this direction. So, the paper and code will likely end up in the garbage bin of academia. And Theano quitting was also not exactly helping indeed. Next directions are exactly what these people do with their ray tracing. The equations of all of these things are rather simple and almost always differentiable. Someone just needs to code them up and start experimenting with what is achievable when you have these exact gradients. If someone is looking for a technical area for a Phd/startup: such a physics engine with a focus on optimization (so both system identification, system control and system optimization with gradient descent, making all internals of your engine differentiable) is worth millions.
  - modeless 8y ago
    Wow, I am sorry to hear that. I am more interested in differentiable physics than differentiable rendering; it seems promising for training controllers for animated game characters or even real-world robots with transfer learning. I might try my hand at it but I think there is a whole literature on rigid body dynamics that I would have to read up on first. I would be interested to hear if you have ideas for specific application areas that would be feasible to tackle as a startup. Animated game characters is the best I've got but I'm not sure it's worth millions.

by 317070 8y ago

modeless 8y ago

I saw this work when you published it. It looks really impressive! Have you continued to work on it? Theano is no longer being developed, so I guess you'd have to switch to something else. What are the next directions to take for this approach?

317070 8y ago
I tried to submit it to two conferences, twice rejected. Had to leave academia after funding ran out (did get my Phd). Moved the manuscript to a journal and just last week got a really bad review (saying this had been done in the nineties already) and it will probably be rejected again. All of this is not encouraging my old research group to continue in this direction. So, the paper and code will likely end up in the garbage bin of academia. And Theano quitting was also not exactly helping indeed. Next directions are exactly what these people do with their ray tracing. The equations of all of these things are rather simple and almost always differentiable. Someone just needs to code them up and start experimenting with what is achievable when you have these exact gradients. If someone is looking for a technical area for a Phd/startup: such a physics engine with a focus on optimization (so both system identification, system control and system optimization with gradient descent, making all internals of your engine differentiable) is worth millions.
- modeless 8y ago
  Wow, I am sorry to hear that. I am more interested in differentiable physics than differentiable rendering; it seems promising for training controllers for animated game characters or even real-world robots with transfer learning. I might try my hand at it but I think there is a whole literature on rigid body dynamics that I would have to read up on first. I would be interested to hear if you have ideas for specific application areas that would be feasible to tackle as a startup. Animated game characters is the best I've got but I'm not sure it's worth millions.