2 ms·
A differentiable renderer can efficiently answer the question "If I tweaked this input parameter (e.g. camera position, object color, light direction) then whic
by modeless 8y ago
A differentiable renderer can efficiently answer the question "If I tweaked this input parameter (e.g. camera position, object color, light direction) then which output pixels on screen would change, and by how much?" With this information you can run an optimization algorithm to repeatedly tweak the input parameters in the right direction to make the output pixels look more like a target that you want, e.g. a photograph.
Even better, using the chain rule you can connect the differentiable renderer to other differentiable things, like a neural network controller for a simulated robot. Then you can ask e.g. "If I tweaked the robot's arm position, which camera pixels would change?" This would allow you to, given an image of a target configuration (e.g. two stacked blocks), optimize the neural network to produce that image by controlling the robot (e.g. by stacking the blocks).