Yes. The problem with working backwards is that some optical calculations have probability elements. A photon that hits a half-silvered mirror has a 50% chanc

Yes. The problem with working backwards is that some optical calculations have probability elements. A photon that hits a half-silvered mirror has a 50% chance of (specular) reflecting and a 50% chance of transmitting. So for ray-tracing, you calculate along both paths and give 50% weight to each. Every time a ray hits a triangle in the scene, the material properties determine how the various components sum up to determine the color of the pixel.