You can. It's just very costly. Rays typically represent just a single sample, so the more probabilistic the projected image becomes (read "unfocused"), the mor

You can. It's just very costly. Rays typically represent just a single sample, so the more probabilistic the projected image becomes (read "unfocused"), the more samples are required to capture it smoothly. This is compounded by the number of lens elements, the material's index of refraction and other complexities. A single element lens adds a minimum of two additional ray/surface evaluations for every sample of which their are many per pixel. Also, as a separate matter many implementations optimize around the assumption that the projection from three dimensions to two is done in a uniform, undistorted space. For example the kind of projection that can be fully described by a 4x4 matrix. Changing that assumption loses those optimizations, and generally leads to substance abuse and mayhem.