This lead me to the question of why hasn't there been an effort to do this with 3D content (that I know of). Because camera angles/lighting/collision detection

This lead me to the question of why hasn't there been an effort to do this with 3D content (that I know of). Because camera angles/lighting/collision detection/etc. at that point would be almost trivial. I guess with the "2D only" approach that is based on actual, acquired video you get way more impressive shots. But the obvious application is for games. Content generation in the form of modeling and animation is actually one the biggest cost centers for most studios these days.