Sorry, I was pretty hand-wavy with that because it was basically just trial and error until it worked sufficiently well. The data I had available to mess with

Sorry, I was pretty hand-wavy with that because it was basically just trial and error until it worked sufficiently well. The data I had available to mess with was the difference in width of the top of the puzzle and the bottom (with some trig you can determine its angle relative to the camera) and the projection matrix of the camera relative to the scene origin. It's not perfect but it works better than having nothing at all.