This could be worked around by making the LEDs more directional and taking input from an array of directional cameras on the other side (or presumably, a single

This could be worked around by making the LEDs more directional and taking input from an array of directional cameras on the other side (or presumably, a single Lytro camera per LED). I don't believe there's a fundamental difficulty in amelioration the problem of parallax.