The position of the pen could be tracked via infrared with existing tech, but you'd also need to be able to track the position and angle of the glasses in order

The position of the pen could be tracked via infrared with existing tech, but you'd also need to be able to track the position and angle of the glasses in order to render the view correctly. Also, I doubt that 3 infrared stations would actually suffice, since your body would occlude at least one of the sensors in most positions. You'd probably need several, and they'd likely need to be placed on the ceiling and walls as well as the floor for full coverage. Not being able to draw on a hard surface would introduce a learning curve, but remember that people already accomplish some pretty intricate designs while never actually touching the surface - Graffiti, airbrushing, etc. I don't think that would actually be too big a stumbling block.