Great question! The precision is not just better than the wavelength of the light. It's also way smaller than the surface roughness of the mirrors! How does it

Great question! The precision is not just better than the wavelength of the light. It's also way smaller than the surface roughness of the mirrors! How does it work?! Like you suggest, and adding to what sleavey mentioned above, I would say the answer is: averaging over time and space. The laser beam is pretty wide, so it averages over a significant area of mirror surface. (The optical system also selects one spatial mode of the laser beam.) And the stated displacement sensitivity ("1/10000 the width of a proton") only occurs when you integrate over the sensitive frequency band.