As user 4ad explains, one of the reasons is that phase information isn't collected by the sensors, and is needed to reconstruct all the image detail. In radio

As user 4ad explains, one of the reasons is that phase information isn't collected by the sensors, and is needed to reconstruct all the image detail. In radio astronomy the phase information is actually collected and sometimes recorded, which is why you can have arrays of radio telescopes far apart that combine. The most extreme version of this is Very Long Baseline Interferometry (https://en.wikipedia.org/wiki/Very-long-baseline_interferometry https://en.wikipedia.org/wiki/Very-long-baseline_interferome...) which was used to image the black hole at the centre of our galaxy (https://en.wikipedia.org/wiki/Event_Horizon_Telescope https://en.wikipedia.org/wiki/Event_Horizon_Telescope). At infra-red and visible wavelengths, it is technically possible to collect some phase information, subject to noise though. So in principle it is possible to collect images including some phase at each mirror location instead of using a mirror, and then stitch them in a similar way to how it's done with radio. However, collecting the phase would be difficult and complex, especially with current technology, and likely to degrade the image so much that it's not worth doing anyway. Using mirrors is better. In future, it is plausible that this will be done to combine images from optical telescopes far apart in space, for a very wide aparture. But it seems just as likely that they will use mirrors far apart in space, directing the incoming light to a small number of focal locations to combine the light in the optical domain first, before converting it to image data.