I think the answer here is simple and not as profound as people might think. Square matrices are linear operators which map an input vector into a space with th

I think the answer here is simple and not as profound as people might think. Square matrices are linear operators which map an input vector into a space with the same dimensionality. We can characterize a few interesting symmetries when a matrix happens to be square. As such, we have more rules and theorems around them, making them suitable for practical usage. For example, physics deeply utilizes Hermitian matrices, which are a special subclass of square matrices. Spectral theory is incredibly important in all sorts of applications.