That is true. Exact solutions have further advantages: they are expressed in terms of parameters (integration constants) which means that one can get deep insig

That is true. Exact solutions have further advantages: they are expressed in terms of parameters (integration constants) which means that one can get deep insights into a system's behavior under changing parameters. In a numerical approach replicating such studies can be very cumbersome. But the irony is that in the broad scheme of things "interesting" exact solutions are very rare, almost freakish. The only systems we can reliably solve are linear :-). So while numerical approaches have weaknesses, they are the de-facto available tool in most cases.