It's a baseline desirable property. It tells us that, at the very least, neutral networks are capable of approximating any continuous function. This isn't true

It's a baseline desirable property. It tells us that, at the very least, neutral networks are capable of approximating any continuous function. This isn't true for linear functions, for example, so we wouldn't want to try and model everything using linear functions.