I’m not sure. My intuition comes from real analysis, where one can think of the “space of all continuous functions” as being kind of geometric like Euclidean sp

I’m not sure. My intuition comes from real analysis, where one can think of the “space of all continuous functions” as being kind of geometric like Euclidean space. Then “Universal Approximation” for some Special set of functions (say those coming from neural networks) is equivalent to saying those special functions are “dense” in the space of all continuous functions. My geometric reasoning is that if a special collection is not dense, then it’s probably something like a lower-dimensional subspace, which really means you’re missing most of the ambient space in some sense. It’s all a bit handwavy but I hope that paints a picture. Getting more in the weeds, there are theorems like https://en.wikipedia.org/wiki/Stone%E2%80%93Weierstrass_theorem#Stone%E2%80%93Weierstrass_theorem,_real_version https://en.wikipedia.org/wiki/Stone%E2%80%93Weierstrass_theo... which suggest that being “dense in the space of continuous functions” is not as hard to achieve as one might guess.