Well this is getting in some rather unfamiliar territory to me, but a function like max(0,x) could be considered a polynomial in the max-plus algebra [1]. In th

Well this is getting in some rather unfamiliar territory to me, but a function like max(0,x) could be considered a polynomial in the max-plus algebra [1]. In this algebra the values kind of behave like the base-b log of a real number, with b taken to infinity [2]. If you replace b by a really big number you're just dealing with regular polynomials. In this sense I think you could consider a big neural network with RELU activation a (really big) polynomial. This is about the extent of my knowledge on the subject though, for more info search for tropical geometry [3]. [1]: https://en.wikipedia.org/wiki/Max-plus_algebra https://en.wikipedia.org/wiki/Max-plus_algebra [2]: https://mathoverflow.net/questions/83624/why-tropical-geometry https://mathoverflow.net/questions/83624/why-tropical-geomet... [3]: https://en.wikipedia.org/wiki/Tropical_geometry https://en.wikipedia.org/wiki/Tropical_geometry