> In particular, RELU activation doesn't explode towards infinity the way a high degree polynomial activation would. Isn't the ReLU a piecewise linear polynomi

> In particular, RELU activation doesn't explode towards infinity the way a high degree polynomial activation would. Isn't the ReLU a piecewise linear polynomial? Moreocer, if you convolve piecewise linear polynomials you get a higher-degree piecewise polynomial, typically referred to as a spline.