3 ms·
Since you are nitpicking, you could well use a sinusoid activation function on the Neural Network, and reach an even smaller loss value.
by leoff 5y ago
Since you are nitpicking, you could well use a sinusoid activation function on the Neural Network, and reach an even smaller loss value.
- credit_guy 5y agoNot sure I understand your point. Do you want to use a bunch of sine functions to approximate a sine function? What would that show? Splines don't know anything about the nature of a function. They approximate any function with piecewise polynomials. Maybe you are trying to say that the default activation function (relu) in sklearn is not smooth. No problem, you can add activation='tanh' inside the definition of the NN, and check the RMSE. Turns out it's for some reason worse.
- marginalia_nu 5y agoI assume they're referring to Fourier expansion. In general you can use a pretty wide set of functions to approximate an arbitrary function. You can do it with polynomials (Taylor expansion), and many others as long as they form a Hilbert space. Producing a given function from a linear combination of other functions isn't groundbreaking in the least.