5 ms·

A One Parameter Equation That Can Exactly Fit Any Scatter Plot

32 points by richardhod 8y ago

woopwoop 8y ago
Their formula is neat, but the fact that you can fit any finite set of data to arbitrary precision with a smoothly varying one parameter family of functions is easy. Here is another construction. Let P_1, P_2, ... be an enumeration of the polynomials with rational coefficients, let phi_i be a smooth function supported on [i-1/2,i+1/2] with phi_i(i)=1, and let f_theta(x) = sum phi_i(theta) P_i(x) = phi_round(theta)(theta) P_theta(x), where round(theta) is the closest integer to theta. Whatever you mean by smoothly varying one-parameter family of functions, this is surely it, since on any open interval the image is contained in a finite dimensional vector space and the curve of functions is smooth. Since we can hit any polynomial with rational coefficients by picking the right (integer) theta, and for any finite set E, f : E to R, and epsilon>0 there exists a polynomial P with rational coefficients such that |P(x) - f(x)|<epsilon for all x in E, we are done.
sgentle 8y ago
Heh, neat. Now all you need is to implement that logistic map solver in hardware and write a function that maps other functions into their equivalent "decimal code" and you've got a pretty fun computer going. The paper notes that these functions are infinitely continuously differentiable. I wonder if they'd have any applications in machine learning?
Waterluvian 8y ago
If I created a formula that randomly plotted each "pixel" on or off, there's a seed out there that will plot my name, no? Is this what the paper is about, or is it something different?
- deleted 8y ago
  [deleted]
mlazos 8y ago
This paper was really interesting and it's awesome, but I'm tired of everyone saying that this means parameter counting is a useless heuristic for the complexity of a model. The more parameters you add to some ML models (for instance NN and SVM) the higher the VC dimension, and that's that. Many models are simply not constructed in a way that a single parameter can encode all of the necessary information. (And still be a continuous function of that parameter)
RobertoG 8y ago
"[..] the paper also tells us that Occam’s Razor is wrong [..]" I doubt the paper say that, and I don't think it's true. It seems to me that Occam's Razor is about the quantity of information, not about how many parameters you use to store the information.
viraptor 8y ago
Doesn't Gödel numbering already guarantee that kind of function?

32 points by richardhod 8y ago

Their formula is neat, but the fact that you can fit any finite set of data to arbitrary precision with a smoothly varying one parameter family of functions is easy. Here is another construction. Let P_1, P_2, ... be an enumeration of the polynomials with rational coefficients, let phi_i be a smooth function supported on [i-1/2,i+1/2] with phi_i(i)=1, and let f_theta(x) = sum phi_i(theta) P_i(x) = phi_round(theta)(theta) P_theta(x), where round(theta) is the closest integer to theta. Whatever you mean by smoothly varying one-parameter family of functions, this is surely it, since on any open interval the image is contained in a finite dimensional vector space and the curve of functions is smooth. Since we can hit any polynomial with rational coefficients by picking the right (integer) theta, and for any finite set E, f : E to R, and epsilon>0 there exists a polynomial P with rational coefficients such that |P(x) - f(x)|<epsilon for all x in E, we are done.

sgentle 8y ago

Heh, neat. Now all you need is to implement that logistic map solver in hardware and write a function that maps other functions into their equivalent "decimal code" and you've got a pretty fun computer going. The paper notes that these functions are infinitely continuously differentiable. I wonder if they'd have any applications in machine learning?

Waterluvian 8y ago

If I created a formula that randomly plotted each "pixel" on or off, there's a seed out there that will plot my name, no? Is this what the paper is about, or is it something different?

deleted 8y ago
[deleted]

mlazos 8y ago

This paper was really interesting and it's awesome, but I'm tired of everyone saying that this means parameter counting is a useless heuristic for the complexity of a model. The more parameters you add to some ML models (for instance NN and SVM) the higher the VC dimension, and that's that. Many models are simply not constructed in a way that a single parameter can encode all of the necessary information. (And still be a continuous function of that parameter)

RobertoG 8y ago

"[..] the paper also tells us that Occam’s Razor is wrong [..]" I doubt the paper say that, and I don't think it's true. It seems to me that Occam's Razor is about the quantity of information, not about how many parameters you use to store the information.

viraptor 8y ago

Doesn't Gödel numbering already guarantee that kind of function?