4 ms·

> There is often no function to find in solving a problem In finance NN can be used to calibrate models, i.e. generate parameters to a model (function) that re

> There is often no function to find in solving a problem In finance NN can be used to calibrate models, i.e. generate parameters to a model (function) that replicates existing data (obsered prices/volatilities).

mjburgess 5y ago
Well i dont think prices are functions of economic variables. Recall a function is `y = f(x)`, not `y1, y2, y3... = f(x)`. So what you're modelling is something like `E_hope[y] = f(x)` where `E_hope` is "a hopeful expectation" that the mean of the underlying ambiguous y1,..yn does "reliably" mean to a unique `x`. This "hopeful expectation" is certainly more common than there being any actual function connecting `y` to `x`, but i think its often quite false too. Ie., even the expectation of prices is genuinely ambiguous. To handle this we might ensemble models `E_ensemble[E1_hope[y], ...En_hope[y]]`, but to repeat a famous idiom in finance, this is very much "building sandcastles in the sky". The idea that you can just "expect" (/statstics) your way out of the need for experimentation is a dangerous superstition which is at the heart of ML. It is impossible to simply "model data", measurement produces genuine ambiguities which can only be resolved by changing the world and seeing-what-happens. There is no function to find.
- thfuran 5y ago
  >Recall a function is `y = f(x)`, not `y1, y2, y3... = f(x)`. y1,...,yn is a perfectly reasonable function output. Functions don't have to produce scalars.
  - mjburgess 5y ago
    Functions have to resolve to one point in the output domain, even if that point is multi-dim. Here, consider `y_houseprice = price(house data, economic data, etc.)`. There isnt a unique house price in terms of those variables. The real world observes many such prices for the same value of those variables. An overly mathematical view of the world has obscured the scientific method from our thinking here. Generally, there arent actually functions from X to Y, and there arent actually stable XY distributions over time. The world, as measured, is basically always ambigouous and discontinuous. Data, as measurement, isnt the foundation of our theory-building. We build theories by changing the world; data comes in as a guiding light to our theory building, not as the basis. ..wwhich is our direct causal interactions with our environment, ie., its the actual stuff of our bodies and the stuff of the world as we change it
    - pedrosorio 5y ago
      > There isnt a unique house price in terms of those variables > An overly mathematical view of the world has obscured the scientific method from our thinking here Since it is understood that houseprice is not a function of just three variables, the mathematical view (statistical learning theory) commonly used when training models, defines house price as a random variable. This takes into account the uncertainty from all the unknown factors they contribute to house prices. The distribution defining this random variable is a function of the 3 input observations. Commonly, the inputs are used to compute the mean, and the shape of the distribution is fixed - a Gaussian, for example - but not necessarily. Given observations of the 3 inputs, each observed y_houseprice is just a sample from this random variable.
      - mjburgess 5y ago
        Well a random variable is a function, from event space to the real line. We return to a single measure by taking its expectation. We dont model `Y = f(X)`. This doesn't play well with the universal theorem in the article. NNs can only be said to model expectations of random variables.
        thfuran 5y ago
        >We return to a single measure by taking its expectation Only if you want to throw away most of the information in your model.
        deleted 5y ago
        [deleted]
        mjburgess 5y ago
        Well, indeed. One then needs to explain how a NN being a "universal fn approximator" helps at all in this context. One models RVs generativey using distributions (and so on), the actual model (eg., of house prices) isnt a function, it's often an infinity of them.

mjburgess 5y ago

Well i dont think prices are functions of economic variables. Recall a function is `y = f(x)`, not `y1, y2, y3... = f(x)`. So what you're modelling is something like `E_hope[y] = f(x)` where `E_hope` is "a hopeful expectation" that the mean of the underlying ambiguous y1,..yn does "reliably" mean to a unique `x`. This "hopeful expectation" is certainly more common than there being any actual function connecting `y` to `x`, but i think its often quite false too. Ie., even the expectation of prices is genuinely ambiguous. To handle this we might ensemble models `E_ensemble[E1_hope[y], ...En_hope[y]]`, but to repeat a famous idiom in finance, this is very much "building sandcastles in the sky". The idea that you can just "expect" (/statstics) your way out of the need for experimentation is a dangerous superstition which is at the heart of ML. It is impossible to simply "model data", measurement produces genuine ambiguities which can only be resolved by changing the world and seeing-what-happens. There is no function to find.

thfuran 5y ago
>Recall a function is `y = f(x)`, not `y1, y2, y3... = f(x)`. y1,...,yn is a perfectly reasonable function output. Functions don't have to produce scalars.
- mjburgess 5y ago
  Functions have to resolve to one point in the output domain, even if that point is multi-dim. Here, consider `y_houseprice = price(house data, economic data, etc.)`. There isnt a unique house price in terms of those variables. The real world observes many such prices for the same value of those variables. An overly mathematical view of the world has obscured the scientific method from our thinking here. Generally, there arent actually functions from X to Y, and there arent actually stable XY distributions over time. The world, as measured, is basically always ambigouous and discontinuous. Data, as measurement, isnt the foundation of our theory-building. We build theories by changing the world; data comes in as a guiding light to our theory building, not as the basis. ..wwhich is our direct causal interactions with our environment, ie., its the actual stuff of our bodies and the stuff of the world as we change it
  - pedrosorio 5y ago
    > There isnt a unique house price in terms of those variables > An overly mathematical view of the world has obscured the scientific method from our thinking here Since it is understood that houseprice is not a function of just three variables, the mathematical view (statistical learning theory) commonly used when training models, defines house price as a random variable. This takes into account the uncertainty from all the unknown factors they contribute to house prices. The distribution defining this random variable is a function of the 3 input observations. Commonly, the inputs are used to compute the mean, and the shape of the distribution is fixed - a Gaussian, for example - but not necessarily. Given observations of the 3 inputs, each observed y_houseprice is just a sample from this random variable.
    - mjburgess 5y ago
      Well a random variable is a function, from event space to the real line. We return to a single measure by taking its expectation. We dont model `Y = f(X)`. This doesn't play well with the universal theorem in the article. NNs can only be said to model expectations of random variables.
      - thfuran 5y ago
        >We return to a single measure by taking its expectation Only if you want to throw away most of the information in your model.
        deleted 5y ago
        [deleted]
        mjburgess 5y ago
        Well, indeed. One then needs to explain how a NN being a "universal fn approximator" helps at all in this context. One models RVs generativey using distributions (and so on), the actual model (eg., of house prices) isnt a function, it's often an infinity of them.