4 ms·

> (...) as if you were ethically responsible for the model only. The model is the result and the outcome of the whole process. The model is supposed to be a re

by rualca 5y ago

> (...) as if you were ethically responsible for the model only. The model is the result and the outcome of the whole process. The model is supposed to be a reliable representation of reality that has acceptable accuracy within predetermined operational limits. If someone tries to generate models that interpolate and/or extrapolate data and the data it uses to seed the model is garbage then the model is garbage, and the person responsible for putting it together is the person responsible for the model failing to do its job. There is no separation between model and the data used to generate the model. Garbage in, garbage out. If a machine learning model uses garbage data then the model is garbage, and throwing machine learning buzzwords at the problem does not justify why garbage was sent in.

mjburgess 5y ago
> he model is supposed to be a reliable representation of reality that has acceptable accuracy within predetermined operational limits. This is the problem. That is NOT what ML models are supposed to be outside of newspaper articles and research grant proposals. ML models interpolate between co-occurrences; that is all they do. Representations of reality are not interpolations between statistical co-occurances -- that is the furthest thing from what they are. Reality has a counter-factual (ie., modal) structure, causal structure, generative structure (etc.) and much more.
- rualca 5y ago
  > This is the problem. That is NOT what ML models are supposed to be outside of newspaper articles and research grant proposals. No, not really. That is the very definition of a model, and the very reason why people bother with then. Stating otherwise requires a gross misunderstanding of the whole subject and problem domain. Also, it makes absolutely no difference if you know a model fails to output accurate results on specific scenarios. The key factor if you know the domain where the model does indeed provide acceptable results. That's why in some applications gravity is modelled as a constant acceleration of 9.8m/s2 pointing straight down, or materials modelled having a linear relationship between stress and strain. Those who use those models know that they are only valid if specific circumstances are met. This is something known forever in engineering, and made famous in statistical model circles by George Box with his quote of "all models are wrong". My take is that there is a kind of naive arrogance plaguing ML circles where throwing novel hightech approaches to old and basic modeling applications leads those in the field into believing that they can take a blank slate approach to modeling and ignore lessons learned throughout the centuries because they aren't needed anymore. And this is the end result.
  - mjburgess 5y ago
    The presumption of ML is that compressions of X are representations of Y. This is just false. The compression of X, ie., f, isnt a representation of Y. Its an estimator of the value of Y within some domain of X,Y. For it to be f to be a representation of Y, it needs to be able to stand-in for Y (at least). And compressions of X cannot. They lack, eg., the right counterfactual behaviour. ie., A representation of a cat enables computing things about a cat in imaginative scenarios, eg., in a video game simulation. Compressions of pixels of cats do not.
    - rualca 5y ago
      > The presumption of ML is that compressions of X are representations of Y. This is just false. The whole point of modeling is that without a doubt compressions of X are indeed representations of Y, because the whole reason X was picked was that it is clearly represents Y with an acceptable degree of accuracy for the use in mind. If a proposed model fails to reproduce and predict a model with acceptable accuracy, it's scrapped in favour of those that can. Why are we discussing this? I frankly do not understand why we are wasting time explaining the very basics of what a model is and why they are created and how they are used. This is not up for discussion. Just ask yourself why people, specially in engineering and physics, bother with models.

by rualca 5y ago

mjburgess 5y ago

> he model is supposed to be a reliable representation of reality that has acceptable accuracy within predetermined operational limits. This is the problem. That is NOT what ML models are supposed to be outside of newspaper articles and research grant proposals. ML models interpolate between co-occurrences; that is all they do. Representations of reality are not interpolations between statistical co-occurances -- that is the furthest thing from what they are. Reality has a counter-factual (ie., modal) structure, causal structure, generative structure (etc.) and much more.

rualca 5y ago
> This is the problem. That is NOT what ML models are supposed to be outside of newspaper articles and research grant proposals. No, not really. That is the very definition of a model, and the very reason why people bother with then. Stating otherwise requires a gross misunderstanding of the whole subject and problem domain. Also, it makes absolutely no difference if you know a model fails to output accurate results on specific scenarios. The key factor if you know the domain where the model does indeed provide acceptable results. That's why in some applications gravity is modelled as a constant acceleration of 9.8m/s2 pointing straight down, or materials modelled having a linear relationship between stress and strain. Those who use those models know that they are only valid if specific circumstances are met. This is something known forever in engineering, and made famous in statistical model circles by George Box with his quote of "all models are wrong". My take is that there is a kind of naive arrogance plaguing ML circles where throwing novel hightech approaches to old and basic modeling applications leads those in the field into believing that they can take a blank slate approach to modeling and ignore lessons learned throughout the centuries because they aren't needed anymore. And this is the end result.
- mjburgess 5y ago
  The presumption of ML is that compressions of X are representations of Y. This is just false. The compression of X, ie., f, isnt a representation of Y. Its an estimator of the value of Y within some domain of X,Y. For it to be f to be a representation of Y, it needs to be able to stand-in for Y (at least). And compressions of X cannot. They lack, eg., the right counterfactual behaviour. ie., A representation of a cat enables computing things about a cat in imaginative scenarios, eg., in a video game simulation. Compressions of pixels of cats do not.
  - rualca 5y ago
    > The presumption of ML is that compressions of X are representations of Y. This is just false. The whole point of modeling is that without a doubt compressions of X are indeed representations of Y, because the whole reason X was picked was that it is clearly represents Y with an acceptable degree of accuracy for the use in mind. If a proposed model fails to reproduce and predict a model with acceptable accuracy, it's scrapped in favour of those that can. Why are we discussing this? I frankly do not understand why we are wasting time explaining the very basics of what a model is and why they are created and how they are used. This is not up for discussion. Just ask yourself why people, specially in engineering and physics, bother with models.