4 ms·

This belief has always been extremely cringe in my opinion, because it somehow implies that dataset engineering is not an important part of machine learning. En

This belief has always been extremely cringe in my opinion, because it somehow implies that dataset engineering is not an important part of machine learning. Ensuring you are feeding good data to your model is a critical part of machine learning practice, but some researchers seem to hand wave it away as merely an implementation detail not worthy of their attention. "It's a data problem" is used as an excuse to be discharged of the moral responsability of the output bias, as if you were ethically responsible for the model only.

darawk 5y ago
I don't think "it's a data problem" is inherently used to hand-wave away the problem. I think it's used to locate the problem in the area where its most readily addressed. Designing algorithms to debias your data is hard, over-sampling under-represented groups is easier. I do think you're right that sometimes people use it that way, but that doesn't mean we should make up false narratives about biased models, either. It just means we should work on de-biasing the data, and also developing algorithms to help mitigate the learning of features we don't want weighted.
- mjburgess 5y ago
  You are right that there are no algorithms which capture semantics, and therefore none which can be constructed not to introduce bias. In that sense it is a "data solution", but it is not a data problem. > de-biasing the data The issue isnt statistical bias. The issue is semantic: occurrence doesnt capture meaning. Negative terms and racial terms can co-occur more frequently in some text (eg., a biography of MLK) without the remedy being to "rebalance them with positive terms".
rualca 5y ago
> (...) as if you were ethically responsible for the model only. The model is the result and the outcome of the whole process. The model is supposed to be a reliable representation of reality that has acceptable accuracy within predetermined operational limits. If someone tries to generate models that interpolate and/or extrapolate data and the data it uses to seed the model is garbage then the model is garbage, and the person responsible for putting it together is the person responsible for the model failing to do its job. There is no separation between model and the data used to generate the model. Garbage in, garbage out. If a machine learning model uses garbage data then the model is garbage, and throwing machine learning buzzwords at the problem does not justify why garbage was sent in.
- mjburgess 5y ago
  > he model is supposed to be a reliable representation of reality that has acceptable accuracy within predetermined operational limits. This is the problem. That is NOT what ML models are supposed to be outside of newspaper articles and research grant proposals. ML models interpolate between co-occurrences; that is all they do. Representations of reality are not interpolations between statistical co-occurances -- that is the furthest thing from what they are. Reality has a counter-factual (ie., modal) structure, causal structure, generative structure (etc.) and much more.
  - rualca 5y ago
    > This is the problem. That is NOT what ML models are supposed to be outside of newspaper articles and research grant proposals. No, not really. That is the very definition of a model, and the very reason why people bother with then. Stating otherwise requires a gross misunderstanding of the whole subject and problem domain. Also, it makes absolutely no difference if you know a model fails to output accurate results on specific scenarios. The key factor if you know the domain where the model does indeed provide acceptable results. That's why in some applications gravity is modelled as a constant acceleration of 9.8m/s2 pointing straight down, or materials modelled having a linear relationship between stress and strain. Those who use those models know that they are only valid if specific circumstances are met. This is something known forever in engineering, and made famous in statistical model circles by George Box with his quote of "all models are wrong". My take is that there is a kind of naive arrogance plaguing ML circles where throwing novel hightech approaches to old and basic modeling applications leads those in the field into believing that they can take a blank slate approach to modeling and ignore lessons learned throughout the centuries because they aren't needed anymore. And this is the end result.
    - mjburgess 5y ago
      The presumption of ML is that compressions of X are representations of Y. This is just false. The compression of X, ie., f, isnt a representation of Y. Its an estimator of the value of Y within some domain of X,Y. For it to be f to be a representation of Y, it needs to be able to stand-in for Y (at least). And compressions of X cannot. They lack, eg., the right counterfactual behaviour. ie., A representation of a cat enables computing things about a cat in imaginative scenarios, eg., in a video game simulation. Compressions of pixels of cats do not.
ad404b8a372f2b9 5y ago
You cannot conflate dataset engineering, machine learning, and researchers as if they're a single discipline and people. This is precisely where the disagreement stems from. Dataset engineering is an important field of research. It is also an important part of the modelling process. Neither of these things are included in model research. Researchers who design computer vision models and other novel mathematical models can be held ethically responsible for the societal results of the novel part of their research but if we agree that their field of research is valuable for society then they cannot be held responsible for the misuse of these models by people who skip or fail integral parts of the modelling process. You cannot ask a researcher to invent a new more performant computer-vision model, invent a way to correct for bias, and design a fair dataset all in one. That's three entirely different careers.
- MichaelGroves 5y ago
  > You cannot ask a researcher to invent a new more performant computer-vision model, invent a way to correct for bias, and design a fair dataset all in one. That's three entirely different careers. Might it not sometimes be reasonable to ask researchers in one domain to slow their roll, if researchers in a related domain are making slower progress? If I invent 10 new forms of nerve gas every year, while you invent 5 new antidotes to nerve gas every year, where does the problem lay? Solely with you, for not working fast enough? I don't think that's reasonable. Does the problem lay with me, for not inventing nerve gas antidotes? I don't think that's reasonable either, antidotes are not my specialty and I can't be expected to create them. But if I'm the one inventing new systems that become a problem for society because your domain of counter-research isn't keeping up, then I have some substantial share in that blame nonetheless.
  - mirker 5y ago
    Most will not pass up an opportunity to keep their job or get promoted, so I think the practical answer lies elsewhere in the research pipeline.

darawk 5y ago

I don't think "it's a data problem" is inherently used to hand-wave away the problem. I think it's used to locate the problem in the area where its most readily addressed. Designing algorithms to debias your data is hard, over-sampling under-represented groups is easier. I do think you're right that sometimes people use it that way, but that doesn't mean we should make up false narratives about biased models, either. It just means we should work on de-biasing the data, and also developing algorithms to help mitigate the learning of features we don't want weighted.

mjburgess 5y ago
You are right that there are no algorithms which capture semantics, and therefore none which can be constructed not to introduce bias. In that sense it is a "data solution", but it is not a data problem. > de-biasing the data The issue isnt statistical bias. The issue is semantic: occurrence doesnt capture meaning. Negative terms and racial terms can co-occur more frequently in some text (eg., a biography of MLK) without the remedy being to "rebalance them with positive terms".

rualca 5y ago

> (...) as if you were ethically responsible for the model only. The model is the result and the outcome of the whole process. The model is supposed to be a reliable representation of reality that has acceptable accuracy within predetermined operational limits. If someone tries to generate models that interpolate and/or extrapolate data and the data it uses to seed the model is garbage then the model is garbage, and the person responsible for putting it together is the person responsible for the model failing to do its job. There is no separation between model and the data used to generate the model. Garbage in, garbage out. If a machine learning model uses garbage data then the model is garbage, and throwing machine learning buzzwords at the problem does not justify why garbage was sent in.

mjburgess 5y ago
> he model is supposed to be a reliable representation of reality that has acceptable accuracy within predetermined operational limits. This is the problem. That is NOT what ML models are supposed to be outside of newspaper articles and research grant proposals. ML models interpolate between co-occurrences; that is all they do. Representations of reality are not interpolations between statistical co-occurances -- that is the furthest thing from what they are. Reality has a counter-factual (ie., modal) structure, causal structure, generative structure (etc.) and much more.
- rualca 5y ago
  > This is the problem. That is NOT what ML models are supposed to be outside of newspaper articles and research grant proposals. No, not really. That is the very definition of a model, and the very reason why people bother with then. Stating otherwise requires a gross misunderstanding of the whole subject and problem domain. Also, it makes absolutely no difference if you know a model fails to output accurate results on specific scenarios. The key factor if you know the domain where the model does indeed provide acceptable results. That's why in some applications gravity is modelled as a constant acceleration of 9.8m/s2 pointing straight down, or materials modelled having a linear relationship between stress and strain. Those who use those models know that they are only valid if specific circumstances are met. This is something known forever in engineering, and made famous in statistical model circles by George Box with his quote of "all models are wrong". My take is that there is a kind of naive arrogance plaguing ML circles where throwing novel hightech approaches to old and basic modeling applications leads those in the field into believing that they can take a blank slate approach to modeling and ignore lessons learned throughout the centuries because they aren't needed anymore. And this is the end result.
  - mjburgess 5y ago
    The presumption of ML is that compressions of X are representations of Y. This is just false. The compression of X, ie., f, isnt a representation of Y. Its an estimator of the value of Y within some domain of X,Y. For it to be f to be a representation of Y, it needs to be able to stand-in for Y (at least). And compressions of X cannot. They lack, eg., the right counterfactual behaviour. ie., A representation of a cat enables computing things about a cat in imaginative scenarios, eg., in a video game simulation. Compressions of pixels of cats do not.

ad404b8a372f2b9 5y ago

You cannot conflate dataset engineering, machine learning, and researchers as if they're a single discipline and people. This is precisely where the disagreement stems from. Dataset engineering is an important field of research. It is also an important part of the modelling process. Neither of these things are included in model research. Researchers who design computer vision models and other novel mathematical models can be held ethically responsible for the societal results of the novel part of their research but if we agree that their field of research is valuable for society then they cannot be held responsible for the misuse of these models by people who skip or fail integral parts of the modelling process. You cannot ask a researcher to invent a new more performant computer-vision model, invent a way to correct for bias, and design a fair dataset all in one. That's three entirely different careers.

MichaelGroves 5y ago
> You cannot ask a researcher to invent a new more performant computer-vision model, invent a way to correct for bias, and design a fair dataset all in one. That's three entirely different careers. Might it not sometimes be reasonable to ask researchers in one domain to slow their roll, if researchers in a related domain are making slower progress? If I invent 10 new forms of nerve gas every year, while you invent 5 new antidotes to nerve gas every year, where does the problem lay? Solely with you, for not working fast enough? I don't think that's reasonable. Does the problem lay with me, for not inventing nerve gas antidotes? I don't think that's reasonable either, antidotes are not my specialty and I can't be expected to create them. But if I'm the one inventing new systems that become a problem for society because your domain of counter-research isn't keeping up, then I have some substantial share in that blame nonetheless.
- mirker 5y ago
  Most will not pass up an opportunity to keep their job or get promoted, so I think the practical answer lies elsewhere in the research pipeline.