5 ms·

Academic machine learning is a paper mill. Sometimes there is a rare success like the transformer model and everyone converges on that. I think the comments he

by kakhag 2y ago

Academic machine learning is a paper mill. Sometimes there is a rare success like the transformer model and everyone converges on that. I think the comments here are arguing past each other. If a machine learning model can correctly classify $attribute by photos in 70% of the cases, that is an academic result. Should it used for the ever popular pre-crime detection scams or for misclassifying feminine looking men as homosexuals? No, that is nasty and is forbidden in the EU. At least for companies, the security state of course does whatever it wants without any repercussions. Should it be used in principle? No, classifying humans by a machine algorithm is dehumanizing. Should research be done at all? If the subjects are volunteers and it is confined to a university, it should be allowed. But researchers should not be surprised it they are confronted with obvious parallels to phrenology. This literally is phrenology.

darosati 2y ago
I just want to push back on Academic Machine Learning is a (low quality with no novelty) paper mill and devaluing researchers efforts. To be clear ML research has “paper mill” problems but we should be careful that we don’t imply that there are only “rare successes” There are many many amazing results published at ICLR, NeurIPs, ICML every year that are important developments that are not only research successes but also commercial and open source success stories. For example LoRA and DPO are two recent incredible development and these are not “rare” - this years ICLR had many promising results that will in turn be built on to produce the next “transformer” level development. Without this work there are no transformers. Even transformers themselves were a contribution whose impact only became valuable through the work of many researches improving the architecture and finding applications (for example LLMs were not a given use case of transformers until additional researches put the work in to develop them)
logicchains 2y ago
> But researchers should not be surprised it they are confronted with obvious parallels to phrenology. This literally is phrenology Phrenology was classified a pseudoscience not because it could lead to socially bad outcomes but because it didn't work; it had no statistical/empirical grounding. If it turns out deep learning really can reliably predict things about people's personality from their faces, that doesn't make it a pseudoscience.
- pc86 2y ago
  No but you don't understand, the GP finds it icky therefore it's "dehumanizing."
- 12_throw_away 2y ago
  > If it turns out deep learning really can reliably predict things about people's personality from their faces, that doesn't make it a pseudoscience. What makes it pseudoscience is that it's not theory-driven. These are statistical models that recapitulate distributions in their training data. It's Brian-Wansink-style p-hacking [1] at a massive scale. [1] https://www.vox.com/science-and-health/2018/9/19/17879102/brian-wansink-cornell-food-brand-lab-retractions-jama https://www.vox.com/science-and-health/2018/9/19/17879102/br...
  - naasking 2y ago
    > What makes it pseudoscience is that it's not theory-driven Science is primarily empirical, and there's nothing inherently wrong with an effective theory that works, until we find a better, principled theory. > It's Brian-Wansink-style p-hacking [1] at a massive scale. Sometimes. Models that generalize are in fact generating theories though.
- soVeryTired 2y ago
  To add to your point - you definitely can make predictions about aspects of someone’s personality by looking at their face with the expectation that you’ll do better than chance. The paper calls out inferring political leanings as an example of pseudoscience. Give me an American’s age, gender, and race (which I can roughly identify by looking) and I’ll tell you, better than chance, whether they’re a trump supporter.
- thrance 2y ago
  Agreed, but I think the parallel with phrenology should be undestood more along the lines of "providing a justification for unfair decisions". I remember a case where Amazon used a resume filtering bot that systematically rejected female candidates, because of bias in the training data. So we might go from "you can't be free because your skull is too small" to "you can't get the job because the computer says so".
pas 2y ago
> Should it be used in principle? No, classifying humans by a machine algorithm is dehumanizing. I see utility in a machine that can help me notice when I'm having a bad day (or a very bad day ie. when having a stroke) or ... help me figure out what the fuck is going on with my totally hypothetical mental illness.
- pstuart 2y ago
  As a personal assistant it has incredible power, but it should effectively be self hosted.
- ab5tract 2y ago
  I see plenty to fear when other people deploy AI to “determine” and judge my mood and my non-hypothetical mental illnesses without my knowledge or consent.
  - pas 2y ago
    Not unreasonably so! I also like my privacy (or what's left of it) ... but when (if? no, unfortunately most likely when, and maybe even "soon") it goes at least I want to have some benefit for myself. Arguably the current generation of school-age kids (genZ? genA?) already lost it. Because even if they don't have a phone, others do. (Though hopefully this trend can be reversed. Classic bullying at school is bad enough now with cyberbullying becoming the "norm" things are definitely not looking great.)

by kakhag 2y ago

darosati 2y ago

I just want to push back on Academic Machine Learning is a (low quality with no novelty) paper mill and devaluing researchers efforts. To be clear ML research has “paper mill” problems but we should be careful that we don’t imply that there are only “rare successes” There are many many amazing results published at ICLR, NeurIPs, ICML every year that are important developments that are not only research successes but also commercial and open source success stories. For example LoRA and DPO are two recent incredible development and these are not “rare” - this years ICLR had many promising results that will in turn be built on to produce the next “transformer” level development. Without this work there are no transformers. Even transformers themselves were a contribution whose impact only became valuable through the work of many researches improving the architecture and finding applications (for example LLMs were not a given use case of transformers until additional researches put the work in to develop them)

logicchains 2y ago

> But researchers should not be surprised it they are confronted with obvious parallels to phrenology. This literally is phrenology Phrenology was classified a pseudoscience not because it could lead to socially bad outcomes but because it didn't work; it had no statistical/empirical grounding. If it turns out deep learning really can reliably predict things about people's personality from their faces, that doesn't make it a pseudoscience.

pc86 2y ago
No but you don't understand, the GP finds it icky therefore it's "dehumanizing."
12_throw_away 2y ago
> If it turns out deep learning really can reliably predict things about people's personality from their faces, that doesn't make it a pseudoscience. What makes it pseudoscience is that it's not theory-driven. These are statistical models that recapitulate distributions in their training data. It's Brian-Wansink-style p-hacking [1] at a massive scale. [1] https://www.vox.com/science-and-health/2018/9/19/17879102/brian-wansink-cornell-food-brand-lab-retractions-jama https://www.vox.com/science-and-health/2018/9/19/17879102/br...
- naasking 2y ago
  > What makes it pseudoscience is that it's not theory-driven Science is primarily empirical, and there's nothing inherently wrong with an effective theory that works, until we find a better, principled theory. > It's Brian-Wansink-style p-hacking [1] at a massive scale. Sometimes. Models that generalize are in fact generating theories though.
soVeryTired 2y ago
To add to your point - you definitely can make predictions about aspects of someone’s personality by looking at their face with the expectation that you’ll do better than chance. The paper calls out inferring political leanings as an example of pseudoscience. Give me an American’s age, gender, and race (which I can roughly identify by looking) and I’ll tell you, better than chance, whether they’re a trump supporter.
thrance 2y ago
Agreed, but I think the parallel with phrenology should be undestood more along the lines of "providing a justification for unfair decisions". I remember a case where Amazon used a resume filtering bot that systematically rejected female candidates, because of bias in the training data. So we might go from "you can't be free because your skull is too small" to "you can't get the job because the computer says so".

pas 2y ago

> Should it be used in principle? No, classifying humans by a machine algorithm is dehumanizing. I see utility in a machine that can help me notice when I'm having a bad day (or a very bad day ie. when having a stroke) or ... help me figure out what the fuck is going on with my totally hypothetical mental illness.

pstuart 2y ago
As a personal assistant it has incredible power, but it should effectively be self hosted.
ab5tract 2y ago
I see plenty to fear when other people deploy AI to “determine” and judge my mood and my non-hypothetical mental illnesses without my knowledge or consent.
- pas 2y ago
  Not unreasonably so! I also like my privacy (or what's left of it) ... but when (if? no, unfortunately most likely when, and maybe even "soon") it goes at least I want to have some benefit for myself. Arguably the current generation of school-age kids (genZ? genA?) already lost it. Because even if they don't have a phone, others do. (Though hopefully this trend can be reversed. Classic bullying at school is bad enough now with cyberbullying becoming the "norm" things are definitely not looking great.)