5 ms·

I manage an applied research group in the machine learning space, particularly AutoML and synthetic data generation. That means is we read the latest literature

I manage an applied research group in the machine learning space, particularly AutoML and synthetic data generation. That means is we read the latest literature and create applications. Sometimes we build new research, most of the time we patent something new. But the goal is to get research to production as fast and as impactful as possible. The first thing I do when sorting research (papers, blog posts, presentations, etc) is sort the research by “B.S.”, “maybe B.S.”, “probably real”. If you look at research papers in computer science, half are what have the following problem: “running a simulation over and over until the authors get the results They like”. I can’t tell you how many papers with thousands of citations are just blatantly wrong. They don’t run corrections based on the number of simulations they run, they don’t take into account other variables, etc This isn’t limited to CS either. Biology, economics, environmental sciences, etc all suffer the same fate. The worst part about it, is it’s PEER REVIEWED. Meaning, others agree this is the way to do things, which is why I don’t trust academia almost at all.

Q57C3HYc7g 5y ago
Maybe this is a dumb question, but what do you mean by "run corrections based on the number of simulations they run"?
- L-four 5y ago
  Say you run 5 simulations and when comparing to the standard approach as a percent the samples are [-1%, -1%, +2%, +2%, +%6]. Your new method is not 6% better the standard it's at best +1.6% better.
- Jabbles 5y ago
  For example, https://en.wikipedia.org/wiki/Look-elsewhere_effect https://en.wikipedia.org/wiki/Look-elsewhere_effect and more generally https://en.wikipedia.org/wiki/Multiple_comparisons_problem https://en.wikipedia.org/wiki/Multiple_comparisons_problem
goalieca 5y ago
I despised how data sets and code were considered a gold-mine and researchers would often refuse to release them lest they give up a serious paper publishing advantage for their group. More often than not, i found code full of serious errors and completely lacking basic test cases and sanity. Sometimes the data sets were much larger than the subset chosen for publication.
galimaufry 5y ago
> They don’t run corrections based on the number of simulations they run, they don’t take into account other variables, etc I think this looks like a bigger problem specifically because you are in AutoML. Suppose you are training a GAN. There's notoriously a certain amount of luck involved in traditional GAN training, because you need the adversary and the generator to balance each other just right. So people try many times until they succeed. Probably they were not even recording each attempt, so they do not report how many times they had to run before getting good results. From an AutoML point of view, this is BS work - the training procedure cannot be automated, and (apart from using the actual seeds) the work cannot be reproduced. But from the point of view of everyone else, maybe it is fine. They get a generator model at the end, it works, other people can run it.
- pnt12 5y ago
  >But from the point of view of everyone else, maybe it is fine I think from a practical perspective, it is fine. You want results and you have a black box algorithm that produces them, fine. From an academic perspective, AI research is a mess. The reason you try something is not from a logical theory, but due from a "hunch" or replicatinga similar algorithm applied in a parallel area. If it does not work, you change some parameters and run it more times. Still not working so maybe you extend the network to include some more inputs and hope for better results. I did my thesis in machine learning and was very disappointed with the state of the field.
  - caddemon 5y ago
    I don't think there's necessarily a problem with trying things on a hunch, some of the best results in science have been due to a hunch or even an accident. The problem comes from trying a dozen hunches and only writing up one, or like you say completely cherry picking hyperparameters.
eeegnu 5y ago
If you keep track of these categories and have amassed a large enough dataset, it sounds like a fun experiment to see if a NN can learn some feature of the papers text to predict if a paper is BS. Some combinations of citations may even be a telling feature.
christophilus 5y ago
It’s a problem and is the reason I default to skepticism. What is the solution, though? Skepticism is crippling— and definitely annoying to others. Drives my family crazy, because it comes off as belligerent contrarianism.

Q57C3HYc7g 5y ago

Maybe this is a dumb question, but what do you mean by "run corrections based on the number of simulations they run"?

L-four 5y ago
Say you run 5 simulations and when comparing to the standard approach as a percent the samples are [-1%, -1%, +2%, +2%, +%6]. Your new method is not 6% better the standard it's at best +1.6% better.
Jabbles 5y ago
For example, https://en.wikipedia.org/wiki/Look-elsewhere_effect https://en.wikipedia.org/wiki/Look-elsewhere_effect and more generally https://en.wikipedia.org/wiki/Multiple_comparisons_problem https://en.wikipedia.org/wiki/Multiple_comparisons_problem

goalieca 5y ago

I despised how data sets and code were considered a gold-mine and researchers would often refuse to release them lest they give up a serious paper publishing advantage for their group. More often than not, i found code full of serious errors and completely lacking basic test cases and sanity. Sometimes the data sets were much larger than the subset chosen for publication.

galimaufry 5y ago

> They don’t run corrections based on the number of simulations they run, they don’t take into account other variables, etc I think this looks like a bigger problem specifically because you are in AutoML. Suppose you are training a GAN. There's notoriously a certain amount of luck involved in traditional GAN training, because you need the adversary and the generator to balance each other just right. So people try many times until they succeed. Probably they were not even recording each attempt, so they do not report how many times they had to run before getting good results. From an AutoML point of view, this is BS work - the training procedure cannot be automated, and (apart from using the actual seeds) the work cannot be reproduced. But from the point of view of everyone else, maybe it is fine. They get a generator model at the end, it works, other people can run it.

pnt12 5y ago
>But from the point of view of everyone else, maybe it is fine I think from a practical perspective, it is fine. You want results and you have a black box algorithm that produces them, fine. From an academic perspective, AI research is a mess. The reason you try something is not from a logical theory, but due from a "hunch" or replicatinga similar algorithm applied in a parallel area. If it does not work, you change some parameters and run it more times. Still not working so maybe you extend the network to include some more inputs and hope for better results. I did my thesis in machine learning and was very disappointed with the state of the field.
- caddemon 5y ago
  I don't think there's necessarily a problem with trying things on a hunch, some of the best results in science have been due to a hunch or even an accident. The problem comes from trying a dozen hunches and only writing up one, or like you say completely cherry picking hyperparameters.

eeegnu 5y ago

If you keep track of these categories and have amassed a large enough dataset, it sounds like a fun experiment to see if a NN can learn some feature of the papers text to predict if a paper is BS. Some combinations of citations may even be a telling feature.

christophilus 5y ago

It’s a problem and is the reason I default to skepticism. What is the solution, though? Skepticism is crippling— and definitely annoying to others. Drives my family crazy, because it comes off as belligerent contrarianism.