3 ms·

One surprising discovery about AI: There's nothing inherently wrong with cherrypicking. I used to think it was this dirty thing. Oh, you're only showing your be

One surprising discovery about AI: There's nothing inherently wrong with cherrypicking. I used to think it was this dirty thing. Oh, you're only showing your best results? But the best results are what matters. If you can create an amazing song from scratch by telling a computer what to do, you only need to do it once. The song is still good. And it's easy to automate. If you have a way of detecting a good song, or at least filtering out rubbish, then you can generate thousands of attempts to get the good result. I like to say "It's hard to pick cherries from a rotten tree." The flipside of this post, though, really hits home. I recently was super excited about a paper called FNet, which showed that "fourier transforms can be competitive with transformers", i.e. just replace multihead attention with a fourier transform. 7x faster on a GPU! Woo! Buried in section 3.1, they casually note that BERT with half the parameter count completely dominates them in terms of accuracy: https://twitter.com/theshawwn/status/1393315603973386240 https://twitter.com/theshawwn/status/1393315603973386240 In other words, the paper was ... advertised in a misleading way, to put it kindly. Most of the transformers they compared with aren't the traditional multihead GPT style transformer that people think of when they hear "transformer". And the one that was (BERT) totally annihilated them. But it's also easy to jump to conclusions too quickly. I've been a hothead in the past, and called out a paper when it turned out that I just didn't understand enough. It's tricky to know for sure.

teruakohatu 5y ago
> . If you have a way of detecting a good song, or at least filtering out rubbish, then you can generate thousands of attempts to get the good result. So often the filtering technique is a human in the loop, manually going through the output. Humans become the hidden, undifferentiatable and undisclosed loss function.
- sillysaurusx 5y ago
  Sure is! And you'd often be amazed at just how much data you can churn through, if you put your mind to it. I once watched Gwern classify thousands of poetry outputs over the course of a day or two, with nothing more than dedication and a bash script to mark "a" or "b". It took a long time to accept that this is both normal and effective. It's not some hidden secret to be ashamed of. The headline images in https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/ are probably the best and most interesting images they could find; they were picked out by humans. It's a positive statement to say that dall-e can do that some of the time, rather than "it fails X% of the time."
coldtea 5y ago
>But the best results are what matters. Not the kind of cherry-picking condemned here, which is more like your drug killing 90% of the animals it was tested on, and making a small difference in 2% of them, and you present it as only that 2% ever happened (and thus represent 100% of the cases).
- sillysaurusx 5y ago
  Oh, you're right: Trying that shiny new algorithm out on a couple dozen seeds, and then only reporting the best few. Running a big hyperparameter sweep on your proposed approach but using the defaults for the baseline. Cherry-picking examples where your model looks good, or cherry-picking whole datasets to test on, where you’ve confirmed your model’s advantage. Making up new problem settings, new datasets, new objectives in order to claim victory on an empty playing field. Proclaiming that your work is a “promising first step” in your introduction, despite being fully aware that nobody will ever build on it. Yeah, I misread; sorry. Testing your model on a small dataset and presenting it as a general solution is all too common. That's why generative models are cooler than classifiers, to me at least. You can show the outputs visually, whether it's text or image or sound. But with classifiers, you're chasing an accuracy rating. I forgot that people often test on CIFAR-10 (smol data) without verifying on Imagenet (big-ish data) and then present their paper as very general. But, I do have something to say about that kind of dataset cherrypicking: Researchers often find it hard to test on large datasets because of limited compute resources. One of the pioneers of DDPM wasn't able to train on imagenet due to lack of GPUs. CIFAR-10 was the best they could do. So it's sometimes hard to tell whether there's intentional deceit, or just a shoestring budget. (I was surprised how much research a lot of people get done in spite of limitations.) Thanks for the correction; cheers.

teruakohatu 5y ago

> . If you have a way of detecting a good song, or at least filtering out rubbish, then you can generate thousands of attempts to get the good result. So often the filtering technique is a human in the loop, manually going through the output. Humans become the hidden, undifferentiatable and undisclosed loss function.

sillysaurusx 5y ago
Sure is! And you'd often be amazed at just how much data you can churn through, if you put your mind to it. I once watched Gwern classify thousands of poetry outputs over the course of a day or two, with nothing more than dedication and a bash script to mark "a" or "b". It took a long time to accept that this is both normal and effective. It's not some hidden secret to be ashamed of. The headline images in https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/ are probably the best and most interesting images they could find; they were picked out by humans. It's a positive statement to say that dall-e can do that some of the time, rather than "it fails X% of the time."

coldtea 5y ago

>But the best results are what matters. Not the kind of cherry-picking condemned here, which is more like your drug killing 90% of the animals it was tested on, and making a small difference in 2% of them, and you present it as only that 2% ever happened (and thus represent 100% of the cases).

sillysaurusx 5y ago
Oh, you're right: Trying that shiny new algorithm out on a couple dozen seeds, and then only reporting the best few. Running a big hyperparameter sweep on your proposed approach but using the defaults for the baseline. Cherry-picking examples where your model looks good, or cherry-picking whole datasets to test on, where you’ve confirmed your model’s advantage. Making up new problem settings, new datasets, new objectives in order to claim victory on an empty playing field. Proclaiming that your work is a “promising first step” in your introduction, despite being fully aware that nobody will ever build on it. Yeah, I misread; sorry. Testing your model on a small dataset and presenting it as a general solution is all too common. That's why generative models are cooler than classifiers, to me at least. You can show the outputs visually, whether it's text or image or sound. But with classifiers, you're chasing an accuracy rating. I forgot that people often test on CIFAR-10 (smol data) without verifying on Imagenet (big-ish data) and then present their paper as very general. But, I do have something to say about that kind of dataset cherrypicking: Researchers often find it hard to test on large datasets because of limited compute resources. One of the pioneers of DDPM wasn't able to train on imagenet due to lack of GPUs. CIFAR-10 was the best they could do. So it's sometimes hard to tell whether there's intentional deceit, or just a shoestring budget. (I was surprised how much research a lot of people get done in spite of limitations.) Thanks for the correction; cheers.