4 ms·

Is there a high level explanation as to why? Best I can imagine it’s that it’s easier to improve a model than make one from scratch, and so AI training AI means

• thomashop 4y ago
  Probably because you can use the error gradients from the generative model somehow
• visarga 4y ago
  The task of generating images is harder than that of classifying, the training signal is also much denser. So it means the diffusion model implicitly learned a better visual model, and now they found a way to invert that and improve the discriminative task as well. We can do the same thing in text. A recent paper showed GPT-4 is better than most human labellers at tagging NLP datasets. I personally used it to generate samples for smaller models, basically the same thing this paper showed. You can even top that with a second LLM round to reflect on its generated data and filter the noisy examples. In the end it still makes errors. And the errors GPT makes are hard to detect at scale.
  • bagels 4y ago
    Can you give an outline of how you used gpt4 to generate samples?