4 ms·

AI devs created GPT-3-beating machine with 99.9% fewer parameters

• rvz 6y ago
  > Their approach fuses a training technique called PET (pattern-exploiting training) with a small pre-trained Albert model, letting them create a system that “outperform GPT-3 on SuperGLUE with 32 training examples, while requiring only 0.1% of its parameters.” Good. A start for once to challenge GPT-3 but using significantly less parameters. We need more challengers like this.