4 ms·
Creating Artificial Mechanical Turks with Pretrained Language Models
- gwern 5y agotldr: using large expensive models to auto-label data to train small cheap models. (I find the 'mechanical turk' framing here to be much more confusing & misleading than clever or helpful, and to make it harder to compare to the considerable number of other papers on using language models to generate new datasets & do self-distillation.)
- deleted 5y ago[deleted]
- unbanned 5y ago>to make it harder to compare to the considerable number of other papers Naturally. There's a reason AI papers are not published in respected journals a significant proportion of the time.
- cuteboy19 5y agoIsn't this just "transfer learning"? Surely there has to be a better way than "momma bird pukes into baby bird's mouth" type of training
- gwern 5y agoNo. Transfer usually means using the same NN model (eg. GPT-3 checkpoints being retrained on Github and then called 'Codex'), or possibly some sort of distillation/sparsifying approach. This is about auto-generating training data, maybe not even meant to be used by a neural net at all.
- funstuff007 5y agoHow far are we away from using open.ai to earn money on Mturk? Or is this like the fusion problem?
- stubish 5y agoWould this even be fraud if it worked, or is it the expected outcome of this sort of market?
- deleted 5y ago[deleted]