3 ms·
Thank you for the link. I'm not sure I'm convinced by OpenAI's claim that their model performs zero-shot learning. It depends on what exactly do they mean by z
by YeGoblynQueenne 7y ago
Thank you for the link.
I'm not sure I'm convinced by OpenAI's claim that their model performs
zero-shot learning. It depends on what exactly do they mean by zero-shot
learning. My understanding, from reading the linked article (again; I remember
it from when it was first published) is that, although their GPT-2 model was
not trained on task-specific datasets there was no attempt to ensure that
testing instances for the various tasks they used to evaluate its zero-shot
accuracy were not included in the training set. The training set was a large
corpus of 40 gigs of internet text. The test set for e.g. the Winograd Schema
challenge was a set o 140 Winograd schemas (i.e. short sentences followed by a
shorter question), so it's very likely that the training set had comprehensive
coverage of the testing set, for this task anyway. I don't know about the
other tasks.