3 ms·
In general GPT3 is not SotA on (any?) classification task, did you just not have enough data to fine tune a discriminative transformer model? Inference should b
by ivalm 6y ago
In general GPT3 is not SotA on (any?) classification task, did you just not have enough data to fine tune a discriminative transformer model? Inference should be cheaper with a smaller transformer/also less lock-in.
- neural_thing 6y agoI can't go into too much detail here about why we couldn't do that, but one aspect that we found VERY useful is that GPT-3 could draw on real world knowledge not present in the dataset to enhance the results.