You can already do that. Take your task first to GPT-3 and collect a bunch of outputs. Then fine-tune a small model on them. Works well, but you need to extend

You can already do that. Take your task first to GPT-3 and collect a bunch of outputs. Then fine-tune a small model on them. Works well, but you need to extend the dataset to cover all edge cases because the small model can't draw on the vast knowledge GPT-3 has.