> I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedbac

> I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedback. I guess if we ignore pretraining, don't sample-efficient fine-tuning on carefully curated instruction datasets sort of achieve this? LIMA and OpenOrca show some really promising results to date.