Synthetic data is a big deal, essentially as a form of “knowledge distillation” from large models or for transforming high-quality text into training data (e.g.

Synthetic data is a big deal, essentially as a form of “knowledge distillation” from large models or for transforming high-quality text into training data (e.g. Q&A pairs). Almost everyone is using GPT-4 for this. Dunno about other domains, as it’s based on the mutability of text, relative to whatever ground truths are embedded therein. This seems less feasible for other kinds of inputs, but who knows.