One of the emergent properties of large language models is that they allow for "in-context learning" by providing "zero-shot", "one-shot", "few-shot" training e

One of the emergent properties of large language models is that they allow for "in-context learning" by providing "zero-shot", "one-shot", "few-shot" training examples. This seems to be consistent across all sorts of transformer based models with language capabilities. Certain kinds of prompts result in certain kinds of completions and "prompt engineering" is a collection of methods related to encourage a specific completion. For example, prompts that do not contain enough facts about a desired question will result in "hallucinations", whereas prompts that have additional context programmatically added to the question will result in more reliably factual completions. I find it useful to think about this grounded in terms of the analytic-synthetic distinction from analytic philosophy. The above guide is an excellent resource! You should read it!