from the article: "we devise the following straightforward procedure: 1. Obtain a small set of manually created prompts for the task. 2. Expand the set of pro

from the article: "we devise the following straightforward procedure: 1. Obtain a small set of manually created prompts for the task. 2. Expand the set of prompts with automatic paraphrasing using a LM (e.g., GPT3) and backtranslation (see Section 3). 3. Rank the list of prompts by perplexity (aver- aged on a representative sample of task inputs, e.g. 1,000). 4. Choose the k (e.g., 3) lowest perplexity prompts. Using this algorithm, we show empirically that it is best to prioritize experimenting with the lowest perplexity prompts, as they perform better than manual prompts on average, and are more stable"