I skim read the paper, the key bits I saw were: > First, given an LLM and an unlabeled dataset, we prompt the LLM to generate output labels along with rational

I skim read the paper, the key bits I saw were: > First, given an LLM and an unlabeled dataset, we prompt the LLM to generate output labels along with rationales to justify the labels > Second, we leverage these rationales in addition to the task labels to train smaller downstream models. This doesn't sound like online use of LLMs to me, but rather using them to pre-generate augmented examples. At the end of the paper is this table: Dataset Train Validation Test e-SNLI 549,367 9,842 9,824 ANLI 16,946 1,000 1,000 CQA 8,766 975 1,221 SVAMP 720 80 200 Let's do a back-of-envelope calculation: OpenAI costs "gpt-3.5-turbo: $0.002 / 1K tokens" let's say each rationale consumes 250 tokens ...that's $275 to augment 550M examples in e-SNLI dataset Even if that estimate is quite far off in terms of token count, it seems like it'll only cost a few thousand dollars at most? GPT4 costs 10x as much. It's more than I personally would ever want to pay as a curious bystander/hobbyist, but it doesn't seem like a serious "moat" to competitors.