Codegen is an area that seems to have the best breakout performance compared to the big FMs. In hindsight it should be obvious given that openai created Codex.

Codegen is an area that seems to have the best breakout performance compared to the big FMs. In hindsight it should be obvious given that openai created Codex. However, I am skeptical that smaller and more focused models will do well for more general tasks. I’ve found gpt-4 to just be flatly excellent for tasks that involve emitting a custom DSL and customer-provided business context (the latter being something we cannot train for, not without immense expense).