When your task is too nuanced to be described in a prompt and a few demonstrations you need to use one of the fine-tuning scripts to bake into the model much mo

When your task is too nuanced to be described in a prompt and a few demonstrations you need to use one of the fine-tuning scripts to bake into the model much more supervised data. That's still doable for AI Engineers, prepare data, call LoRA fine tuning script, deploy model. The problem is having a good supervised training set. But recently it can be generated with LLMs + plugins and language chains, basically amplified LLMs. In other cases you need to collect human preference data and apply a different kind of fine-tuning. Still mostly dataset curation and iterative model building, something and AI Engineer should be able to do. Working with LLMs is very different from building neural nets like in 2017. Tons of those old skills are not needed anymore, so many of the old tasks are basically solved at human level. And a whole new set of problems appeared.