> This is where compound systems are a valuable framework because you can break down the problem into bite-sized chunks that smaller LLMs can solve. Just a rem

> This is where compound systems are a valuable framework because you can break down the problem into bite-sized chunks that smaller LLMs can solve. Just a reminder that smaller fine tuned models are just as good at solving the problems they are trained to solve, as large models are. > Oftentimes, a call to Llama-3 8B might be enough if you need to a simple classification step or to analyze a small piece of text. Even 3B param models are powerful now days, especially if you are willing to put the time into prompt engineering. My current side project is working on simulating a small fantasy town using a tiny locally hosted model. > When you have a pipeline of LLM calls, you can enforce much stricter limits on the outputs of each stage Having an LLM output a number from 1 to 10, or "error" makes your schema really hard to break. All you need to do is parse the output and it if isn't a number from 1 to 10... just assume it is garbage. A system built up like this is much more resilient, and also honestly more pleasant to deal with.