Most software that uses statistical algorithms can't make guarantees about certain outcomes. For example, Netflix can't be sure that you'll actually like its re

Most software that uses statistical algorithms can't make guarantees about certain outcomes. For example, Netflix can't be sure that you'll actually like its recommendations. However, they do have aggregate measures that give a holistic picture on their algorithm's behavior. We can do the same for LLMs: have big-picture summaries that speak to their behavior in aggregate. We can also restrict their action-space by programming deterministic rules or filters on top of their output. While these algorithms will never be as deterministic as a simple circuit, there are measure you can take to restrict and understand their behavior.