> The more an LLM talks the smarter it gets I have a blog post in coming on this topic, but yes, this is right. My method is to first get the LLM to answer th

> The more an LLM talks the smarter it gets I have a blog post in coming on this topic, but yes, this is right. My method is to first get the LLM to answer the question, and THEN feed the answer back the LLM to extract the answer using constraints + grammar/logit bias/regex to parse the answer. Previously, I constrained to a single true/false token, which worked, but fails on complex queries. So I split the decision making into a "justification" portion[0], and a "parsing" portion. I found that even crafting the prompt matters here, if you start with or end with, "It's very important to the response includes 'The answer is:'", then the model will lead with that response or only reply with that response. So I put it in the middle of the prompt, and end with with a request to justify the response. As a result, most models will reason their way to the answer, and then end with 'The answer is:'. https://github.com/ShelbyJenkins/llm_client/blob/e3c4a860dda5513b3459b2af847fc2d90014d391/src/agents/deciders/custom.rs#L9C1-L10C1 https://github.com/ShelbyJenkins/llm_client/blob/e3c4a860dda...