3 ms·
This doesn't work if I use a reasoning model like o3, which does not allow setting max_tokens. Without reasoning, how can I be SURE a number is even?
by ukuina 1y ago
This doesn't work if I use a reasoning model like o3, which does not allow setting max_tokens.
Without reasoning, how can I be SURE a number is even?
- hartator 1y agoMultiple models?
- parpfish 1y agoI think before you would deploy this to prod, you should wrap it with a few guardrails to make sure it’s not hallucinating. Pretty simple — just take the output from the llm and see if it agrees with a simple mod2 operation. Of it agrees, return model output to the user. Otherwise do a couple of retries with different prompts.
- mrheosuper 1y agomod2 is outdated and has not been updated for years, nobody uses it anymore