> totally wrong 20-50% of the time That to me feels like there's some prompting improvements that you could do. It could be that your problem is just harder fo

> totally wrong 20-50% of the time That to me feels like there's some prompting improvements that you could do. It could be that your problem is just harder for LLMs than ours, but 20-50% of the time isn't what we observed after several prompting changes. The other thing is that we do regularly get outputs that are "mostly correct" and we can actually correct them manually, so while the model may have a higher fault rate, the actual end-user experience has a lower one.