To this point, guaranteeing that your fine-tuning doesn't actually weaken the hallucination-resistance of state-of-the-art base models, by overfitting to specif

To this point, guaranteeing that your fine-tuning doesn't actually weaken the hallucination-resistance of state-of-the-art base models, by overfitting to specific examples in your training set, is a really hard thing to measure. Unless you have robust ways of validating LLM output so that you can do real cross-validation, few-shot prompts, where you can instruct the LLM exactly how much you want it to generalize from those examples, may be far better at achieving many goals.