Would a model actually perform better fine tuned on such a small data set? Seems to me that just giving it the data during inference of new stuff would net bett

Would a model actually perform better fine tuned on such a small data set? Seems to me that just giving it the data during inference of new stuff would net better results, using something like RAG.