That's kind of going in a different direction. The big picture is that LLMs have until this point gotten better and better from larger datasets alone. See "The

That's kind of going in a different direction. The big picture is that LLMs have until this point gotten better and better from larger datasets alone. See "The Bitter Lesson". But now we're running out of datasets and so the only way we know of to improve models' reasoning abilities and everything, is coming to an end. You're talking about fine tuning, which yes is a technique that's being used and explored in different domains, but my understanding is it's not a very good way for models to acquire knowledge. Instead larger context windows and RAG works better for something like case law. Fine tuning works for things like giving models a certain "voice" in how they produce text, and general alignment things. At least that's my understanding as an interested but not totally involved follower of this stuff.