> For example, it should be really easy to generate training data for a LLM using more-formal systems like Wolfram Alpha; not least by having your LLM generate

> For example, it should be really easy to generate training data for a LLM using more-formal systems like Wolfram Alpha; not least by having your LLM generate an arbitrary large list of "questions for Wolfram Alpha", then take that query and put it into WA, then attach the results to your LLM training set. I don't think this would be "really easy." I think you would quickly run into memory limits because the size of "arbitrary large" would need to be really really large to be enough training data to be able to have enough grounding for arbitrary problems (of course there are an infinite number of possible questions for Wolfram Alpha; how would you even determine what the right subset are to train on?). At least that's my educated guess.