It's more like instead of asking for a chatbot to try and hallucinate/synthesize/guess answers like large math questions or the population of Geneseo, NY (whic

It's more like instead of asking for a chatbot to try and hallucinate/synthesize/guess answers like large math questions or the population of Geneseo, NY (which it is bad at) to introduce a computing environment [read: eval(completionText)] so it can instead translate questions into executable code or further LLM queries but with a provided context. This is currently how a number of existing tools work, both in published literature and in the wild with tools like BingChat. I have personally found this analytic-synthetic distinction to be useful. I'm also a huge fan of Immanuel Kant and I really love the idea that I can loosely prove that Frege was wrong about math problems being analytic propositions!