> Khanmigo now uses a calculator to solve numerical problems instead of using AI’s predictive capabilities. If you’ve been using Khanmigo recently, you may have

> Khanmigo now uses a calculator to solve numerical problems instead of using AI’s predictive capabilities. If you’ve been using Khanmigo recently, you may have seen that it will sometimes say it is “doing math.” This is when the math problem is running through the calculator behind the scenes. > We’ve upgraded parts of Khanmigo to a more capable large language model, which is the software that generates human language. The more capable large language model is called GPT-4 Turbo. Our internal testing shows an improvement in math after we made the switch. > We are beginning to test the capabilities of a new large language model called GPT-4o, and we’re evaluating other models too to see if they are stronger at math. > We’ve improved the way AI “thinks” during a tutoring session before responding to a student. We have instructed the AI to write out all the ways in which the student may have arrived at their answer. This approach mimics how a tutor in real life works with a student. We’ve found it significantly improves the quality of math interactions. > We’ve built new tools to track our progress on math. > We’re sharing math examples and learnings with others in our field so that we can learn from each other. > We’re studying the latest research papers on math performance. Sounds like most of what they're doing is related to prompting, chain-of-thought reasoning and similar, on top of a 'vanilla' foundation model. Sounds like something an ambitious student could replicate / improve upon, so given their mission, it'd be cool if they published the exact techniques they're using and their benchmark results.