I don't doubt the input and output modules will need some work, but in the grand scheme of things probably not that much. The big missing piece imo is just mode

I don't doubt the input and output modules will need some work, but in the grand scheme of things probably not that much. The big missing piece imo is just models with better world models—and it doesn't look like the scaling wars will stop anytime soon, nor does it look like size will stop helping, at least not anytime soon. (https://www.gwern.net/newsletter/2020/05#baking-the-cake https://www.gwern.net/newsletter/2020/05#baking-the-cake)