Exactly. This is what the letter should have focused on. The issue is not alignment, per se. If AI can do the workmanlike stuff at breakneck speeds, academia wi

Exactly. This is what the letter should have focused on. The issue is not alignment, per se. If AI can do the workmanlike stuff at breakneck speeds, academia will decide between two options: 1) Keep the average mathematician employed doing redundant mathematics just to have capable humans around. Every now and then you get one of those mathematicians actually developing new ideas, but most departments will entirely be just for show. It's doable though. College sports is able to keep itself going along these lines (and I mean the track, softball, and polo type sports). 2) You get unemployed mathematicians who were still developing their abilities. And as a result you lose the institutions that generate the mathematicians that contribute new ideas. You get to keep the handful of exceptional mathematicians that are developing new ideas, and that's survivable if you happen to be a Fields medalist. None of this really has to do with the AI companies aligning themselves to giving credit and developing insights. If AI companies became aligned to that, then the AI's would be replacing the Fields medalists, too.