I guess that's because the flying pilot might become incapacitated at any time (i.e. stroke, heart attack, ...), and then the other pilot has to take over as qu

I guess that's because the flying pilot might become incapacitated at any time (i.e. stroke, heart attack, ...), and then the other pilot has to take over as quickly as possible, and an additional switch that they have to locate and flip would need more time than just moving the controls?