I would assume that this is caused by emacs having some pretty complex logic to minimize screen updates to absolute minimum in order to support slow serial link

I would assume that this is caused by emacs having some pretty complex logic to minimize screen updates to absolute minimum in order to support slow serial links to text terminals which is presumably also used in graphical modes.