Actually, the correct term is text mode. It is precisely whether the display hardware is operating in text mode or graphics mode that is the subject of discuss

Actually, the correct term is text mode. It is precisely whether the display hardware is operating in text mode or graphics mode that is the subject of discussion here. With the hardware in text mode, even without CRTC tricks, scrolling is relatively fast compared to graphics mode. Scrolling the display in graphics mode involves shifting much more data around, as does rendering individual characters. The confusion arises because people, as you have done, erroneously conflate the kernel virtual terminals with "text mode". In fact, the norm nowadays is for kernel virtual terminals to use graphics mode. It permits a far larger glyph repertoire and more colours, for starters, as well as things like software cursors that can be a wider range of shapes and sprites for mouse pointers.