The "trick of X" you speak of is that it figures out X window ID of the terminal, then asks the X server the coordinates of that window, then creates borderless

The "trick of X" you speak of is that it figures out X window ID of the terminal, then asks the X server the coordinates of that window, then creates borderless windows at the correct coordinates above that window to show images. However, in addition to this "trick of X" mode, links also supports rendering its own X window; with proportional fonts and everything; no terminal emulator needed. However, the graphics system isn't based on X; X is just one backend; it can also render to the Linux framebuffer, so it doesn't even need X11 to be running, it can render full graphics straight in your TTY.