FWIW I think the only way hackers learn this stuff is by reading and then one day going "Huh I understand CPU caches [or similar]". It isn't forbidden knowledg

FWIW I think the only way hackers learn this stuff is by reading and then one day going "Huh I understand CPU caches [or similar]". It isn't forbidden knowledge by any means, so just follow your curiosity and see where it takes you. For interfaces, I would maybe start with something like the art of electronics just for the basics. If I want to understand the basics of some bus or whatever, I usually do some tactical googling alongside terms like "FPGA" to see what comes up so I don't have to make sense of a 150000 page spec.