A big part of it is learning computer architecture. If you have a solid grasp of things like how memory access works and how the CPU pipeline works, you can sta

A big part of it is learning computer architecture. If you have a solid grasp of things like how memory access works and how the CPU pipeline works, you can start to get a better mental picture of how fast a particular piece of code will run based on what instructions the CPU is executing and what the memory layout is. There are textbooks and classes on computer architecture. Funny enough, many of them use MIPS, which is the architecture used in the N64. Optimizing for N64 also requires understanding how the RCP works, which is a separate topic.