I'm thinking it's likely the number of supported colors and integer vs floating point precision. You'll notice that quake textures are very similar to PS1 text

I'm thinking it's likely the number of supported colors and integer vs floating point precision. You'll notice that quake textures are very similar to PS1 textures in that they are pixelated and use a limited number of colors, whereas N64 textures have more of a smooth gradient. Likely also there are differences in the lighting systems as well. This is why I think people compare Quake II or even Quake III Arena to UE1. The OG quake really was a hack just to be the first that did 6 degrees of freedom textured 3D graphics on a PC, which I think they were the first for those exact constraints. My history is a little fuzzy, SEGA certainly had them beat by multiple years on arcade boards but those were all custom, and other games that had 6 degrees of freedom were not textured. It was a busy time !