Pretty much the only time I hear the M1 fans are when I'm doing something with tensorflow-metal, ffmpeg'ing or rendering video, or otherwise deliberately thrash

Pretty much the only time I hear the M1 fans are when I'm doing something with tensorflow-metal, ffmpeg'ing or rendering video, or otherwise deliberately thrashing the CPU. e.g. Minecraft (albeit modded with Sodium et al) makes it noticeably warm but no fans.