I have been on both sides of this. I worked for a time at a startup with decent salary in a generalist role, such that sometimes I did things directly related

I have been on both sides of this. I worked for a time at a startup with decent salary in a generalist role, such that sometimes I did things directly related to my skillsets and interests, and other times got to pick up entirely new skillsets. This broadened my horizons significantly and put me in a decent position financially, though nowhere near FIRE. After that I took a pay cut to pursue something fun and interesting (moving to a new country to help an old colleague establish a research group). This gave me a chance to work on my own projects and passions but also pulled me out of the fast-paced and wide-reaching environment that you find in a startup. The freedom is great but the cash compensation hurts a bit, so the rest of the package has to make up for it -- most days it does. All things considered, you can find interesting things to do (and people to do them with) in most positions. Taking one with higher compensation and comparable commitment of time and energy is rarely a mistake. Especially if that compensation makes you financially independent with reasonable probability.