I think it more to do with the historical lifestyle and culture - Swedes, Finns and Norwegians have been integrated into the continental mainstream of settled

I think it more to do with the historical lifestyle and culture - Swedes, Finns and Norwegians have been integrated into the continental mainstream of settled agriculture and Christianity for centuries while Saami held out as a more nomadic, hunter-gatherer culture for most of modernity.