It's that followers are countable. "Less" and "fewer" get more mixed in usage, but "many" vs "much" is more clear IMO. "I don't have many followers" is consider

It's that followers are countable. "Less" and "fewer" get more mixed in usage, but "many" vs "much" is more clear IMO. "I don't have many followers" is considered more proper than "I don't have much followers" (but you could say "I don't have much of a following").