Privacy and security are good reasons, but I think the more common thought is "this is another tech thing that will break in some way I don't understand and can

Privacy and security are good reasons, but I think the more common thought is "this is another tech thing that will break in some way I don't understand and cannot get fixed, plus none of it will work with other IoT brands". People encounter enough tech complexity online that they have a gut-level negative reaction to having more of it in the physical world.