I would be worried if someone felt that "being intelligent" was a primary ability of yours - not because others think it reflects badly on your other abilities,

I would be worried if someone felt that "being intelligent" was a primary ability of yours - not because others think it reflects badly on your other abilities, but rather because it isn't an important ability in itself and I expect that a part of being intelligent is realising that intelligence isnt important in itself, it is just an ability that lets you make better use of your other abilities, and that can also act as a proxy for some other abilities (as in - you can use intelligence to simulate other, useful abilities). Stereotyping, there is a group of people who are convinced they are intelligent and that it is important to be intelligent - those guys are usually not actually that good at abstract problem solving. Someone who truly is good at it tends to be more laid back about it, and wouldnt consider it a big deal or important.