There is nothing wrong with looking up to people, or to have idols. It's a large part of inspiration in my experience. Maybe the better phrase would be not to

There is nothing wrong with looking up to people, or to have idols. It's a large part of inspiration in my experience. Maybe the better phrase would be not to meet your idols.