You are exactly right. Making usability and user design into a science is difficult because we'd be trying to model the human mind, which is, well, pretty much

You are exactly right. Making usability and user design into a science is difficult because we'd be trying to model the human mind, which is, well, pretty much impossible. For example, in the information retrieval community, companies like Google continue to employ a lot of people to mark search results as relevant or not. Even with all this data, it's still hard for us to quantify what is "relevant" to a user. Likewise, it is hard for us to quantify what is "usable." But even though the programmer may own this toolbox of quantitative methods and whatnot, most of them do not use it when it comes to usability. Why? Because the toolbox is quite empty. The advantage that these "snobby" designers (just like how programmers are "snobby" about their choice of programming languages and tools) have over programmers is that designers have more empathy towards the everyday man. So there is some emotion involved. Sure, there might be a science behind all of this, but since no one figured it out yet, we'll have to go use our intuition of what a good design is.