If C is binary or categorical, then confidence intervals don't make sense to talk about (although you could still talk about them if C is integer valued), and t

If C is binary or categorical, then confidence intervals don't make sense to talk about (although you could still talk about them if C is integer valued), and the equivalent thing is P(C|x). Unless I guess if you have a posterior distribution over distribution... The definition of a Q (e.g Q = 0.95) confidence interval a < X < b is just that P(a < X < b) = Q. It's not even necessary that the interval contain the most probable value of X, but when you actually calculate them to want to tack on extra constraints to prevent stuff like that, like minimising the length of the interval.