My intuition for this is that a high dimensional vector with 16bits per dimension reduced to 4bits would be ok if these are effectively low cardinality dimensio

My intuition for this is that a high dimensional vector with 16bits per dimension reduced to 4bits would be ok if these are effectively low cardinality dimensions and even with 4 bits things can be distinctly mapped into the vector space. It should break down on high cardinality dimensions (say things which are only distinguished in a few significant dimensions & rest being effectively zerovalue). In otherwords, given a vector of say 12288 dimensions (GPT), a 4-bit dimension, if vectors were uniformly distributed in the embedding space, is a choice space of 16^12288. That's in 4 bits. The 16-bit space is huge. I think serious errors will crop up only if we're looking at items that cluster in a small subset 'd' of those 12288 dimensions. So at some small d, 16^d will result in vector collisions for certain type or category of inputs.