Usually, we destructively compress (mean-pooling) both the query and the document, and then compare the two compressed forms. With ColBERT, we compare first -

Usually, we destructively compress (mean-pooling) both the query and the document, and then compare the two compressed forms. With ColBERT, we compare first - at the full token level - for more detailed comparison. Then reduce the full set of comparisons to a single vector. Naturally this takes more memory and compute to do the more comparisons. The idea is it’s worth it because the more detailed comparisons lead to better results tokens —> reduced vector —> comparison Vs tokens —> comparisons —> reduced vector