4 ms·
On your last point: I guess recalculating 10 million embeddings takes much longer than the 20-30 mins to re-index? Or perhaps we care because calculating the e
by rahimnathwani 4y ago
On your last point: I guess recalculating 10 million embeddings takes much longer than the 20-30 mins to re-index?
Or perhaps we care because calculating the embeddings can be done in parallel with no limit, but the indexing is somehow constrained?
- ethanahte 4y agoYeah, depending on the model, calculating the 10 million embeddings could take longer sequentially, but, as you mention, it's also an embarrassingly parallel operation. I don't think that indexing can be performed in parallel, but I may be wrong on that one.