3 ms·
This analysis is bad. The embedding is generated once. Search is done whenever a user inputs a query. The cosine similarity is also not done on a single embedd
by brigadier132 3y ago
This analysis is bad.
The embedding is generated once. Search is done whenever a user inputs a query. The cosine similarity is also not done on a single embedding, it's done on millions or billions of embeddings if you are not using an index. So what the actual conclusion is, is that once you have a billion embeddings a single search operation costs as much as generating an embedding.
But then, you are not even taking into account the massive cost of keeping all of these embeddings in memory ready to be searched.
- _ea1k 3y agoI think the context was prototyping.
- gdiamos 3y agoPrototyping is one scenario I have seen this in. Prototyping is iterative - you experiment with the chunk size, chunk content, data sources, data pipeline, etc. every change means regenerating the embeddings Another one is where the data is sliced based on a key, eg user id, particular document being worked on right now, etc