4 ms·
"More complex workloads such as joins". How far we have fallen.
by jurip 2y ago
"More complex workloads such as joins". How far we have fallen.
- esafak 2y agoYou joke, but you would not blame redis for not doing joins. Vector databases are really more like caches that do similarity search. Maybe we should call them vector stores.
- flawsofar 2y agoI can see uses for joining vector queries. Give me all the people who are attracted to people like this one. Boom, embedded.
- esafak 2y ago> Give me all the people who are attracted to people like this one This can be handled with a simple batch query rather than a join, even if you want to query for multiple users, fetching all of their related users. In qdrant, for example: https://qdrant.tech/blog/batch-vector-search-with-qdrant/ https://qdrant.tech/blog/batch-vector-search-with-qdrant/
- Culonavirus 2y agoThis is what gets me more and more with LLMs and their "ability to code". A join is a bedrock of a database. Always has been. Always will be. All LLMs are an absolute shitshow when it comes to any SQL logic involving joins. It's hilariously bad. But what can I expect from an LLM when apparently even people consider a JOIN a "more complex workload"? What a joke.
- sgarland 2y agoEh? SQL is one area where I’ve found LLMs – at least, ChatGPT – to be surprisingly decent. I’ve even had it translate relational algebra back and forth.
- randomdata 2y agoObviously it is a more complex workload. Consider a simple database like a hash map. Accessing one record in that map is easy, but it should be quite apparent that you need quite a bit more logic if you want to join that record with other records. And that's not far off how current generation vector stores are implemented. They are not designed for efficient joins (nor aggregations) and changing that, like above, does require a more complex workload. There is no free lunch in engineering, just a choice in which trade-offs you are willing to pay for.