5 ms·

Slightly meta, but I find its a good sign that we're back to designing and blogging about in-house data storage systems/ Query engines again. There was an explo

by pm90 1y ago

Slightly meta, but I find its a good sign that we're back to designing and blogging about in-house data storage systems/ Query engines again. There was an explosion of these in the 2010's which seemed to slow down/refocus on AI recently.

8n4vidtmkvmk 1y ago
Is it good? What's left to innovate on in this space? I don't really want experimental data stores. Give me something rock solid.
- cfors 1y ago
  I don't disagree that rock solid is a good choice, but there is a ton of innovation necessary for data stores. Especially in the context of embedding search, which this article is also trying to do. We need database that can efficiently store/query high-dimensional embeddings, and handle the nuance of real-world applications as well such as filtered-ANN. There is a ton of innovation in this space and it's crucial to powering the next generation architectures of just about every company out there. At this point, data-stores are becoming a bottleneck for serving embedding search and I cannot understate that advancements in this are extremely important for enabling these solutions. This is why there is an explosion of vector-databases right now. This article is a great example of where the actual data-providers are not providing the solutions companies need right now, and there is so much room for improvement in this space.
  - whakim 1y ago
    I do not think data stores are a bottleneck for serving embedding search. I think the raft of new-fangled vector db services (or pgvector or whatever) can be a bottleneck because they are mostly optimized around the long tail of pretty small data. Real internet-scale search systems like ES or Vespa won’t struggle with serving embedding search assuming you have the necessary scale and time/money to invest in them.
    - mdaniel 1y ago
      > Real internet-scale search systems like ES Oh, then you must have the secret sauce that allows scaling ES vector search beyond 10,000 results without requiring infinite RAM. I know their forums would welcome it, because that question comes up a lot Or I guess that's why you included the qualifier about money to invest
      - whakim 1y ago
        Would you mind putting aside the snark? I have a couple questions. How large is the corpus? I am also curious about the use-case for top-k ANN, k > 10000?
        farsa 1y ago
        Not the person you have asked but at work (we are a CRM platform) we allow our clients to arbitrarily query their userbase to find matching users for marketing campaigns (email, sms, whatsapp). These campaigns can some times target a few hundred thousand people. We are on a really ancient version of ES, but it sucks at this job in terms of throughput. Some experimenting with bigquery indicates it is so much better at mass exporting.
        whakim 1y ago
        Fair; my question was mostly in the context of ANN, since that was the discussion point - I have to assume ES (as a search engine) would not necessarily be the right tool for data warehousing types of workloads.
    - cfors 1y ago
      Sure they can handle the basic case of ANN. But ANN still doesn’t have good stories for lots of real-world problems. * filterable ANN, decomposes into prefiltering or postfiltering. * dynamic updates and versioning is still very difficult * slow building of graph indexes * adding other signals into the search, such as query time boosting for recent docs. I don’t disagree these systems can work but innovation is still necessary. We are not in a “data stores are solved” world.
- weego 1y ago
  Agreed. The only caveat to that being a global rule is: 'At scale in a particular niche, even an excellent generalist platform might not be good enough' But then the follow on question begs: "Am I really suffering the same problems that a niche already-scaled business is suffering" A question that is relevant to all decision making. I'm looking at you, people who use the entire react ecosystem to deploy a blog page.
- sieabahlpark 1y ago
  [dead]
0xbadcafebee 1y ago
It slowed down not because of AI, but because it turned out it was mostly pointless. Highly specialized stacks that could usually be matched in performance by tweaking an existing system or scaling a different way. In-house storage/query systems that are not a product being sold by itself are NIH syndrome by a company with too much engineering resources.
wavemode 1y ago
NoSQL/alternative databases became kind of a meme once people realized that 95% of enterprises can do fine with just Postgres.

by pm90 1y ago

8n4vidtmkvmk 1y ago

Is it good? What's left to innovate on in this space? I don't really want experimental data stores. Give me something rock solid.

cfors 1y ago
I don't disagree that rock solid is a good choice, but there is a ton of innovation necessary for data stores. Especially in the context of embedding search, which this article is also trying to do. We need database that can efficiently store/query high-dimensional embeddings, and handle the nuance of real-world applications as well such as filtered-ANN. There is a ton of innovation in this space and it's crucial to powering the next generation architectures of just about every company out there. At this point, data-stores are becoming a bottleneck for serving embedding search and I cannot understate that advancements in this are extremely important for enabling these solutions. This is why there is an explosion of vector-databases right now. This article is a great example of where the actual data-providers are not providing the solutions companies need right now, and there is so much room for improvement in this space.
- whakim 1y ago
  I do not think data stores are a bottleneck for serving embedding search. I think the raft of new-fangled vector db services (or pgvector or whatever) can be a bottleneck because they are mostly optimized around the long tail of pretty small data. Real internet-scale search systems like ES or Vespa won’t struggle with serving embedding search assuming you have the necessary scale and time/money to invest in them.
  - mdaniel 1y ago
    > Real internet-scale search systems like ES Oh, then you must have the secret sauce that allows scaling ES vector search beyond 10,000 results without requiring infinite RAM. I know their forums would welcome it, because that question comes up a lot Or I guess that's why you included the qualifier about money to invest
    - whakim 1y ago
      Would you mind putting aside the snark? I have a couple questions. How large is the corpus? I am also curious about the use-case for top-k ANN, k > 10000?
      - farsa 1y ago
        Not the person you have asked but at work (we are a CRM platform) we allow our clients to arbitrarily query their userbase to find matching users for marketing campaigns (email, sms, whatsapp). These campaigns can some times target a few hundred thousand people. We are on a really ancient version of ES, but it sucks at this job in terms of throughput. Some experimenting with bigquery indicates it is so much better at mass exporting.
        whakim 1y ago
        Fair; my question was mostly in the context of ANN, since that was the discussion point - I have to assume ES (as a search engine) would not necessarily be the right tool for data warehousing types of workloads.
  - cfors 1y ago
    Sure they can handle the basic case of ANN. But ANN still doesn’t have good stories for lots of real-world problems. * filterable ANN, decomposes into prefiltering or postfiltering. * dynamic updates and versioning is still very difficult * slow building of graph indexes * adding other signals into the search, such as query time boosting for recent docs. I don’t disagree these systems can work but innovation is still necessary. We are not in a “data stores are solved” world.
weego 1y ago
Agreed. The only caveat to that being a global rule is: 'At scale in a particular niche, even an excellent generalist platform might not be good enough' But then the follow on question begs: "Am I really suffering the same problems that a niche already-scaled business is suffering" A question that is relevant to all decision making. I'm looking at you, people who use the entire react ecosystem to deploy a blog page.
sieabahlpark 1y ago
[dead]

0xbadcafebee 1y ago

It slowed down not because of AI, but because it turned out it was mostly pointless. Highly specialized stacks that could usually be matched in performance by tweaking an existing system or scaling a different way. In-house storage/query systems that are not a product being sold by itself are NIH syndrome by a company with too much engineering resources.

wavemode 1y ago

NoSQL/alternative databases became kind of a meme once people realized that 95% of enterprises can do fine with just Postgres.