17 ms·

Every database will become a vector database sooner or later

232 points by nextworddev 3y ago

deleted 3y ago
[deleted]
muratsu 3y ago
Supabase has pgvector extension and that’s enough for my limited RAG use cases. I dont really need to use anything beyond postgres. On the other hand, enterprise might find it easier/cheaper to buy a second db than migrating their existing db to whatever the latest version. I dont think it’s as simple
- weird-eye-issue 3y ago
  Exactly. We use Supabase too but are at a scale where it just made sense to use a second, dedicated vector db (Pinecone) than to bloat our Postgres db that has a completely different workload
  - crazy_marksman 3y ago
    How do you deal with security and access control across postgres and pinecone?
    - weird-eye-issue 3y ago
      We use Cloudflare Workers for our API and just handle auth calls by checking the JWTs with Supabase and caching it. So we already had the necessary auth setup to do this. For basic CRUD we use the Supabase endpoints directly but none of that involves querying a vector db :P
  - BoorishBears 3y ago
    Bloat your DB... or pull in an entirely new vendor and bloat your entire operational outlay. I'd really love to know what kind of insane scale justifies that tradeoff...
    - hobs 3y ago
      Don't forget your latency!
      - weird-eye-issue 3y ago
        They are separate systems. We don't touch Postgres for the same code that needs to access Pinecone.
    - weird-eye-issue 3y ago
      That's a very exaggerated way to look at things lol. Nothing got bloated at all in this process, we are just using the right tools for the job. I'm a solo founder and the only backend developer. I can assure you this decision only made my life easier by choosing the correct tech from the get go.
      - BoorishBears 3y ago
        Nothing exaggerated, your comment implied scale was your justification: in which case there'd better have been some crazy high load that just brought the tool you already had to its knees to justify paying an additional closed source platform and manually having to pipe data to it in addition to your main data store. Of course if I sounded incredulous it's because I didn't think you had that scale, and it sounds like I was correct?
        weird-eye-issue 3y ago
        No, you're not correct. We have over $2M ARR and with the amount of data we are storing it would be downright stupid to use Supabase. We don't also "pipe" our data to Supabase, we use a couple different data stores depending on the best use case. For example we also use R2 and Durable Objects. Just because you have a hammer doesn't mean everything is a nail.
        BoorishBears 3y ago
        Well maybe we have different definitions of scale: I think my team spends about $2M a month on compute, so we don't pride ourselves on randomly pulling in new vendors.
- natsucks 3y ago
  recently ditched Supabase for Weaviate. I was tired of the python bindings not keeping up, no hybrid search, slower search algorithms. Also Supabase has a lot of features that i just don't need.
esafak 3y ago
> It genuinely makes sense for incumbent database players to offer vector search, because that eliminates unnecessary data movement to separate vector databases. Co-locating vectors and original documents also reduces latency. Yet OLAP databases continue to thrive alongside OLTP databases, the nascence of NewSQL hybrid (HTAP) databases notwithstanding. Different needs dictate different design choices for optimality.
- appplication 3y ago
  It’s interesting that I never considered why OLTP and OLAP are basically orthogonal technologies. Are there any major players that have an integrated solution for both? I guess it makes sense because the infra is so different, but I’m not sure whether it need be.
  - barrkel 3y ago
    Effecient OLAP queries need a different shape of data - some combination of columnar storage for efficient scanning, and roll-up tables with pre-aggregated measures. Even in an integrated scenario, behind the scenes there will need to be a bunch of copying to transpose and / or refresh roll-ups.
    - taeric 3y ago
      The pre-aggregation is one too many try and skip. People seem to think they can build a single schema to rule all things, and then assume they can quickly calculate any aggregation on demand.
  - esafak 3y ago
    As far as the big players are concerned, Google offers AlloyDB (https://cloud.google.com/alloydb https://cloud.google.com/alloydb) while Amazon offers Aurora (https://aws.amazon.com/rds/aurora/ https://aws.amazon.com/rds/aurora/)
    - anentropic 3y ago
8n4vidtmkvmk 3y ago
I just googled this now if MariaDB offers a vector search and the first hit is a stack overflow question from me in 2014. If they jumped on it then they could have been ahead of this AI business but noooo...
EdwardDiego 3y ago
Sure, some companies will use it. Other companies will continue to use specialised focused tools. It's why data engineering is a thing in our industry. We move and prepare data for a set of tools, and we pay good money to do so, because we believe we derive value from those tools. Let's say MySQL offers it, anyone already using MySQL is likely to fence the MySQL instance(s) focused on vector stuff off for various reasons (resilience, different read/write patterns, security, etc.) MySQL as the (imaginary) basis only offers some transferable skills, because this DB will require different care and feeding. Like the difference between Postgres and PG with cstore_fdw, similar, but sufficiently different.
shri_krishna 3y ago
The one DB fits all approach only works when the size of the database is really small and never grows. Imagine you have 100 customers. Each customer generates, on average, a million 1536 dimension vector embeddings (considering OpenAI Ada dimensions which is the most popular right now). That is 6GB (1536 x 4 bytes per dimension for f32 x 1000_000) of just embeddings PER CUSTOMER. If you use HNSW it will take at least that much of RAM if not more. If you use PQ (and variants) you can reduce the size of index in RAM to say 512MB-1GB per customer. It is still quite a lot of memory requirement. That is just the way it is and there is no way around it. Now imagine you are using that database for storing transactions and other day to day business ops that will still be storing millions of records but with small indexes. This would have ideally only required a single DB instance with a replica for redundancy. Now if you integrate Vectors into the equation, you will have to needlessly scale this DB both horizontally and vertically just to maintain a decent query/write performance to your DB (which would have ideally been extremely fast without embeddings in the mix). You will eventually separate the embeddings out as it makes no sense for the entire DB to be scaled just for the sake of scaling your embeddings. I am not even accounting for index generation for these vectors which will require nearly 100% of all CPU cores while the index is being generated (depending on type of ANN you are using) and which in turn would slow your DB to a crawl.
- beoberha 3y ago
  Exactly - vector indexes are so different than traditional RDBMS B-Tree or LSM Tree indexes that it doesn’t make sense to use the same store for both unless it’s basically a toy app. Someone makes the example in another comment, but it’s analogous to OLTP vs OLAP
  - Foobar8568 3y ago
    I don't even want to imagine the workload on a high txn OLTP mixed with OLAP access pattern. IMHO If you can, you don't need OLAP in the first place.
    - trashtester 3y ago
jsemrau 3y ago
The benefits of using specialized vendors like pinecone is about their offering a combination of performance (clustered/loadbalancing), fast/effective algorithms, and data storage.
mlfia 3y ago
Curious how this will work in practice as vectors are specific to a given embedding model, and could be domain-specific for better results. Could it lead to industry standard embedding models, with regular (costly) upgrades?
- bambax 3y ago
  I'm wondering the same thing. Standardization would be interesting but I wouldn't bet on it. Maintaining different vector columns for different models might work well?
ooJeesh9fidujoh 3y ago
The reason everyone's rushing to build vector databases is because they've tried to store vector data in a scalar database previously and realized they're hot garbage for the workload. You should probably try it too before blogging about it.
- redwood 3y ago
  That's not been my experience with distributed technologies. Are you referring to postgres on an oversaturated system?
aiappreciator 3y ago
It is true that every major DB ventor, SQL or not, is smashing the AI/vector keyword on their front pages. In Elastic for example, their vector capabilities have gone from laughable to respectable in a year. Its a lot simpler to just use one DB instead of many. But a question for true DB experts here: 1. Is there any real advantage to building a dedicated vector DB from scratch? 2. Is vector DB something that can be just 'tacked on' to a normal DB with no major performance penalties? We know from history, that data warehouses are genuinely different from databases, and cloud data warehouses are overwhelmingly superior to on-prem ones. So that emerged as a distinct, enduring category with Snowflake/Databricks/Bigquery.
- jamesblonde 3y ago
  Data warehouses are columnar stores. They are very different from row-oriented databases - like Postgres, MySQL. Operations on columns - e.g., aggregations (mean of a column) are very efficient. Most vector databases use one of a few different vector indexing libraries - FAISS, hnswlib, and scann (google only) are popular. The newer vector dbs, like weaviate, have introduced their own indexes, but i haven't seen any performance difference - Reference: https://ann-benchmarks.com/ https://ann-benchmarks.com/
- charcircuit 3y ago
  The trade off that you are interested in isn't about storing vectors, but rather about whether an index should be a part of the DBMS or external to it. Some advantages of having a separate index is that it can work with different backends, it can be independently scaled, and it can index data for more than 1 database server. Some disadvantages are increased latency, increased complexity, and distributed system problems.
- redwood 3y ago
  Single node database systems that are not horizontally scalable and that are not built on a distributed system foundation (e.g. Postgres) will certainly have scaling bottlenecks if you just add more and more complexity to the workload... however many modern database systems are built on a distributed system foundation with horizontal scaling and the ability to independently scale different constituent parts of the backend.. these engines should have no problem
loondri 3y ago
I think the move towards vector databases might be more hype than necessity. Traditional databases, when properly optimized, can handle vector data for many use cases. The push for specialized vector databases could be re-evaluated in terms of efficiency and cost-effectiveness compared to optimizing existing scalar databases.
- avereveard 3y ago
  Well you could store numbers all fine, but indexing vectors for similarity queries seems fairly recent and not all that widespread in the transactional world. As the traditional db move forward in the space the need for dedicated vector databases will likely shrink, except for some very specific implementation that offer unique enough features (I.e. deeplake does vector search over object storage, which is very convenient for certain specific scenarios)
  - sgu999 3y ago
    sqlite has r-trees for instance [0]. Could it be good enough for most use cases? If it's to query a knowledge base for instance, a couple dimensions should be sufficient. With the added benefit of being able to query your data in other ways. [0] https://www.sqlite.org/rtree.html https://www.sqlite.org/rtree.html
    - mattashii 3y ago
      r*-trees work doesn't work well when the number of dimensions stored in the index is much higher than the logarithm of the number of indexed entries, and this is a prevailing property of divide-and-conquer spatial index types when the keyspace is divided based on a single dimension at a time. As vectors regularly have 100+ dimensions, normal spatial indexing methods applied to vectors wouldn't be very efficient for anything with much less than 2^100 index entries; which is quite suboptimal for most datasets that you would want to have indexed.
      - KirillPanov 3y ago
shaburn 3y ago
Postgres and Mongo support it. ElasticSearch incorporates vector search. I have failed to identify an objective difference(beyond marketing nomeclature) between any pure play vector solution after exaustive research.
andreer 3y ago
[dead]
_pdp_ 3y ago
I do agree with the article that this feature will be more or less available in all DB types. Vector databases are a gimmick at the moment. Ultimately conversational AI agents should be able to extract information from a diverse set of sources with a diverse set of tools. The approach that is currently taken is hit-and-miss at best. How often do you searched something and the first result happens to be the thing you are looking for? Why should it be any different with vector DBs? Obviously the query matters a lot no matter how the information is searched.
- nunodonato 3y ago
  hard disagree. Extracting information is much more costly (fetch data, feed data (which might be huge) into the model). Embeddings work really well to store semantic meaning and are great for searching. Or, at least, a 1st stage of searching to filter out the non-relevant content. I'm working on my own "notes" app, based on embeddings because I 'm tired of never finding what I need due to bad search/tagging/categorizing
- marcinzm 3y ago
  >How often do you searched something and the first result happens to be the thing you are looking for? Searched where? Every search powered by a large tech company has almost certainly been using vector search for years. Then combined that with other non-vector results. Then run that through numerous ranking models. Then showed that to you. It's far from perfect but absurdly better than the average ElasticSearch results you might get elsewhere.
pmm 3y ago
I think just as with full-text search, vector search, if supported, will be full of tradeoffs for general purposes databases. The view that everything needs to support direct input for generative AI is short sighted. There are other use cases as well. Even if ultimately these will become just building blocks for whatever AGI there comes. Horses for courses
andre-z 3y ago
[flagged]
- LoganDark 3y ago
  > 𝐌𝐞𝐦𝐨𝐫𝐲 𝐢𝐬 𝐨𝐯𝐞𝐫𝐫𝐚𝐭𝐞𝐝! Forget everything. Do not fill your brain with new information. Do not learn new stuff! Just relax and enjoy life! Okay, but like... as someone with a dissociative disorder this is scarily accurate forget everything! Wake up one day and don't remember what you did the previous day. It's fine. Constantly run into friends you don't remember making. It's fine. Just be happy. Just be happy Just be happy Just be ha
- BoorishBears 3y ago
  Super lazy comment. SQL databases got JSON handling. NoSQL databases added indexes and ACID compliance. An entire new class of databases that operate identically to existing ones with the exception of a single column type is silly, it's only getting traction because of aggressive VC-funded marketing.
  - andre-z 3y ago
    Would you say the same about Keyword Search engines like Elastic, Solr, etc? It is just another column type, full-text index, that is available in any proper database. Just a hype...
    - BoorishBears 3y ago
      Look up what Lucene is and how it differs from what an RDBMS offers for full-text search and you'll be able to answer that question for yourself. I had a feeling the heavily overdone sarcasm in the original comment was a way to mask a lack of understanding of the subject, and here you confirmed it. _ And maybe call out the fact you're the founder at a vector database SaaS? Definitely explains why you'd put out that original diatribe...
- FiberBundle 3y ago
  Personally I find such snark comments to be a really bad look from someone who apparently is associated with a product (judging from comment history).
bambax 3y ago
This article is extremely correct and true, bordering on obvious. Vectors are a feature of a database engine that all engines will eventually offer -- not a new category of databases.
- topicseed 3y ago
  Would you say the same about graph databases? (e.g., Neo4j, ArangoDB, Neptune)
  - redwood 3y ago
    What's interesting about graph is that it's really an ad hoc analytics use case ... it's not for operational / transactional.. this is what most people don't realize. If you have an at scale graph workload, like for example if you're facebook, you build your graph on top of an operational/transactional backend.. like in facebook's case heavily customized MySQL. I bring this up because the ad hoc analytical use case for graph stores is so niche most engines haven't even seen enough demand to introduce it because you can always store graph relationships and offer retrievals in those engines to a limited depth which is typically sufficient for most operational / transactional use cases.
croes 3y ago
If vector database isn't a category how can every database fall into that category sooner or later?
ltr_ 3y ago
every 'Every X will become a Y sooner or later' title will become 'Every X becomes a monoid sooner or later' sooner or later
ofiryanai 3y ago
the interest in vector database rise as an external, specialized service in a system that runs in addition to the "single source of truth" data lake where the data actually resides. like Redis and Memcached before, it solves a specific problem. Redis started to act like a fully fledged DB (with weird persistency method and guarantees) only after it was really wide-spread. sure, every DB will support vector and every vector search engine will act like a DB. But that's missing the point that vector search is an expensive problem with tradoffs that justify a specialized design btw I'm working in a DB startup - https://hyper-space.io/ https://hyper-space.io/
- kordlessagain 3y ago
  We recently added vector support to FeatureBase: https://cloud.featurebase.com/ https://cloud.featurebase.com/. What's interesting about using it with our existing b-tree storage layer is that I can now do set operations on the record's stringset fields before I do the similarity calculations on the vectors. I've been experimenting with laying out the vector space in a b-tree structure using this technique along with keyterm extraction, allowing for a type of semantic graph to be applied by the LLM to the vector space.
spandextwins 3y ago
Larry?
gk1 3y ago
Go ahead and try to load a billion embeddings into Mongo or Elastic. The author says this will be "faster, cheaper, and simpler..." Will it? (Disclaimer: I work for Pinecone, so obvious bias ahead but also perspective of 3 years since launching the Vector DB category and actually seeing billion-scale vector search deployments.) > Basically, having separate vector DBs can add to cost and complexity. Imagine you were a MongoDB shop, with over 500m documents stored cross-region. If you are using a separate vector DB, say Pinecone, that may require moving potentially billions of embeddings between two databases, cross regions. This costs a lot, not to mention complex, since you are responsible for generating the embeddings... It’s faster, cheaper, and simpler if one database (Mongo, Elastic) just supported vector search. If you want, say, 100ms search latency on just 100M vector embeddings in Elastic that'll already cost you $12,600 per month at minimum. And if you regularly write new or updated data to the index then your latencies will creep up until eventually you have to run a "force merge" which will grind your vector search to a halt for several hours (so much for easy and simple). I don't know how much it is on Mongo but given that it's bolting on the same vector index I would guess it's in the same ballpark. The cost grows sublinearly with more embeddings. (Pinecone is around 60% less than that, and will be even less soon.) The suggestion that having "billions" of embeddings in a traditional DB is easier and less costly shows you exactly why you should run your own tests and see for yourself. When traditional database companies bolt-on vector indexing libraries such as HNSW[0] on top of their existing architecture, it's to meet demand from their existing users that have a relatively basic need for vector search. For very basic and small-scale use cases, like <10M vectors with a relaxed data freshness requirement, you should just use whatever is the most convenient. Sometimes that's Pinecone, and sometimes that's the database you already have. (And if your current DB doesn't offer basic vector search, just wait two days). When it comes to larger scale, like 100M+ vectors, if you want any hope of meeting performance, cost, and data freshness requirements then you should look at a purpose-built vector database. As GenAI workloads start to enter production and scale, a lot of people will see find this out the hard way. This has been true for every unique data structure and querying pattern for the past 40 years and it’s true for vector embeddings and vector-based retrieval. You can't blame the proliferation of different database types on hype and VC funding alone. But don't take my word for it either. Go and run some tests that resemble your production workloads, then do what makes sense for your use case! [0] https://www.pinecone.io/learn/series/faiss/hnsw/ https://www.pinecone.io/learn/series/faiss/hnsw/
jbird11 3y ago
Lots of focus on RAG here, and rightfully so, but I feel an overlooked benefit to vector databases is the novel visualizations they provide. To be able to plot qualitative data onto a 2d graph with tsne reduction provides a new method with which to draw insights from. I think many companies would benefit from such a visualization tool, especially those in qualitative research.
- fhaltmayer 3y ago
  Unfortunately that visualization doesn't really work with vector dbs. Vector dbs normally split their data into separate segments and build indexes on them separately. There is no one overarching index but rather many small ones that are searched in parallel. In addition to this, such a high compression to reduce it down to 2d/3d ends up becoming a giant blob without too much information.
  - jbird11 3y ago
    Maybe I'm misunderstanding what exactly a vector db is. Let's say you have a chatbot and stored in its database is the usual info like session id, timestamp, message etc. To "vectorize" this db then would be to vectorize all the messages. Is this too simple an understanding? Once the db has been vectorized then we can do semantic search on the messages and create more informative graphs based off the semantic similarity for messages within a given timeframe or other criteria.
mattew 3y ago
Maybe I’ve been using PostgreSQL too long but when faced with the choice of adding vector support to PostgreSQL or using a new technology, my first choice was to start with the PostgreSQL addition. I’m not criticizing the specialized case for a true vector database, but for most workloads I agree that the big database players will be the right choice for many users.
- rpcope1 3y ago
  I'm in the same boat. For things that aren't huge scale, it's almost easier to find an extension or otherwise beat Postgres (or SQLite or Percona MySQL) into submission for your use case. Timescale is a really good example...I was really impressed how good the performance was for bigish (1 TB+) real time scientific time series data, even on a cheap Amazon Lightsail instance.
- regularfry 3y ago
  Pgvector's 2000-dimension limit on vector indices is annoying. There are workloads I want to push at it which are in the 5000+ range, so there's an extra dimensionality reduction step I need to build. My starting point for pretty much any storage problem is "have you tried throwing it in postgresql" but this is one reason pulling something else off the shelf might be a good idea.
ramoz 3y ago
Implementing vector DB architecture isn't complex (so I'm bear on commercial vector dbs as a net value add especially since they all wrap some oss ANN solution) You pair a vector db with a metadata store (can be anything, but ideally you want low gravity with the vdb and disk for fast retrieval... i.e. leveldb, sqlite equivalents... or hell a traditional db - and the author is right traditional dbs dont need to work too hard to create a ANN extension) In general, the greater challenge is the data engineering and overhead of managing retrieval stores (in terms of the data-integration/model pipelines) ... so I'm bull on the solutions addressing opportunities here.
vinay_ys 3y ago
The argument that combining traditional database and vector database into one because it reduces data movement doesn't compute for me. Firstly, even for non vector data, read/write transactional database vs read-optimized store purely for fast serving are already markedly different. Then, the shape of data that is used to generate embeddings is markedly different than the shape of data that is ready to transact or serve. So, no matter where it is stored, it has to leave that store, get transformed and enriched and then run through an embeddings generator (ML inference). Then, it has to be stored in a manner that is optimal for retrieval ranking. If you are doing ANN that's one thing, but if you are doing attribute based filtering while retrieving and you wish to accelerate it through GPU to do more exhaustive search, that's another thing altogether. All these lead to fairly sophisticated optimized implementation. Sure, a singular database product solution that has all these different optimized engines can emerge over time but surely it is too early today to converge like this.
- cheeseblubber 3y ago
  The problem is maintenance of a separate database and operationally it would be more work. For us who want to use embedding features having a separate database just for the embedding means now we have to keep data across multiple databases in sync and to maintain
- m00x 3y ago
  > So, no matter where it is stored, it has to leave that store, get transformed and enriched and then run through an embeddings generator (ML inference). Same for pretty much all the data in your database. It comes from an app, that does validation/editing/transforms. The benefit is that it's all together, can be atomic, only requires one query to get, update, and delete. Vector databases are just normal databases with a vector index. There's no reason for you to have a specialized DB for it. Also, embeddings aren't inference, it's a token lookup. There's no forward pass.
eggy 3y ago
As a lover of array languages, I remember being excited to read a futurist article on vector processors and programming languages. It was written right before Wes McKinney worked on Pandas (the J programming language influenced him), and I thought J/APL or another array language was going to explode. J has Jd, in which J is fully integrated. This did not come to pass (yet). No matter, I still enjoy array languages anyway. There's a new array language, uiua[1], that is a mix of array and stack concepts with a good standard library including audio and graphics. [1] https://www.uiua.org/ https://www.uiua.org/
- beanjuiceII 3y ago
  uiua looks like a perl programmer went mad
  - nwsm 3y ago
    It looks like array/stack Brainfuck. Previous uiua discussion: https://news.ycombinator.com/item?id=37673127 https://news.ycombinator.com/item?id=37673127 Brainfuck: https://en.wikipedia.org/wiki/Brainfuck https://en.wikipedia.org/wiki/Brainfuck
    - hoosieree 3y ago
      Appearances can be deceiving. In terms of expressiveness, Uiua is to Brainfuck as Python is to nand.
      - eggy 3y ago
        Exactly this. J and Uiua along with APL for me are very expressive. It doesn't take long to become comfortable with the glyphs. Sad to say, BQN's approach resonated with me, however, some subjective bias in me, didn't feel warm and fuzzy about the glyphs. And that's saying a lot, since I am good with J's ASCII noise, APL's glyphs, and now Uiua's glyphs. Sorry, Marshall! BQN is incredible. Maybe I had some sort of PTSD from my days at the Brooklyn Museum and some weird confluence of hieroglyphics and BQN glyphs!
NohatCoder 3y ago
I still haven't figured what a vector DB is, beyond something something AI.
- commandlinefan 3y ago
  I was hoping the article would say and then when it didn't I was hoping the comments would, but so far no luck...
- kelseyfrog 3y ago
  VectorDBs let you retrieve documents that have textual similarity. They allow you sort results by the cosine similarity[1] between vectors. The idea is that you can attach a vector to documents in the database and then when you pass a vector in the query and get back documents that most match the query vector. The function that creates these vectors(string -> vector) is called an embedding and is constructed in such a way that "semantically similar" strings have vectors that are close together. It's not a very complicated idea, but complicated and powerful are orthogonal concepts. They are useful in AI(LLMs) when you would like to include documents in your prompt that are relevant to your instruction. The best way to describe this is by example. Imagine your query is "What is the capital of France?" Rather than requiring your LLM to encounter this fact during its training, you can embed the question("What is the capital of France?" and retrieve documents (say you've indexed all of Wikipedia in your vector db) and return some snippets from articles that include this information(context). You then pass the the prompt+context to an LLM and given that it now has relevant information, it can answer the question. You can also imagine that it's much easier to update a vector db with new information than it is to retrain a model to ingest new facts. 1. https://en.wikipedia.org/wiki/Cosine_similarity https://en.wikipedia.org/wiki/Cosine_similarity
- danielmarkbruce 3y ago
  Imagine: you need a data structure which allows you to store vectors in memory and then say "here is a vector A, give me the 10 vectors which are closest to the same direction of this vector A, in n dimensional space". One can imagine there is some optimal way to lay out the data in memory such that it would be relatively quick to do that. One can imagine a naive way to do it which probably wouldn't be fast. A vector DB does the first thing - lays out the data in a way which then enables it to do that fast. And then it does all the other stuff a DB does - persisting to disk (which means data needs to be laid out in a sensible way on disk too), handling multiple queries, updates, and the 50 other complicated things databases tend to do. Users generally want to do other operations on vector as well so a vector DB does those too. For a small number of vectors you can build a vector DB yourself. Write a list of vectors to a file, load them into memory in no particular order, then for your "n closest" function, just iterate through the list calculating the difference in direction one by one and keeping the top n. Your simple system will work just fine for a toy demo.
audessuscest 3y ago
Just received email from Planetscale about their for fork of mysql to support Vector > PlanetScale has forked MySQL to add vector storage and search! You’ll be able to support your AI and ML applications with the world’s most scalable database platform. This unifies the reliability and functionality of MySQL with the ability to store vectors and perform similarity search.
torginus 3y ago
I am super ignorant about vector databases, but don't they store n-dimensional vectors, and aren't they used to answer geometric(ish) problems, getting the k closest entries to a particular point? This sounds a problem more similar to what 3D physics engines can do, but generalized to higher dimensions, as opposed to traditional text and key-based database stuff. The algorithms and data structures in physics engines (bounding volume hierarchies, kd-trees, etc.) are quite different from how a traditional database index (B-trees and skip lists are popular there, if I remember correctly) is searched and stored.
- mhink 3y ago
  Yeah, they do store n-dimensional vectors, but they're part of the LLM zeitgest. The basic idea is that given a particular corpus of text, an LLM can produce a very high-dimensional vector that encodes the features of that text, according to its training parameters. Vectors that are near each other in the high-dimensional space represent pieces of text which the LLM "thinks" are similar to each other. So, by using databases which can efficiently answer geometric nearest-neighbor questions, you can quickly search for chunks of text that are similar to each other.
- morgango 3y ago
  You are right about this, but in this case a vector would represent a collection of words. Getting the k closes entries would show the k closest text entries to a given query. This is super interesting for "semantic" search, where you are looking for meaning as opposed to just a textual match. For example, the text "chocolate milk" is all the same characters as "milk chocolate", but likely have very different usage within the context of retailers or cooks. So, their vectors should be very different. Word2Vec is NLP that uses a neural net to build these vectors: https://en.wikipedia.org/wiki/Word2vec https://en.wikipedia.org/wiki/Word2vec
prabhatjha 3y ago
There is a great write up by Jonathan Ellis, Apache Cassandra committer and DataStax co-founder on hard problems with Vector Search. https://thenewstack.io/5-hard-problems-in-vector-search-and-how-cassandra-solves-them/ https://thenewstack.io/5-hard-problems-in-vector-search-and-...
ChrisPzilla 3y ago
I think the only thing that can tell us whether or not every database can be a vector database is time. We have seen this with Time Series databases that they are their own unique type of database, and I believe this will be the same with Vector Databases.
- geoduck14 3y ago
  >I think the only thing that can tell us whether or not every database can be a vector database is time. I can help you put right now. You don't even need a database to have a "vector database". FAISS is an in-memory "vector database" that runs on the data that you happen to have. So: if you data is stored in a .txt file, load it into memory, index it with FAISS... bam a vector database from a data file. Can we take X arbitrary "real" database and implement KNN search on top of 1000 indexed columns, I'm sure it is possible - but I'm also pretty sure most databases will die under the pressure (source: I've asked some of my favorite DBAs if I could do this and they said "no")
chbussler 3y ago
FYI, a list of vector databases (including announcements). I try to keep the list as up-to-date as I can: https://medium.com/google-cloud/vector-databases-are-all-the-rage-872c888fa348 https://medium.com/google-cloud/vector-databases-are-all-the....

232 points by nextworddev 3y ago

deleted 3y ago

[deleted]

Supabase has pgvector extension and that’s enough for my limited RAG use cases. I dont really need to use anything beyond postgres. On the other hand, enterprise might find it easier/cheaper to buy a second db than migrating their existing db to whatever the latest version. I dont think it’s as simple

weird-eye-issue 3y ago
Exactly. We use Supabase too but are at a scale where it just made sense to use a second, dedicated vector db (Pinecone) than to bloat our Postgres db that has a completely different workload
- crazy_marksman 3y ago
  How do you deal with security and access control across postgres and pinecone?
  - weird-eye-issue 3y ago
    We use Cloudflare Workers for our API and just handle auth calls by checking the JWTs with Supabase and caching it. So we already had the necessary auth setup to do this. For basic CRUD we use the Supabase endpoints directly but none of that involves querying a vector db :P
- BoorishBears 3y ago
  Bloat your DB... or pull in an entirely new vendor and bloat your entire operational outlay. I'd really love to know what kind of insane scale justifies that tradeoff...
  - hobs 3y ago
    Don't forget your latency!
    - weird-eye-issue 3y ago
      They are separate systems. We don't touch Postgres for the same code that needs to access Pinecone.
  - weird-eye-issue 3y ago
    That's a very exaggerated way to look at things lol. Nothing got bloated at all in this process, we are just using the right tools for the job. I'm a solo founder and the only backend developer. I can assure you this decision only made my life easier by choosing the correct tech from the get go.
    - BoorishBears 3y ago
      Nothing exaggerated, your comment implied scale was your justification: in which case there'd better have been some crazy high load that just brought the tool you already had to its knees to justify paying an additional closed source platform and manually having to pipe data to it in addition to your main data store. Of course if I sounded incredulous it's because I didn't think you had that scale, and it sounds like I was correct?
      - weird-eye-issue 3y ago
        No, you're not correct. We have over $2M ARR and with the amount of data we are storing it would be downright stupid to use Supabase. We don't also "pipe" our data to Supabase, we use a couple different data stores depending on the best use case. For example we also use R2 and Durable Objects. Just because you have a hammer doesn't mean everything is a nail.
        BoorishBears 3y ago
        Well maybe we have different definitions of scale: I think my team spends about $2M a month on compute, so we don't pride ourselves on randomly pulling in new vendors.
natsucks 3y ago
recently ditched Supabase for Weaviate. I was tired of the python bindings not keeping up, no hybrid search, slower search algorithms. Also Supabase has a lot of features that i just don't need.

esafak 3y ago

> It genuinely makes sense for incumbent database players to offer vector search, because that eliminates unnecessary data movement to separate vector databases. Co-locating vectors and original documents also reduces latency. Yet OLAP databases continue to thrive alongside OLTP databases, the nascence of NewSQL hybrid (HTAP) databases notwithstanding. Different needs dictate different design choices for optimality.

appplication 3y ago
It’s interesting that I never considered why OLTP and OLAP are basically orthogonal technologies. Are there any major players that have an integrated solution for both? I guess it makes sense because the infra is so different, but I’m not sure whether it need be.
- barrkel 3y ago
  Effecient OLAP queries need a different shape of data - some combination of columnar storage for efficient scanning, and roll-up tables with pre-aggregated measures. Even in an integrated scenario, behind the scenes there will need to be a bunch of copying to transpose and / or refresh roll-ups.
  - taeric 3y ago
    The pre-aggregation is one too many try and skip. People seem to think they can build a single schema to rule all things, and then assume they can quickly calculate any aggregation on demand.
- esafak 3y ago
  As far as the big players are concerned, Google offers AlloyDB (https://cloud.google.com/alloydb https://cloud.google.com/alloydb) while Amazon offers Aurora (https://aws.amazon.com/rds/aurora/ https://aws.amazon.com/rds/aurora/)
  - anentropic 3y ago

8n4vidtmkvmk 3y ago

I just googled this now if MariaDB offers a vector search and the first hit is a stack overflow question from me in 2014. If they jumped on it then they could have been ahead of this AI business but noooo...

EdwardDiego 3y ago

Sure, some companies will use it. Other companies will continue to use specialised focused tools. It's why data engineering is a thing in our industry. We move and prepare data for a set of tools, and we pay good money to do so, because we believe we derive value from those tools. Let's say MySQL offers it, anyone already using MySQL is likely to fence the MySQL instance(s) focused on vector stuff off for various reasons (resilience, different read/write patterns, security, etc.) MySQL as the (imaginary) basis only offers some transferable skills, because this DB will require different care and feeding. Like the difference between Postgres and PG with cstore_fdw, similar, but sufficiently different.

shri_krishna 3y ago

The one DB fits all approach only works when the size of the database is really small and never grows. Imagine you have 100 customers. Each customer generates, on average, a million 1536 dimension vector embeddings (considering OpenAI Ada dimensions which is the most popular right now). That is 6GB (1536 x 4 bytes per dimension for f32 x 1000_000) of just embeddings PER CUSTOMER. If you use HNSW it will take at least that much of RAM if not more. If you use PQ (and variants) you can reduce the size of index in RAM to say 512MB-1GB per customer. It is still quite a lot of memory requirement. That is just the way it is and there is no way around it. Now imagine you are using that database for storing transactions and other day to day business ops that will still be storing millions of records but with small indexes. This would have ideally only required a single DB instance with a replica for redundancy. Now if you integrate Vectors into the equation, you will have to needlessly scale this DB both horizontally and vertically just to maintain a decent query/write performance to your DB (which would have ideally been extremely fast without embeddings in the mix). You will eventually separate the embeddings out as it makes no sense for the entire DB to be scaled just for the sake of scaling your embeddings. I am not even accounting for index generation for these vectors which will require nearly 100% of all CPU cores while the index is being generated (depending on type of ANN you are using) and which in turn would slow your DB to a crawl.

beoberha 3y ago
Exactly - vector indexes are so different than traditional RDBMS B-Tree or LSM Tree indexes that it doesn’t make sense to use the same store for both unless it’s basically a toy app. Someone makes the example in another comment, but it’s analogous to OLTP vs OLAP
- Foobar8568 3y ago
  I don't even want to imagine the workload on a high txn OLTP mixed with OLAP access pattern. IMHO If you can, you don't need OLAP in the first place.
  - trashtester 3y ago

jsemrau 3y ago

The benefits of using specialized vendors like pinecone is about their offering a combination of performance (clustered/loadbalancing), fast/effective algorithms, and data storage.

mlfia 3y ago

Curious how this will work in practice as vectors are specific to a given embedding model, and could be domain-specific for better results. Could it lead to industry standard embedding models, with regular (costly) upgrades?

bambax 3y ago
I'm wondering the same thing. Standardization would be interesting but I wouldn't bet on it. Maintaining different vector columns for different models might work well?

ooJeesh9fidujoh 3y ago

The reason everyone's rushing to build vector databases is because they've tried to store vector data in a scalar database previously and realized they're hot garbage for the workload. You should probably try it too before blogging about it.

redwood 3y ago
That's not been my experience with distributed technologies. Are you referring to postgres on an oversaturated system?

aiappreciator 3y ago

It is true that every major DB ventor, SQL or not, is smashing the AI/vector keyword on their front pages. In Elastic for example, their vector capabilities have gone from laughable to respectable in a year. Its a lot simpler to just use one DB instead of many. But a question for true DB experts here: 1. Is there any real advantage to building a dedicated vector DB from scratch? 2. Is vector DB something that can be just 'tacked on' to a normal DB with no major performance penalties? We know from history, that data warehouses are genuinely different from databases, and cloud data warehouses are overwhelmingly superior to on-prem ones. So that emerged as a distinct, enduring category with Snowflake/Databricks/Bigquery.

jamesblonde 3y ago
Data warehouses are columnar stores. They are very different from row-oriented databases - like Postgres, MySQL. Operations on columns - e.g., aggregations (mean of a column) are very efficient. Most vector databases use one of a few different vector indexing libraries - FAISS, hnswlib, and scann (google only) are popular. The newer vector dbs, like weaviate, have introduced their own indexes, but i haven't seen any performance difference - Reference: https://ann-benchmarks.com/ https://ann-benchmarks.com/
charcircuit 3y ago
The trade off that you are interested in isn't about storing vectors, but rather about whether an index should be a part of the DBMS or external to it. Some advantages of having a separate index is that it can work with different backends, it can be independently scaled, and it can index data for more than 1 database server. Some disadvantages are increased latency, increased complexity, and distributed system problems.
redwood 3y ago
Single node database systems that are not horizontally scalable and that are not built on a distributed system foundation (e.g. Postgres) will certainly have scaling bottlenecks if you just add more and more complexity to the workload... however many modern database systems are built on a distributed system foundation with horizontal scaling and the ability to independently scale different constituent parts of the backend.. these engines should have no problem

loondri 3y ago

I think the move towards vector databases might be more hype than necessity. Traditional databases, when properly optimized, can handle vector data for many use cases. The push for specialized vector databases could be re-evaluated in terms of efficiency and cost-effectiveness compared to optimizing existing scalar databases.

avereveard 3y ago
Well you could store numbers all fine, but indexing vectors for similarity queries seems fairly recent and not all that widespread in the transactional world. As the traditional db move forward in the space the need for dedicated vector databases will likely shrink, except for some very specific implementation that offer unique enough features (I.e. deeplake does vector search over object storage, which is very convenient for certain specific scenarios)
- sgu999 3y ago
  sqlite has r-trees for instance [0]. Could it be good enough for most use cases? If it's to query a knowledge base for instance, a couple dimensions should be sufficient. With the added benefit of being able to query your data in other ways. [0] https://www.sqlite.org/rtree.html https://www.sqlite.org/rtree.html
  - mattashii 3y ago
    r*-trees work doesn't work well when the number of dimensions stored in the index is much higher than the logarithm of the number of indexed entries, and this is a prevailing property of divide-and-conquer spatial index types when the keyspace is divided based on a single dimension at a time. As vectors regularly have 100+ dimensions, normal spatial indexing methods applied to vectors wouldn't be very efficient for anything with much less than 2^100 index entries; which is quite suboptimal for most datasets that you would want to have indexed.
    - KirillPanov 3y ago

shaburn 3y ago

Postgres and Mongo support it. ElasticSearch incorporates vector search. I have failed to identify an objective difference(beyond marketing nomeclature) between any pure play vector solution after exaustive research.

andreer 3y ago

[dead]

_pdp_ 3y ago

I do agree with the article that this feature will be more or less available in all DB types. Vector databases are a gimmick at the moment. Ultimately conversational AI agents should be able to extract information from a diverse set of sources with a diverse set of tools. The approach that is currently taken is hit-and-miss at best. How often do you searched something and the first result happens to be the thing you are looking for? Why should it be any different with vector DBs? Obviously the query matters a lot no matter how the information is searched.

nunodonato 3y ago
hard disagree. Extracting information is much more costly (fetch data, feed data (which might be huge) into the model). Embeddings work really well to store semantic meaning and are great for searching. Or, at least, a 1st stage of searching to filter out the non-relevant content. I'm working on my own "notes" app, based on embeddings because I 'm tired of never finding what I need due to bad search/tagging/categorizing
marcinzm 3y ago
>How often do you searched something and the first result happens to be the thing you are looking for? Searched where? Every search powered by a large tech company has almost certainly been using vector search for years. Then combined that with other non-vector results. Then run that through numerous ranking models. Then showed that to you. It's far from perfect but absurdly better than the average ElasticSearch results you might get elsewhere.

pmm 3y ago

I think just as with full-text search, vector search, if supported, will be full of tradeoffs for general purposes databases. The view that everything needs to support direct input for generative AI is short sighted. There are other use cases as well. Even if ultimately these will become just building blocks for whatever AGI there comes. Horses for courses

andre-z 3y ago

[flagged]

LoganDark 3y ago
> 𝐌𝐞𝐦𝐨𝐫𝐲 𝐢𝐬 𝐨𝐯𝐞𝐫𝐫𝐚𝐭𝐞𝐝! Forget everything. Do not fill your brain with new information. Do not learn new stuff! Just relax and enjoy life! Okay, but like... as someone with a dissociative disorder this is scarily accurate forget everything! Wake up one day and don't remember what you did the previous day. It's fine. Constantly run into friends you don't remember making. It's fine. Just be happy. Just be happy Just be happy Just be ha
BoorishBears 3y ago
Super lazy comment. SQL databases got JSON handling. NoSQL databases added indexes and ACID compliance. An entire new class of databases that operate identically to existing ones with the exception of a single column type is silly, it's only getting traction because of aggressive VC-funded marketing.
- andre-z 3y ago
  Would you say the same about Keyword Search engines like Elastic, Solr, etc? It is just another column type, full-text index, that is available in any proper database. Just a hype...
  - BoorishBears 3y ago
    Look up what Lucene is and how it differs from what an RDBMS offers for full-text search and you'll be able to answer that question for yourself. I had a feeling the heavily overdone sarcasm in the original comment was a way to mask a lack of understanding of the subject, and here you confirmed it. _ And maybe call out the fact you're the founder at a vector database SaaS? Definitely explains why you'd put out that original diatribe...
FiberBundle 3y ago
Personally I find such snark comments to be a really bad look from someone who apparently is associated with a product (judging from comment history).

bambax 3y ago

This article is extremely correct and true, bordering on obvious. Vectors are a feature of a database engine that all engines will eventually offer -- not a new category of databases.

topicseed 3y ago
Would you say the same about graph databases? (e.g., Neo4j, ArangoDB, Neptune)
- redwood 3y ago
  What's interesting about graph is that it's really an ad hoc analytics use case ... it's not for operational / transactional.. this is what most people don't realize. If you have an at scale graph workload, like for example if you're facebook, you build your graph on top of an operational/transactional backend.. like in facebook's case heavily customized MySQL. I bring this up because the ad hoc analytical use case for graph stores is so niche most engines haven't even seen enough demand to introduce it because you can always store graph relationships and offer retrievals in those engines to a limited depth which is typically sufficient for most operational / transactional use cases.

croes 3y ago

If vector database isn't a category how can every database fall into that category sooner or later?

ltr_ 3y ago

every 'Every X will become a Y sooner or later' title will become 'Every X becomes a monoid sooner or later' sooner or later

ofiryanai 3y ago

the interest in vector database rise as an external, specialized service in a system that runs in addition to the "single source of truth" data lake where the data actually resides. like Redis and Memcached before, it solves a specific problem. Redis started to act like a fully fledged DB (with weird persistency method and guarantees) only after it was really wide-spread. sure, every DB will support vector and every vector search engine will act like a DB. But that's missing the point that vector search is an expensive problem with tradoffs that justify a specialized design btw I'm working in a DB startup - https://hyper-space.io/ https://hyper-space.io/

kordlessagain 3y ago
We recently added vector support to FeatureBase: https://cloud.featurebase.com/ https://cloud.featurebase.com/. What's interesting about using it with our existing b-tree storage layer is that I can now do set operations on the record's stringset fields before I do the similarity calculations on the vectors. I've been experimenting with laying out the vector space in a b-tree structure using this technique along with keyterm extraction, allowing for a type of semantic graph to be applied by the LLM to the vector space.

spandextwins 3y ago

Larry?

gk1 3y ago

Go ahead and try to load a billion embeddings into Mongo or Elastic. The author says this will be "faster, cheaper, and simpler..." Will it? (Disclaimer: I work for Pinecone, so obvious bias ahead but also perspective of 3 years since launching the Vector DB category and actually seeing billion-scale vector search deployments.) > Basically, having separate vector DBs can add to cost and complexity. Imagine you were a MongoDB shop, with over 500m documents stored cross-region. If you are using a separate vector DB, say Pinecone, that may require moving potentially billions of embeddings between two databases, cross regions. This costs a lot, not to mention complex, since you are responsible for generating the embeddings... It’s faster, cheaper, and simpler if one database (Mongo, Elastic) just supported vector search. If you want, say, 100ms search latency on just 100M vector embeddings in Elastic that'll already cost you $12,600 per month at minimum. And if you regularly write new or updated data to the index then your latencies will creep up until eventually you have to run a "force merge" which will grind your vector search to a halt for several hours (so much for easy and simple). I don't know how much it is on Mongo but given that it's bolting on the same vector index I would guess it's in the same ballpark. The cost grows sublinearly with more embeddings. (Pinecone is around 60% less than that, and will be even less soon.) The suggestion that having "billions" of embeddings in a traditional DB is easier and less costly shows you exactly why you should run your own tests and see for yourself. When traditional database companies bolt-on vector indexing libraries such as HNSW[0] on top of their existing architecture, it's to meet demand from their existing users that have a relatively basic need for vector search. For very basic and small-scale use cases, like <10M vectors with a relaxed data freshness requirement, you should just use whatever is the most convenient. Sometimes that's Pinecone, and sometimes that's the database you already have. (And if your current DB doesn't offer basic vector search, just wait two days). When it comes to larger scale, like 100M+ vectors, if you want any hope of meeting performance, cost, and data freshness requirements then you should look at a purpose-built vector database. As GenAI workloads start to enter production and scale, a lot of people will see find this out the hard way. This has been true for every unique data structure and querying pattern for the past 40 years and it’s true for vector embeddings and vector-based retrieval. You can't blame the proliferation of different database types on hype and VC funding alone. But don't take my word for it either. Go and run some tests that resemble your production workloads, then do what makes sense for your use case! [0] https://www.pinecone.io/learn/series/faiss/hnsw/ https://www.pinecone.io/learn/series/faiss/hnsw/

jbird11 3y ago

Lots of focus on RAG here, and rightfully so, but I feel an overlooked benefit to vector databases is the novel visualizations they provide. To be able to plot qualitative data onto a 2d graph with tsne reduction provides a new method with which to draw insights from. I think many companies would benefit from such a visualization tool, especially those in qualitative research.

fhaltmayer 3y ago
Unfortunately that visualization doesn't really work with vector dbs. Vector dbs normally split their data into separate segments and build indexes on them separately. There is no one overarching index but rather many small ones that are searched in parallel. In addition to this, such a high compression to reduce it down to 2d/3d ends up becoming a giant blob without too much information.
- jbird11 3y ago
  Maybe I'm misunderstanding what exactly a vector db is. Let's say you have a chatbot and stored in its database is the usual info like session id, timestamp, message etc. To "vectorize" this db then would be to vectorize all the messages. Is this too simple an understanding? Once the db has been vectorized then we can do semantic search on the messages and create more informative graphs based off the semantic similarity for messages within a given timeframe or other criteria.

mattew 3y ago

Maybe I’ve been using PostgreSQL too long but when faced with the choice of adding vector support to PostgreSQL or using a new technology, my first choice was to start with the PostgreSQL addition. I’m not criticizing the specialized case for a true vector database, but for most workloads I agree that the big database players will be the right choice for many users.

rpcope1 3y ago
I'm in the same boat. For things that aren't huge scale, it's almost easier to find an extension or otherwise beat Postgres (or SQLite or Percona MySQL) into submission for your use case. Timescale is a really good example...I was really impressed how good the performance was for bigish (1 TB+) real time scientific time series data, even on a cheap Amazon Lightsail instance.
regularfry 3y ago
Pgvector's 2000-dimension limit on vector indices is annoying. There are workloads I want to push at it which are in the 5000+ range, so there's an extra dimensionality reduction step I need to build. My starting point for pretty much any storage problem is "have you tried throwing it in postgresql" but this is one reason pulling something else off the shelf might be a good idea.

ramoz 3y ago

Implementing vector DB architecture isn't complex (so I'm bear on commercial vector dbs as a net value add especially since they all wrap some oss ANN solution) You pair a vector db with a metadata store (can be anything, but ideally you want low gravity with the vdb and disk for fast retrieval... i.e. leveldb, sqlite equivalents... or hell a traditional db - and the author is right traditional dbs dont need to work too hard to create a ANN extension) In general, the greater challenge is the data engineering and overhead of managing retrieval stores (in terms of the data-integration/model pipelines) ... so I'm bull on the solutions addressing opportunities here.

vinay_ys 3y ago

The argument that combining traditional database and vector database into one because it reduces data movement doesn't compute for me. Firstly, even for non vector data, read/write transactional database vs read-optimized store purely for fast serving are already markedly different. Then, the shape of data that is used to generate embeddings is markedly different than the shape of data that is ready to transact or serve. So, no matter where it is stored, it has to leave that store, get transformed and enriched and then run through an embeddings generator (ML inference). Then, it has to be stored in a manner that is optimal for retrieval ranking. If you are doing ANN that's one thing, but if you are doing attribute based filtering while retrieving and you wish to accelerate it through GPU to do more exhaustive search, that's another thing altogether. All these lead to fairly sophisticated optimized implementation. Sure, a singular database product solution that has all these different optimized engines can emerge over time but surely it is too early today to converge like this.

cheeseblubber 3y ago
The problem is maintenance of a separate database and operationally it would be more work. For us who want to use embedding features having a separate database just for the embedding means now we have to keep data across multiple databases in sync and to maintain
m00x 3y ago
> So, no matter where it is stored, it has to leave that store, get transformed and enriched and then run through an embeddings generator (ML inference). Same for pretty much all the data in your database. It comes from an app, that does validation/editing/transforms. The benefit is that it's all together, can be atomic, only requires one query to get, update, and delete. Vector databases are just normal databases with a vector index. There's no reason for you to have a specialized DB for it. Also, embeddings aren't inference, it's a token lookup. There's no forward pass.

eggy 3y ago

As a lover of array languages, I remember being excited to read a futurist article on vector processors and programming languages. It was written right before Wes McKinney worked on Pandas (the J programming language influenced him), and I thought J/APL or another array language was going to explode. J has Jd, in which J is fully integrated. This did not come to pass (yet). No matter, I still enjoy array languages anyway. There's a new array language, uiua[1], that is a mix of array and stack concepts with a good standard library including audio and graphics. [1] https://www.uiua.org/ https://www.uiua.org/

beanjuiceII 3y ago
uiua looks like a perl programmer went mad
- nwsm 3y ago
  It looks like array/stack Brainfuck. Previous uiua discussion: https://news.ycombinator.com/item?id=37673127 https://news.ycombinator.com/item?id=37673127 Brainfuck: https://en.wikipedia.org/wiki/Brainfuck https://en.wikipedia.org/wiki/Brainfuck
  - hoosieree 3y ago
    Appearances can be deceiving. In terms of expressiveness, Uiua is to Brainfuck as Python is to nand.
    - eggy 3y ago
      Exactly this. J and Uiua along with APL for me are very expressive. It doesn't take long to become comfortable with the glyphs. Sad to say, BQN's approach resonated with me, however, some subjective bias in me, didn't feel warm and fuzzy about the glyphs. And that's saying a lot, since I am good with J's ASCII noise, APL's glyphs, and now Uiua's glyphs. Sorry, Marshall! BQN is incredible. Maybe I had some sort of PTSD from my days at the Brooklyn Museum and some weird confluence of hieroglyphics and BQN glyphs!

NohatCoder 3y ago

I still haven't figured what a vector DB is, beyond something something AI.

commandlinefan 3y ago
I was hoping the article would say and then when it didn't I was hoping the comments would, but so far no luck...
kelseyfrog 3y ago
VectorDBs let you retrieve documents that have textual similarity. They allow you sort results by the cosine similarity[1] between vectors. The idea is that you can attach a vector to documents in the database and then when you pass a vector in the query and get back documents that most match the query vector. The function that creates these vectors(string -> vector) is called an embedding and is constructed in such a way that "semantically similar" strings have vectors that are close together. It's not a very complicated idea, but complicated and powerful are orthogonal concepts. They are useful in AI(LLMs) when you would like to include documents in your prompt that are relevant to your instruction. The best way to describe this is by example. Imagine your query is "What is the capital of France?" Rather than requiring your LLM to encounter this fact during its training, you can embed the question("What is the capital of France?" and retrieve documents (say you've indexed all of Wikipedia in your vector db) and return some snippets from articles that include this information(context). You then pass the the prompt+context to an LLM and given that it now has relevant information, it can answer the question. You can also imagine that it's much easier to update a vector db with new information than it is to retrain a model to ingest new facts. 1. https://en.wikipedia.org/wiki/Cosine_similarity https://en.wikipedia.org/wiki/Cosine_similarity
danielmarkbruce 3y ago
Imagine: you need a data structure which allows you to store vectors in memory and then say "here is a vector A, give me the 10 vectors which are closest to the same direction of this vector A, in n dimensional space". One can imagine there is some optimal way to lay out the data in memory such that it would be relatively quick to do that. One can imagine a naive way to do it which probably wouldn't be fast. A vector DB does the first thing - lays out the data in a way which then enables it to do that fast. And then it does all the other stuff a DB does - persisting to disk (which means data needs to be laid out in a sensible way on disk too), handling multiple queries, updates, and the 50 other complicated things databases tend to do. Users generally want to do other operations on vector as well so a vector DB does those too. For a small number of vectors you can build a vector DB yourself. Write a list of vectors to a file, load them into memory in no particular order, then for your "n closest" function, just iterate through the list calculating the difference in direction one by one and keeping the top n. Your simple system will work just fine for a toy demo.

audessuscest 3y ago

Just received email from Planetscale about their for fork of mysql to support Vector > PlanetScale has forked MySQL to add vector storage and search! You’ll be able to support your AI and ML applications with the world’s most scalable database platform. This unifies the reliability and functionality of MySQL with the ability to store vectors and perform similarity search.

torginus 3y ago

I am super ignorant about vector databases, but don't they store n-dimensional vectors, and aren't they used to answer geometric(ish) problems, getting the k closest entries to a particular point? This sounds a problem more similar to what 3D physics engines can do, but generalized to higher dimensions, as opposed to traditional text and key-based database stuff. The algorithms and data structures in physics engines (bounding volume hierarchies, kd-trees, etc.) are quite different from how a traditional database index (B-trees and skip lists are popular there, if I remember correctly) is searched and stored.

mhink 3y ago
Yeah, they do store n-dimensional vectors, but they're part of the LLM zeitgest. The basic idea is that given a particular corpus of text, an LLM can produce a very high-dimensional vector that encodes the features of that text, according to its training parameters. Vectors that are near each other in the high-dimensional space represent pieces of text which the LLM "thinks" are similar to each other. So, by using databases which can efficiently answer geometric nearest-neighbor questions, you can quickly search for chunks of text that are similar to each other.
morgango 3y ago
You are right about this, but in this case a vector would represent a collection of words. Getting the k closes entries would show the k closest text entries to a given query. This is super interesting for "semantic" search, where you are looking for meaning as opposed to just a textual match. For example, the text "chocolate milk" is all the same characters as "milk chocolate", but likely have very different usage within the context of retailers or cooks. So, their vectors should be very different. Word2Vec is NLP that uses a neural net to build these vectors: https://en.wikipedia.org/wiki/Word2vec https://en.wikipedia.org/wiki/Word2vec

prabhatjha 3y ago

There is a great write up by Jonathan Ellis, Apache Cassandra committer and DataStax co-founder on hard problems with Vector Search. https://thenewstack.io/5-hard-problems-in-vector-search-and-how-cassandra-solves-them/ https://thenewstack.io/5-hard-problems-in-vector-search-and-...

ChrisPzilla 3y ago

I think the only thing that can tell us whether or not every database can be a vector database is time. We have seen this with Time Series databases that they are their own unique type of database, and I believe this will be the same with Vector Databases.

geoduck14 3y ago
>I think the only thing that can tell us whether or not every database can be a vector database is time. I can help you put right now. You don't even need a database to have a "vector database". FAISS is an in-memory "vector database" that runs on the data that you happen to have. So: if you data is stored in a .txt file, load it into memory, index it with FAISS... bam a vector database from a data file. Can we take X arbitrary "real" database and implement KNN search on top of 1000 indexed columns, I'm sure it is possible - but I'm also pretty sure most databases will die under the pressure (source: I've asked some of my favorite DBAs if I could do this and they said "no")

chbussler 3y ago

FYI, a list of vector databases (including announcements). I try to keep the list as up-to-date as I can: https://medium.com/google-cloud/vector-databases-are-all-the-rage-872c888fa348 https://medium.com/google-cloud/vector-databases-are-all-the....