7 ms·
If you want to play with a vector database and already use postgres, there's pgvector[0]. It's easy to add as an extension (supports Postgres 11+). Supabase wr
by mritchie712 3y ago
If you want to play with a vector database and already use postgres, there's pgvector[0]. It's easy to add as an extension (supports Postgres 11+).
Supabase wrote a solid tutorial[1] (you don't need to run it on Supabase).
0 - https://github.com/pgvector/pgvector https://github.com/pgvector/pgvector
1 - https://supabase.com/blog/openai-embeddings-postgres-vector https://supabase.com/blog/openai-embeddings-postgres-vector
- imaurer 3y agoI am bullish Pgvector because I am “postgres for everything guy”. Current concerns are the scaling and recall performance. The author is looking at product quantization along with other ideas: https://github.com/pgvector/pgvector/issues/27 https://github.com/pgvector/pgvector/issues/27 More details on product quantization: https://mccormickml.com/2017/10/13/product-quantizer-tutorial-part-1/ https://mccormickml.com/2017/10/13/product-quantizer-tutoria... A nice repo that tracks the ANN relative performance of different indexes: https://mccormickml.com/2017/10/13/product-quantizer-tutorial-part-1/ https://mccormickml.com/2017/10/13/product-quantizer-tutoria... Also shoutout to Weaviate because they have great docs, are open source and have very informative YouTube channel. https://weaviate.io/ https://weaviate.io/
- mritchie712 3y agoYep, we're (https://www.definite.app/ https://www.definite.app/) using pgvector and I was initially concerned about scaling, but it doesn't seem it will be a problem for our use case. I definitely wouldn't use it if I was building a feature for Slack, but works for us!
- overview 3y agoAt a glance, your product seems like a fit for my team. However, your landing page doesn’t give specifics. What exactly does it do?
- mritchie712 3y agoYes, working on that landing page right now (currently it's pretty week)! We're building an AI data analyst. You can ask questions of your database and get answers immediately. We also auto generate entire dashboards based on common patterns (e.g. a "Sales Dashboard", "Marketing Dashboard", "Finance / Burn" etc.). If you want to give it a try (there's a demo database embedded in the app), you can use it here: https://ui.definite.app/ https://ui.definite.app/
- KyeRussell 3y agoAmen. After suffering through many years of people telling me to use document databases when I was much better served with—at most—Postgres with a jsonb field, I feel vindicated enough to feel justified in doing my due diligence before going off the beaten track. Not that document databases don’t have their place, but…MongoDB is webscale and all that.
- akiselev 3y agoObligatory Youtube video for historical purposes: https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs
- videlov 3y agoOver the past couple of days I tried 11 different vector databases, in order to evaluate and decide which one we'd choose for our use case. I ended up choosing Weaviate specifically because of the nice docs, but beyond that, time will tell.
- technics256 3y agoCurious if you tried Vespa?
- dunefox 3y agoAny thoughts on Milvus?
- canadiantim 3y agoHave you tried cozodb? Newest kid on the block, looks very promising
- mmaia 3y agoI would love to read more about your experience. We need more content with feature, peformance, and architecture comparisons. Currently, there's a lot of developer evangelism hype in the space.
- bobvanluijt 3y agoThanks, that's nice feedback
- supriyo-biswas 3y agoCan you search both by an equality comparison and a vector search in weaviate? I’d like to do something along the lines of `SELECT * FROM table t WHERE cosine_dist(:my_embedding, t.doc_embedding) < :x AND some_column = “XYZ”`
- imaurer 3y agoWell Weaviate is graphql and it has filtering and hybrid search which is a great feature that pg can’t fully support because it doesn’t have bm25 https://weaviate.io/developers/weaviate/api/graphql/filters https://weaviate.io/developers/weaviate/api/graphql/filters https://weaviate.io/blog/hybrid-search-explained https://weaviate.io/blog/hybrid-search-explained I have a ChatGPT session where I have asked it to do a hybrid search using filtering, pg fts and vector search. Looks reasonable just need to test it and write it up somewhere.
- swe_dima 3y agoGoogle Cloud still doesn't support this plugin, big shame
- imaurer 3y agoAWS just added yesterday. Hosting options tracked here: https://github.com/pgvector/pgvector/issues/54 https://github.com/pgvector/pgvector/issues/54
- mmaia 3y agoYup. pgvector will do it for a lot of projects, specially if you're just trying things out. It think of it as using PostgreSQL full text search before you need to deploy a decidated solution.
- justchad 3y agoI'm bullish on pgvector as well. Now that RDS supports it as well as plenty of other cloud providers it seems like a no-brainer to be able to stick with your existing stack (assuming it's postgres). Andrew Kane is such a prolific open-source maintainer as well.
- fudged71 3y agoReplit now has Postgres databases. Do you know if it's possible to use pgvector on replit?
- time_to_smile 3y agoOut of curiosity, what's the use case here? It seems like if the goal is to "play around with vector databases", why not just install it on your local machine? Part of using these tools is learning how they work and configuring them yourself. If the goal is "start developing products using vector data bases" then it seems like you would surely want something a bit more under your control than using replit.
- fudged71 3y agoI would say my use case is the same as many other people who use it. Repl.it is fantastic for getting started, for sharing your code, and for creating small applications.
- tornato7 3y agoAlso plugging my crappy vector database, which you probably shouldn't use for anything but a fun project, however it can be set up and used in seconds. https://github.com/corlinp/Victor https://github.com/corlinp/Victor