3 ms·
> I'd say Postgres + pgvector is even simpler For development, perhaps. For production, absolutely not. I wish HN had some bot that would just delete any comm
by threeseed 3y ago
> I'd say Postgres + pgvector is even simpler
For development, perhaps. For production, absolutely not.
I wish HN had some bot that would just delete any comment from people recommending installing databases. Because 99.9% of the time it's from those who have no experience in running one in a Production environment. Keeping it secure and ensuring backup/restore cycle works is seriously non-trivial.
- andrewmutz 3y agoWhat's wrong with Postgres + pgvector in production? pgvector is supported by RDS. Keeping an RDS instance running in production isn't exactly rocket science.
- fiedzia 3y agoIt's no match for lucene (in practice ES or Solr) in terms of performance and features, as it's very different model of indexing and operating. Keeping it running while 1000s of customers run search queries AND the app uses db AND users want faceting or other features is not an option.
- zosima 3y agoBut for an internal knowledge base backup/restore may be irrelevant (as documents are all copies of data and the database reconstructed fast at will) and well security is really not that difficult with a single system user.
- baz00 3y agoKeeping postgres alive is considerably less work than keeping anything Lucene based alive.
- doctor_eval 3y agoSecurity and backups are important regardless of the technology you’re using. Managed Postgres is a thing, probably much more common than managed lucerne. I can name three managed Postgres providers off the top of my head (AWS, Vultr, Supabase).
- threeseed 3y agoThere is no such thing as managed Lucene. It's a library. There are search engines e.g. Solr, ElasticSearch but those aren't what this paper is proposing.
- doctor_eval 3y agoRight. So now my existing stack needs a backup strategy that’s independent of the database. That’s really tough in production; Lucerne is adding complexity. To be clear I object to this: > I wish HN had some bot that would just delete any comment from people recommending installing databases. Most applications already use a database. Pooh-poohing Postgres as a solution is silly. For many people, it’s actually going to be less complex to take to production than installing a library, especially one with its own complex storage needs. Also: technically, isn’t Lucerne a database?
- jstx1 3y agoIsn't this solved by managed databases? Is there a problem with paying more and having AWS/Azure/GCP take care of the difficult parts for you?
- moonchrome 3y agoEnabling pgvector in Azure was trivial. I think if you're past pgvector performance you won't be listening to a random guy talking about pgvector but have a good understanding of the space. If you're new (like I was a few months ago) save yourself the time I wasted on the noobtraps I mentioned. It scales way better than the OpenAI API for my use cases. Lucene (or rather elastic/open search) is way overkill for my needs
- uoaei 3y agoI wish HN had some bot that would just delete any comment from people assuming everyone runs databases exposed to the internet.