3 ms·
Pretty cool. A little slow for a search engine, have you tried in-database embeddings for semantic search like PostgresML?
by levkk 2y ago
Pretty cool. A little slow for a search engine, have you tried in-database embeddings for semantic search like PostgresML?
- jnnnthnn 2y agoThanks for trying it out! Agreed it could be faster for uncached queries. The embeddings retrieval itself is actually pretty fast (uses pgvector). However, I found that having a LLM rerank results + generate summaries related to the search query made results more useful, which is what accounts for much of the latency. Maybe I should make that a user-customizable setting!
- montanalow 2y agoYou can do all of that in a single SQL query, with pgml.embed() and then pgml.train() a custom reranker with xgboost, to pgml.predict() the conversion score of a search result based on click-through-rate, or other objective. If you'd like free hosting, feel free to reach out. I'm one of the founders at postgresml.org.
- jnnnthnn 2y agoSweet. I'll follow-up off HN. Thank you!