3 ms·

Nice use of your service for a Show HN! The number of comments about the Karma being divergent on people's matches suggest you could add a simple ensemble appr

Nice use of your service for a Show HN! The number of comments about the Karma being divergent on people's matches suggest you could add a simple ensemble approach with a karma heuristic on top of the vector similarity search to help results. If the data used isn't already reflecting Karma, it could be a useful metric for representing a whole bunch of things (quality of comments, participation, length of time), and make the vector similarity more meaningful. It would be interesting to see if users perceive the results as higher quality if you added a simple Karma similarity filter (maybe 400 points or something based on the standard deviation of the average karma score), and then returned the closest matches filtered by that metric. Vector similarity search as a service looks like a good market. Out of interest, what would the cost translate to for running something like this in practice using the API as a customer?

jedwhite 5y ago
PS Possibly you could add a floor (say less than 20 to exclude shill or throwaway accounts), or ignoring karma difference over a threshold (say 1000). I think applying simple filters based on common sense or domain knowledge can help vector similarity searches with sparse data or pollution. Just a random thought :)
gk1 5y ago
> Out of interest, what would the cost translate to for running something like this in practice using the API as a customer? We offer usage-based billing at $0.10 per GB of memory hour. (https://www.pinecone.io/pricing/ https://www.pinecone.io/pricing/) For this app, our eng team knows for sure but I think the entire index is less than 1GB so it would be just $73/month if we keep it running 24/7. Vector similarity search is new for most companies, so we want to make it very easy to try and test stuff out in production, without cost being a barrier. Even for larger volumes (40GB+) we offer volume and pre-commitment discounts.

jedwhite 5y ago

PS Possibly you could add a floor (say less than 20 to exclude shill or throwaway accounts), or ignoring karma difference over a threshold (say 1000). I think applying simple filters based on common sense or domain knowledge can help vector similarity searches with sparse data or pollution. Just a random thought :)

gk1 5y ago

> Out of interest, what would the cost translate to for running something like this in practice using the API as a customer? We offer usage-based billing at $0.10 per GB of memory hour. (https://www.pinecone.io/pricing/ https://www.pinecone.io/pricing/) For this app, our eng team knows for sure but I think the entire index is less than 1GB so it would be just $73/month if we keep it running 24/7. Vector similarity search is new for most companies, so we want to make it very easy to try and test stuff out in production, without cost being a barrier. Even for larger volumes (40GB+) we offer volume and pre-commitment discounts.