3 ms·

1) Semantic search for legal documents. Found it annoying that, as a lawyer, I have to use search engines that require me to guess the keywords that might be us

by leobg 4y ago

1) Semantic search for legal documents. Found it annoying that, as a lawyer, I have to use search engines that require me to guess the keywords that might be used in the documents I'm looking for... as well as to skim through the entire documents the system found in order to judge whether or not they are relevant to my case. 2) Discovering audiobooks I'll enjoy. Someone here on HN mentioned "random sampling" as a great technique to get a feel for a domain without getting biased by one's pre-conceived notions. I thought that would be awesome for books, to literally "not judge them by their cover". 3) Search engine for pre-owned Teslas. Quickly see what's good value based on age/mileage per price, as well as being able to see how long I'll have to wait, on average, to get a particular set of features for a particular price (based on historical market data). Plus several others.

gk1 4y ago
Have you seen https://www.parrothq.com https://www.parrothq.com? (I’m not affiliated.) Or you can build something using a combination of LLM models (OpenAI, Cohere, Hugging Face) and Pinecone (offers keyword-aware semantic search which is what you probably want for domain-specific content like legal… I am affiliated).
- leobg 4y ago
  I trained my own models, actually. Also tried many of the vector search engines out there. But ended up with a custom solution using hnswlib and SQLite to get subsecond speed with millions of documents.
nonstopnonsense 4y ago
About (1): May I request you to throw some light? Is your reference that the keywords are not synonymized to find equivalents within documents? Help?
- leobg 4y ago
  It’s not just about synonyms. It’s about context. A passage can talk about exactly the topic I’m researching, even though none of the words match. On the other hand, a passage may have ask the right words in it, but in a completely different context.

by leobg 4y ago

gk1 4y ago

Have you seen https://www.parrothq.com https://www.parrothq.com? (I’m not affiliated.) Or you can build something using a combination of LLM models (OpenAI, Cohere, Hugging Face) and Pinecone (offers keyword-aware semantic search which is what you probably want for domain-specific content like legal… I am affiliated).

leobg 4y ago
I trained my own models, actually. Also tried many of the vector search engines out there. But ended up with a custom solution using hnswlib and SQLite to get subsecond speed with millions of documents.

nonstopnonsense 4y ago

About (1): May I request you to throw some light? Is your reference that the keywords are not synonymized to find equivalents within documents? Help?

leobg 4y ago
It’s not just about synonyms. It’s about context. A passage can talk about exactly the topic I’m researching, even though none of the words match. On the other hand, a passage may have ask the right words in it, but in a completely different context.