3 ms·

Trieve is a very impressive achievement, you've managed to slam together every AI buzzword into a semi usable product! It's written in Rust, which makes the en

Trieve is a very impressive achievement, you've managed to slam together every AI buzzword into a semi usable product! It's written in Rust, which makes the entire proposal even more perfect for HN :). Love it. Some questions (and thanks for the detailed "about" page, it answered several of my initial questions!) ~ Will you re-index and keep updating the system to improve the quality of results, or what is the plan? It'd be awesome to have something more nuanced than Algolia which stays updated in near real time, like Algolia. How easy or challenging is it to bootstrap / re-index? Is it possible to ingest new data with partial updates to the existing indices, or is a full indexing from zero always required? Are GPUs strictly necessary? Is it possible to use only CPUs if indexing speed isn't a great concern? Does it really require a terabyte of working memory to index and serve all of the data for HN? (4x 256GB / 128 CPUs is mentioned in your ops details) This is a lot of resources! Like A LOT! Have you thought considered indexing other high-quality data sources? For example: * Lobste.rs (I think you can email them requesting a DB dump, they want disclosures about the intended purpose) * Slashdot (debatable quality, but goes back 27 years which could be interesting) * Review sites: Chipsandcheese, TomsHardware, Anandtech, HardOCP * Lwn, Phoronix * I'm surely missing other good ones, the discussion in The End of Anandtech article from today mentions a bunch of interesting sources: https://news.ycombinator.com/item?id=41399872 https://news.ycombinator.com/item?id=41399872 I wonder if getting some of these data sources through CommonCrawl or archive.org would reduce the crawl+parse annoyance? At some point I want to put together an HN-Awesome-Search page which covers all the custom search indexes HN folks have made over the years. Thank you!

skeptrune 2y ago
>managed to slam together every AI buzzword into a semi usable product! THANK YOU! Initial motivation for us was, to a large extent, making it easier to throw data into an API and test to see if all the latest AI buzzword tech was useful for your problem. Hearing that we've done that in a semi-usable way is energizing! > re-index and keep updating Yes. It updates in real-time and we are collecting CTR data and analytics to continually fine-tune. There's also a "Rate Results" button which we are looking at the feedback from to improve. > partial updates Partial updates work really well and we are doing it constantly with no impact to performance or uptime. Full indexing from zero is only required if we want to make an index-wide change to improve relevance. > GPUs strictly necessary? Is it possible to use only CPUs if indexing speed isn't a great concern? They are not strictly required. But, inference on CPU is going to be 300+ms at minimum which makes the search feel very laggy. Indexing on CPU is actually less painful than search, because it's lot more ok for that to be slow. > Does it really require a terabyte of working memory to index and serve all of the data for HN? (4x 256GB / 128 CPUs is mentioned in your ops details) We didn't build our own db engine solution and are running Qdrant for this. We certainly tried to get memory usage down and this is the best we got. Qdrant's calculator says this should only take 128gb's with Dense vectors and binary quantization, but we just didn't find that to be true. However, it may be the case that we can improve this further with time. > indexing other high-quality data sources Maybe? I'm somewhat active on lobste.rs, but don't partipate much in the other communities which reduces my interest. HN is really cool since I understand it enough to get a feel for how good the search is which helps us tune things for the overall product. > HN-Awesome_Search page You should! Checkout out the last section of our "History of HN Search" blog to get a starting point at least --> https://trieve.ai/history-of-hnsearch/ https://trieve.ai/history-of-hnsearch/ Tried to be very thorough.

skeptrune 2y ago

>managed to slam together every AI buzzword into a semi usable product! THANK YOU! Initial motivation for us was, to a large extent, making it easier to throw data into an API and test to see if all the latest AI buzzword tech was useful for your problem. Hearing that we've done that in a semi-usable way is energizing! > re-index and keep updating Yes. It updates in real-time and we are collecting CTR data and analytics to continually fine-tune. There's also a "Rate Results" button which we are looking at the feedback from to improve. > partial updates Partial updates work really well and we are doing it constantly with no impact to performance or uptime. Full indexing from zero is only required if we want to make an index-wide change to improve relevance. > GPUs strictly necessary? Is it possible to use only CPUs if indexing speed isn't a great concern? They are not strictly required. But, inference on CPU is going to be 300+ms at minimum which makes the search feel very laggy. Indexing on CPU is actually less painful than search, because it's lot more ok for that to be slow. > Does it really require a terabyte of working memory to index and serve all of the data for HN? (4x 256GB / 128 CPUs is mentioned in your ops details) We didn't build our own db engine solution and are running Qdrant for this. We certainly tried to get memory usage down and this is the best we got. Qdrant's calculator says this should only take 128gb's with Dense vectors and binary quantization, but we just didn't find that to be true. However, it may be the case that we can improve this further with time. > indexing other high-quality data sources Maybe? I'm somewhat active on lobste.rs, but don't partipate much in the other communities which reduces my interest. HN is really cool since I understand it enough to get a feel for how good the search is which helps us tune things for the overall product. > HN-Awesome_Search page You should! Checkout out the last section of our "History of HN Search" blog to get a starting point at least --> https://trieve.ai/history-of-hnsearch/ https://trieve.ai/history-of-hnsearch/ Tried to be very thorough.