3 ms·

Memory lane. Way back in the mists of time, circa 2004 or 5 I wanted to learn about search indexing. I was a pre-rails ruby head and translated a large portion

Memory lane. Way back in the mists of time, circa 2004 or 5 I wanted to learn about search indexing. I was a pre-rails ruby head and translated a large portion of lucene's index code into ruby as a learning exercise. The result was abysmal and somewhat wonky, but I did learn a bunch. Both about lucene and ruby's FFI. Reading and translating code is such a great way to internalize a concept that you're unfamiliar with, while getting a glimpse into someone else's mental model. Lots of people tell you to read code, but it's hard to overstate the power of filtering a codebase through your brain and out your fingertips.

rjbwork 5y ago
Neat! A bit over a decade ago, I had a similar experience with Lucene (not that I implemented it from scratch, but certainly used it in a fairly unorthodox, for the time, manner). I had to implement some search stuff and Elastic Search was still in its infancy so was not necessarily the "obviously right choice" as it has been recently for this kind of document search job. I implemented a multi-tenant search engine on top of Lucene using C# and Azure Blob Storage under the direction of my manager at the time. This was actually pretty cool, because I had actually learned about TF-IDF and search technologies in school so it was nice to be using some of that knowledge. And there were a lot of problems to solve with regards to locking, index update coordination, etc. that, as we know, ES takes care of for us today. Anyway, the project was a success, and launched, and backed a couple of products for a couple of years until it was decommissioned due to outside forces basically making it irrelevant. That knowledge and experience seems to have ultimately led me down the path to becoming the resident ES "expert" at my current position.

3 ms·

Memory lane. Way back in the mists of time, circa 2004 or 5 I wanted to learn about search indexing. I was a pre-rails ruby head and translated a large portion

by weeksie 5y ago

rjbwork 5y ago
Neat! A bit over a decade ago, I had a similar experience with Lucene (not that I implemented it from scratch, but certainly used it in a fairly unorthodox, for the time, manner). I had to implement some search stuff and Elastic Search was still in its infancy so was not necessarily the "obviously right choice" as it has been recently for this kind of document search job. I implemented a multi-tenant search engine on top of Lucene using C# and Azure Blob Storage under the direction of my manager at the time. This was actually pretty cool, because I had actually learned about TF-IDF and search technologies in school so it was nice to be using some of that knowledge. And there were a lot of problems to solve with regards to locking, index update coordination, etc. that, as we know, ES takes care of for us today. Anyway, the project was a success, and launched, and backed a couple of products for a couple of years until it was decommissioned due to outside forces basically making it irrelevant. That knowledge and experience seems to have ultimately led me down the path to becoming the resident ES "expert" at my current position.