3 ms·

I don't totally understand that middle step "So why not search filters instead of products". It just means searching over the human-edited metadata instead of f

I don't totally understand that middle step "So why not search filters instead of products". It just means searching over the human-edited metadata instead of free-for-all free text fields that were scraped from who knows where? When we get to the last step, of implementing bitmaps for filters, and then trying to optimize to deal with the RAM implications... why not just use Solr or ElasticSearch instead of reinventing the wheel? That's basically what lucene (which Solr and ES are built on) is doing, something like that implementation you came up with, but with lots of optimizations and tunings on top of the naive implementation.

deleted 4y ago
[deleted]
danpalmer 4y ago
> It just means searching over the human-edited metadata instead of free-for-all free text fields that were scraped from who knows where? Not quite. Rather than the free text search being over products, it's over combinations of filters. These filters can then be applied to produce "exact" results, rather than "fuzzy" results based on textual matches. > why not just use Solr or ElasticSearch instead of reinventing the wheel A few reasons. Firstly, the faceted search in these took a fair bit of effort to expose to users in a way that wouldn't result in bad results (like "blue shirt" returning blue non-shirts, and non-blue shirts) Secondly, these are new infrastructure components that take a lot of work to run. ElasticSearch is notorious for being a pain to operate. We were a small engineering team, only 3-5 backend engineers in the company at the time (depending on how you count), running the whole stack, infra, everything.