2 ms·
Lifecycle management is also a big part of it, I think. My search engine is a hybrid architecture, with some monoliths and some microservices. * Index - 5+ mi
by marginalia_nu 5y ago
Lifecycle management is also a big part of it, I think.
My search engine is a hybrid architecture, with some monoliths and some microservices.
* Index - 5+ minute start time and uses 60% of system RAM.;
* Search Server (query parser, etc.) - 30 second start time due to having to load a bunch of term frequency data, low resources, ephemeral state
* Assistant Server (dictionary lookups, spell checks, etc.) - fast start, medium resources, stateless
* Crawler - only runs sometimes, high resources, statefull
* Archive Server- fast start, low resources, ephemeral state
* Public API gateway - 5 second start time, low resources, ephemeral state
+ a few others
A lot of the ways it's divided is along the lines of minimizing disruption when deploying changes. I don't have the server resources to run secondary instances of any of these, so I'm working with what I've got. If I patch the crawler, I don't want the search function to go down. If I patch the search function, I don't want to have to wait 5 minutes to restart the index.
It would certainly be cleaner to say break apart the Index service in terms of design, as it does several disparate things, but those things have a resource synergy which means I can't, not without buying another server and running a small 100 GbE network between them. Seems silly for a living room operation.