4 ms·
Ex-Google search engineer here, now using hosted ElasticSearch extensively in my startup. This is a really good overview. If there's one part I want to highli
by nostrademons 9y ago
Ex-Google search engineer here, now using hosted ElasticSearch extensively in my startup.
This is a really good overview. If there's one part I want to highlight, it's that you should expect to spend a lot of time fine-tuning your ranking function for your particular product & corpus. The default ElasticSearch ranking function kinda sucks. It was changed in ES 5.0 to Okapi BM25, which is the current academic state-of-the-art in non-machine-learned ranking functions. However, search is one field where the current academic state-of-the-art is at least a couple decades behind where things are in industry. When you use a service with good search that just works, chances are that there's been a lot of engineer hours devoted to identifying exactly which signals are most useful in your corpus, how they relate to subjective evaluations of relevance, and how to clean them up so that noise doesn't dominate the signal.
These signals are tightly tied to the corpus you're working with. One slightly unpleasant surprise striking out on my own was just how little the stuff I learned about websearch is relevant to working with different corpora (though I guess I should've predicted this, having seen how many different teams across Google Search operated and how all their signals differed from core websearch). I'll reiterate the article's point about process and method being more important than any specific function or algorithm; the most useful stuff I learned at Google was actually a process for taking a vague domain where I'm not sure what's out there, and then learning how to turn that into a system for getting useful information out of it.
And then once you've got users, you probably want to feed usage data back into it through some sort of machine learning. In that order, though; with all the hotness around AI, it's really tempting to think "Oh, I'll just read the latest LtR papers, Mechanical Turk a bunch of query evaluations, and feed the data into the algorithm", but without a deeper understanding of the particular corpus you're working with and how users want to access it, it's unlikely an initial LtR system will perform well enough to attract enough users to bootstrap the system.
- makmanalp 9y ago> When you use a service with good search that just works, chances are that there's been a lot of engineer hours devoted to identifying exactly which signals are most useful in your corpus I imagine at a place like Google, your "corpus" is just about everything under the sun, as are your queries. (i.e. less chance for a subject-specific tuning) What happens then?
- nostrademons 9y agoGoogle's search departments were for years organized by the type of content being indexed, i.e. Web Search, Image Search, Video Search, Product Search, Book Search, Blog Search, Discussion Search (RIP), Google News, Scholar, etc. For a long time these distinctions were apparent in the UI (they were de-emphasized after I left, which IMHO was a very good thing), but the differences go way down through the search stack, with different indexes, different ranking functions, and in some cases different ingestion mechanisms. As for what happens when these become inadequate for the queries users ask...well, buy Metaweb and rebrand it as the Knowledge Graph. ;-)
- mayank 9y ago> I imagine at a place like Google, your "corpus" is just about everything under the sun, as are your queries. > What happens then? You accept the fact that a good search engine is inevitably going to be a multi-layered beast that requires constant effort to improve, with heuristics, machine learning, magic constants, special lists of words, and regular expressions. After that, it's just a matter of picking a commercially useful objective function ("instant search", "query autocomplete", "maximize user satisfaction") and pointing large teams of well-resourced PhDs at it. Probably a whole weekend at least.
- garysieling 9y agoIf you can identify categories of queries or documents that interest people, you can treat tuning that part of the search engine as if it was a new product.
- thatusernametho 9y agowhat does your startup do?
- nostrademons 9y agoCross between an RSS-reader and a search engine - it lets you subscribe to a topic instead of a site, so that you don't have to manage hundreds of subscriptions, and if a new discussion pops up somewhere on the web that's relevant to your interests, it'll find it and let you know. As forums get boring or off-topic and new ones spring up, it adjusts automatically, so you don't need to do the "Does anyone know of other places on the web like X?" dance. Still under development, but drop me an e-mail if you (or anyone else reading this) is interested in beta-testing. I'm starting out e-mail first, so the initial UI is just that you get a daily digest of links & snippets to threads related to your interests.
- Gigablah 9y agoAh, so something like ElasticSearch percolater queries but applied to the web.
- nostrademons 9y agoYep, pretty much. The percolater queries are fairly handy in this instance, and one of the things we didn't really have at Google.
- exikyut 9y agoI just read https://stackoverflow.com/questions/21536599/what-does-percolator-mean-do-in-elasticsearch https://stackoverflow.com/questions/21536599/what-does-perco..., and was strongly reminded of Google Wave's "search for documents that haven't been created yet, and they show up when they appear" functionality. Without trying to get too much off track, I gotta say that it would be so nice to be able to use the original theme data used in Wave (very obviously sans branding). Wave In A Box is... blech, in terms of design, I have to admit.