4 ms·
Are you saying you wrote an alternative to Lucene because users don't like to type in search queries that Lucene understands? Lucene is a low-level library for
by DavidMcLaughlin 15y ago
Are you saying you wrote an alternative to Lucene because users don't like to type in search queries that Lucene understands? Lucene is a low-level library for implementing a search experience. If you don't understand that then I don't think you understand how a sophisticated search engine like Google works.
Say you type in:
"Lucene equivalent in C++ or C#"
This query is not passed as-is to some magic implementation of a Search Engine by Google.
First they need to parse this query. This involves a tokenizer step which will have many rules depending on the context and the language requirements, etc. An industrial strength tokenizer is a significant undertaking in itself.
The next step is turn this user query into something that a search engine understands. Unless your default mode of operation is "match this user query EXACTLY" then that means having some kind of query language. For any search engine, this will end up looking like the syntax you have a problem with in your post.
Next, to have any kind of useful and competitive search experience you will need to take the user query and generates tens, maybe even hundreds or thousands, of different sub-queries that try to explore different estimates of user intent and fuzzy-matching. For example, you might do some analysis and realise that equivalent in this context is not very important and decide to generate another twenty queries with all-known synonyms for the word equivalent. You might decide to try the query with certain words dropped altogether.
And of course, you also need to be able to rank results. Not just within sub-queries but when merging all the results from sub-queries together. This takes sophisticated and highly customised scoring algorithms that will be dependent on context and a whole lot of other things. Essentially you need to guess what is the best match based on what you think the user wants.
Finally, to be an industry leader like Google you will use a data-driven approach to scoring that makes use of machine-learning techniques to keep your advantage more than a purely technology one.
For companies that just want a basic search that will behave intuitively for people used to the Google model, Lucene helps with A LOT of this. Their default query parser, scorer and other vital components are good enough that most websites can just use the default configurations out of the box and get a good search experience.
All that you're left with in the end is to create the basic user interface abstraction - the "one box" experience. Solr helps enormously with this, but in the end all the functionality it exposes is Lucene.
For more complicated search experiences, there isn't much that I described in the above approach that you can't do in Lucene.
So yeah, Lucene is awesome.