4 ms·

Although in the past I've implemented Xapian [1] over Sphinx [2], Sphinx today seems to be much better, but both Xapian and Sphinx are under-appreciated compare

by kolev 12y ago

Although in the past I've implemented Xapian [1] over Sphinx [2], Sphinx today seems to be much better, but both Xapian and Sphinx are under-appreciated compared to Solr [3] and Elasticsearch [4]. [1] http://xapian.org/ http://xapian.org/ [2] http://sphinxsearch.com/ http://sphinxsearch.com/ [3] http://lucene.apache.org/solr/ http://lucene.apache.org/solr/ [4] http://www.elasticsearch.org/ http://www.elasticsearch.org/

brongondwana 12y ago
All of those last three are awesome if you either put all the user's search indexes into a single engine, or have a shit-ton of memory. With sphinx, we found we had to start and stop daemons all over the place to manage memory, and it was just unworkable. It was either that or run one big index per machine, but there are operational reasons I'd rather not be doing that. We try to keep everything user-sized. That said, there's still stub Sphinx code in there. Both engines are have GPL licensing on them, which means compiling against Cyrus (BSD licensed) causes a non-BSD licensed end result. Not an issue for us, since we publish all our Cyrus code anyway. There is talk of building an Elasticsearch backed into Cyrus as well - feel free, it's all open source. We'd definitely take the patch if it's good code (he says with his Cyrus Project Board Member hat on rather than his FastMail Director hat on)
- gregnbanks 12y ago
  Sphinx also had a bunch of really bad bugs around server startup and shutdown, and some ugly code. I ended re-writing some of their pthreads code and submitting patches, I have no idea if they ever used them or not because their development tree is internal and not visible outside their company. Solr (well, Lucene) has awesome natural language stemming abilities, many more languages supported than Xapian. In particular it's much smarter than Xapian about Chinese. But a) the memory requirement makes running multiple shards on the same machine hard, and b) nobody in the company wanted to learn how to handle Java operationally. EDIT: since 2012 Sphinx appear to have made a public mirror of their internal tree at https://code.google.com/p/sphinxsearch/source/checkout https://code.google.com/p/sphinxsearch/source/checkout

by kolev 12y ago

brongondwana 12y ago

All of those last three are awesome if you either put all the user's search indexes into a single engine, or have a shit-ton of memory. With sphinx, we found we had to start and stop daemons all over the place to manage memory, and it was just unworkable. It was either that or run one big index per machine, but there are operational reasons I'd rather not be doing that. We try to keep everything user-sized. That said, there's still stub Sphinx code in there. Both engines are have GPL licensing on them, which means compiling against Cyrus (BSD licensed) causes a non-BSD licensed end result. Not an issue for us, since we publish all our Cyrus code anyway. There is talk of building an Elasticsearch backed into Cyrus as well - feel free, it's all open source. We'd definitely take the patch if it's good code (he says with his Cyrus Project Board Member hat on rather than his FastMail Director hat on)

gregnbanks 12y ago
Sphinx also had a bunch of really bad bugs around server startup and shutdown, and some ugly code. I ended re-writing some of their pthreads code and submitting patches, I have no idea if they ever used them or not because their development tree is internal and not visible outside their company. Solr (well, Lucene) has awesome natural language stemming abilities, many more languages supported than Xapian. In particular it's much smarter than Xapian about Chinese. But a) the memory requirement makes running multiple shards on the same machine hard, and b) nobody in the company wanted to learn how to handle Java operationally. EDIT: since 2012 Sphinx appear to have made a public mirror of their internal tree at https://code.google.com/p/sphinxsearch/source/checkout https://code.google.com/p/sphinxsearch/source/checkout