4 ms·

Check out Sphinx and the amazing ThinkingSphinx gem. Having used both extensively, I greatly prefer Sphinx over Solr. http://sphinxsearch.com/ http://sphinxsea

by getsat 16y ago

Check out Sphinx and the amazing ThinkingSphinx gem. Having used both extensively, I greatly prefer Sphinx over Solr. http://sphinxsearch.com/ http://sphinxsearch.com/ http://freelancing-god.github.com/ts/en/ http://freelancing-god.github.com/ts/en/ CollectiveIdea also maintains a good fork of delayed_job for those who are interested in that.

riffraff 16y ago
care to share the reasons of this preference?
- thibaut_barrere 16y ago
  I use both; I prefer sunspot if I need to index arbitrary ruby methods results (eg: methods that result in multiple queries/data clean-up) or if I work with MongoDB. I prefer sphinx when I need a more lightweight setup on RAM, or need an easy geosearch, or work with mysql. EDIT: - sunspot allows for per-request weighting in the search (see http://blog.logeek.fr/2011/3/4/hackerbooks-books-stackoverflow-hackernews http://blog.logeek.fr/2011/3/4/hackerbooks-books-stackoverfl... for an example), not sure if it's possible with ts
  - getsat 16y ago
    That flexible weighting looks nice! Thanks for the link. Yeah, if I ever use a NoSQL datastore on a project I'll likely end up using Solr. Faster than writing a new Sphinx interface for it.
- getsat 16y ago
  Easier to setup, doesn't depend on Java, insanely fast indexing, delta indexes. Basically, it's just a fast little search server that handles everything I've thrown at it. That, and I had a terrible experience using acts_as_solr back in the day...
zmoazeni 16y ago
Thanks getsat for the shout out. Our guys have put in a lot of work adding features and keeping delayed_job up to snuff. I do have to disagree with you on ThinkingSphinx. I have used both, and I gel much better with Solr. Don't get me wrong, the guy(s) who did ThinkingSphinx have worked hard. I just don't like the moving pieces of ThinkingSphinx: continuous rake tasks, the fact that it sits directly on top of the database, extra re-deployment steps when using capistrano. I prefer having the search server sitting apart.
- gtani 16y ago
  Sphinx does have the advantage that it runs fine on a little 256 Meg VPS, where you're not going to shoehorn in Tomcat or Jetty
  - getsat 16y ago
    This and Sphinx's insanely fast indexing speed are two of the main reasons I like it so much. Java is a huge dependency in my opinion. Different strokes for different folks. :)
    - gtani 16y ago
      Yeah, i would say insanely fast indexing speed is a winner at 3 stages of a) developing the 2 or 3 indexes you need for good search results b) running test suites c) production That said, in the past, I haven't been able to transfer the index configuration from sphinx to SOLR indexes and get 98% matches between the 2 engines. Sphinx (thinking sphinx in rails) used to do funny stuff in the pre-1.0, like if you put in 2 search terms you would get X results, if you put in those 2 terms plus another, you would get more results. I think they've fixed most of that in 1.0
postit 16y ago
If you have a really huge dataset go away from sphinx, it takes forever to index and doesn't offer incremental updates, as well it crashes regularly
- brndnhy 16y ago
  No, there is no incremental mode to the indexer, but that doesn't mean it can't be done. Create a counter table to keep track of what records you've indexed and cron frequent indexer runs using the latest records as offsets. If you have documents that frequently change, maintain your full index less often and instead merge the two on a more regular basis. Sphinx's indexer speed is one of its advantages, but it's largely dependent on the efficiency of your SQL and underlying MySQL indices. Perhaps something else is indirectly influencing your indexing performance.
- getsat 16y ago
  What size dataset are you talking about? I've never seen Sphinx crash even when indexing hundreds of gigs of documents. Are there any relevant bug reports?
- thorax 16y ago
  Yeah, we use a large dataset (so does craigslist, etc). I've not seen crashes. I'd be curious to hear more about what you ran into. The indexing speed is so very fast-- I can't say enough good things about sphinx.
brndnhy 16y ago
First of all, Sphinx is fantastic and a majority of people will find it suitable for full-text search, but there are some weaknesses that might make Solr (Lucene, really) more powerful and appropriate for full-text: 1.'did-you-mean' support 2. facets 3. replication 4. spell checking 5. auto-complete is easier to implement with Solr 6. Sphinx doesn't return whole documents, only the doc id 7. Sphinx doc ids must be integers

by getsat 16y ago

riffraff 16y ago

care to share the reasons of this preference?

thibaut_barrere 16y ago
I use both; I prefer sunspot if I need to index arbitrary ruby methods results (eg: methods that result in multiple queries/data clean-up) or if I work with MongoDB. I prefer sphinx when I need a more lightweight setup on RAM, or need an easy geosearch, or work with mysql. EDIT: - sunspot allows for per-request weighting in the search (see http://blog.logeek.fr/2011/3/4/hackerbooks-books-stackoverflow-hackernews http://blog.logeek.fr/2011/3/4/hackerbooks-books-stackoverfl... for an example), not sure if it's possible with ts
- getsat 16y ago
  That flexible weighting looks nice! Thanks for the link. Yeah, if I ever use a NoSQL datastore on a project I'll likely end up using Solr. Faster than writing a new Sphinx interface for it.
getsat 16y ago
Easier to setup, doesn't depend on Java, insanely fast indexing, delta indexes. Basically, it's just a fast little search server that handles everything I've thrown at it. That, and I had a terrible experience using acts_as_solr back in the day...

zmoazeni 16y ago

Thanks getsat for the shout out. Our guys have put in a lot of work adding features and keeping delayed_job up to snuff. I do have to disagree with you on ThinkingSphinx. I have used both, and I gel much better with Solr. Don't get me wrong, the guy(s) who did ThinkingSphinx have worked hard. I just don't like the moving pieces of ThinkingSphinx: continuous rake tasks, the fact that it sits directly on top of the database, extra re-deployment steps when using capistrano. I prefer having the search server sitting apart.

gtani 16y ago
Sphinx does have the advantage that it runs fine on a little 256 Meg VPS, where you're not going to shoehorn in Tomcat or Jetty
- getsat 16y ago
  This and Sphinx's insanely fast indexing speed are two of the main reasons I like it so much. Java is a huge dependency in my opinion. Different strokes for different folks. :)
  - gtani 16y ago
    Yeah, i would say insanely fast indexing speed is a winner at 3 stages of a) developing the 2 or 3 indexes you need for good search results b) running test suites c) production That said, in the past, I haven't been able to transfer the index configuration from sphinx to SOLR indexes and get 98% matches between the 2 engines. Sphinx (thinking sphinx in rails) used to do funny stuff in the pre-1.0, like if you put in 2 search terms you would get X results, if you put in those 2 terms plus another, you would get more results. I think they've fixed most of that in 1.0

postit 16y ago

If you have a really huge dataset go away from sphinx, it takes forever to index and doesn't offer incremental updates, as well it crashes regularly

brndnhy 16y ago
No, there is no incremental mode to the indexer, but that doesn't mean it can't be done. Create a counter table to keep track of what records you've indexed and cron frequent indexer runs using the latest records as offsets. If you have documents that frequently change, maintain your full index less often and instead merge the two on a more regular basis. Sphinx's indexer speed is one of its advantages, but it's largely dependent on the efficiency of your SQL and underlying MySQL indices. Perhaps something else is indirectly influencing your indexing performance.
getsat 16y ago
What size dataset are you talking about? I've never seen Sphinx crash even when indexing hundreds of gigs of documents. Are there any relevant bug reports?
thorax 16y ago
Yeah, we use a large dataset (so does craigslist, etc). I've not seen crashes. I'd be curious to hear more about what you ran into. The indexing speed is so very fast-- I can't say enough good things about sphinx.

brndnhy 16y ago

First of all, Sphinx is fantastic and a majority of people will find it suitable for full-text search, but there are some weaknesses that might make Solr (Lucene, really) more powerful and appropriate for full-text: 1.'did-you-mean' support 2. facets 3. replication 4. spell checking 5. auto-complete is easier to implement with Solr 6. Sphinx doesn't return whole documents, only the doc id 7. Sphinx doc ids must be integers