7 ms·

Major Changes from Solr 4 to Solr 5

103 points by chatman 12y ago

deleted 12y ago
[deleted]
thinkcomp 12y ago
Does Solr 5.0 support password-protecting the admin interface yet without spending hours trying to wrangle custom XML files? It seems like a pretty basic requirement for a web-based application. I've tried things like this http://community.zimbra.com/documentation/w/documentation/securing-solr-on-tomcat http://community.zimbra.com/documentation/w/documentation/se... repeatedly. They never seem to work right.
- Tharkun 12y ago
  Solr is not a web-based application. You shouldn't directly expose your Solr instance to anyone. Regardless of whether or not you secure your admin interface. That's not Solr's core business, and I don't see why they should was their efforts on it. Have Solr listen on localhost and have your web app talk to Solr. If your Solr is visible to the world, you're doing it wrong. Edit: by saying that it's not a web-based application I mean that it shouldn't be on teh interwebz -- it's obviously a webapp in the sense that it mostly speaks HTTP.
  - tomp 12y ago
    Well, regardless of whether it's web-facing or not, it's not unreasonable to want to limit the access to its admin panel (e.g. in a big company with different teams). I agree however that SOLR is best off doing one thing well, web page security can be implemented e.g. by Apache.
    - imaginenore 12y ago
      So limit it to only whitelisted IPs. That's why you have sysadmins.
      - untog 12y ago
        Or use Nginx as a proxy, or something. It's frustrating, but I can see the argument for Solr to just delegate this kind of task to other projects that do it better.
    - Tharkun 12y ago
      You would be right if the Solr admin page were the only administrative interface. It's not. You could send delete queries, create additional indexes, etc all without using the admin page, simply by sending http requests to the relevant Solr components. Instead of adding overhead by securing each individual call, they leave it up to you. Or to whichever friendly consultant you decide to hire to help out with that. Wink wink. Nudge nudge.
  - BonoboBoner 12y ago
    While you are technically right, that kind of thinking gave people access to a ton of MongoDB databases last week.
  - gregors 12y ago
    The admin is a web application I can tell this because I'm looking at my browser. ;) Not wanting to deal with the boring parts of web apps -- that's understandable but not business/user savvy.
- gchanan 12y ago
  Disclaimer: I work at Cloudera on Solr and related technologies. I don't think there's anything out of the box in Solr 5.0 that changes that. SOLR-4470[0] should be able to do that, but it hasn't been committed. Apache Sentry[1] adds role based access control to Solr, but it's only been tested up to Solr 4.10 and with kerberos (not basic password protection). It comes nicely integrated out-of-the-box with Solr as part of Cloudera Search[2]; otherwise, you'll have to do some manual setup to get it to work. [0] https://issues.apache.org/jira/browse/SOLR-4470 https://issues.apache.org/jira/browse/SOLR-4470 [1] https://sentry.incubator.apache.org/ https://sentry.incubator.apache.org/ [2] http://www.cloudera.com/content/cloudera/en/products-and-services/cdh/search.html http://www.cloudera.com/content/cloudera/en/products-and-ser...
- jonbaer 12y ago
  If you need a solid easy-to-config security wrapped around Solr I would recommend taking a peek @ Fusion from Lucidworks, http://lucidworks.com/product/solr-enterprise/ http://lucidworks.com/product/solr-enterprise/
- ankit-singh 12y ago
  Make your solr listen to localhost, and put a reverse-proxy system like nginx to access it via web and control authentication from there.
sagivo 12y ago
Any reasons to prefer Solr over elastic-search?
- chatman 12y ago
  Solr is backed by the Apache community while ES is backed by a private entity. Is that a good reason?
  - matthewmacleod 12y ago
    Not really, no. If ElasticsearchCo were to go belly-up, the technology would almost certainly be backed by someone else.
  - capkutay 12y ago
    I don't know what you mean when you say backed by the Apache Community, but Elasticsearch is also an open source, apache-licensed project that has a commercial, private counterpart. I think every major open source project has this now (Hadoop has Cloudera/Hortonworks/MapR, Spark has databricks).
    - TallGuyShort 12y ago
      >> I don't know what you mean when you say backed by the Apache Community It's a matter of who does the releasing (i.e. curating of patches, responsibility for building community, etc.): Solr is governed by the Apache Software Foundation and associated volunteers, ElasticSearch is governed by the private entity. IMO (and this is an increasingly rare opinion) the license is the most important piece and they're both under the ASL 2.0.
      - nemothekid 12y ago
        Governing alone - I don't that thats a good reason. There are plenty of Apache projects that are effectively governed by a private entity (Cassandra, Samza, Kafka, and these are all in the "big data" space), that would most likely be as useful as Elasticsearch if their supporting companies disappeared overnight.
deleted 12y ago
[deleted]
krat0sprakhar 12y ago
At my company, we've been beating our heads to the wall[0] in getting multi-term synonyms to work correctly in SOLR. e.g. fruit extractor => fruit juicer, citrus juicer Does anyone experienced enough have a clue if SOLR 5 can help with that? [0] - http://opensourceconnections.com/blog/2013/10/27/why-is-multi-term-synonyms-so-hard-in-solr/ http://opensourceconnections.com/blog/2013/10/27/why-is-mult...
- elchief 12y ago
  This guy fixed it for 3 and 4 https://github.com/healthonnet/hon-lucene-synonyms https://github.com/healthonnet/hon-lucene-synonyms The Solr guys don't give a flying F about this issue though
  - detnavillus 12y ago
    As I said I'm one Solr guy that does 'effin care about this issue - I should have replied on this sub thread, sorry. Will have a submission ready soon for my AutophrasingTokenFilter - when I get the JIRA number, I'll let you know.
- AznHisoka 12y ago
  I've yet to find a NLP library that can give you multi-term synonyms as accurate that.. let alone a SOLR module/fix.
- oomkiller 12y ago
  You could possibly change the text before indexing to solve this problem.
mikeblum 12y ago
looking at the solr downloads page, I can't seem to find the Solr 5 tarball... Clicking downloads: http://lucene.apache.org/solr/mirrors-solr-latest-redir.html http://lucene.apache.org/solr/mirrors-solr-latest-redir.html redirects to the 4. tarball, i added 5.0.0.to the url and got the latest tarbal: http://www.apache.org/dyn/closer.cgi/lucene/solr/5.0.0 http://www.apache.org/dyn/closer.cgi/lucene/solr/5.0.0
AznHisoka 12y ago
Still no percolator/reverse search. ElasticSearch still is my go-to search technology.
- johnx123-up 12y ago
  I've checked http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-percolate.html http://www.elasticsearch.org/guide/en/elasticsearch/referenc... But, couldn't grasp the usecase. Can anyone share some thoughts, where will it be helpful?
  - altcognito 12y ago
    Your customers pay you to let them know when new data is available.
  - AznHisoka 12y ago
    Think Google Alerts or Think "Tell me when price reaches $X"
detnavillus 12y ago
krat0sprakhar: Check out my blogs on this https://lucidworks.com/blog/solution-for-multi-term-synonyms-in-lucenesolr-using-the-auto-phrasing-tokenfilter/ https://lucidworks.com/blog/solution-for-multi-term-synonyms... I also did a Meetup on this just this week http://www.slideshare.net/detnavillus/the-well-tempered-search-application http://www.slideshare.net/detnavillus/the-well-tempered-sear... Check out slide 18 - autophrasing + synonyms: Precision 100% recall 100% Bag of words OOTB Solr/Lucene NOT so! The code is on github and is a Lucene TokenFilter so it should work. I used 4.10.3 for the Meetup demo
detnavillus 12y ago
elchief: I am a Solr guy and I do give a flying F about this. I've been remiss in submitting this to Solr though - my bad. Working on this now. Nolan's fix is also good. I referenced this work in my blog post

103 points by chatman 12y ago

deleted 12y ago

[deleted]

Does Solr 5.0 support password-protecting the admin interface yet without spending hours trying to wrangle custom XML files? It seems like a pretty basic requirement for a web-based application. I've tried things like this http://community.zimbra.com/documentation/w/documentation/securing-solr-on-tomcat http://community.zimbra.com/documentation/w/documentation/se... repeatedly. They never seem to work right.

Tharkun 12y ago
Solr is not a web-based application. You shouldn't directly expose your Solr instance to anyone. Regardless of whether or not you secure your admin interface. That's not Solr's core business, and I don't see why they should was their efforts on it. Have Solr listen on localhost and have your web app talk to Solr. If your Solr is visible to the world, you're doing it wrong. Edit: by saying that it's not a web-based application I mean that it shouldn't be on teh interwebz -- it's obviously a webapp in the sense that it mostly speaks HTTP.
- tomp 12y ago
  Well, regardless of whether it's web-facing or not, it's not unreasonable to want to limit the access to its admin panel (e.g. in a big company with different teams). I agree however that SOLR is best off doing one thing well, web page security can be implemented e.g. by Apache.
  - imaginenore 12y ago
    So limit it to only whitelisted IPs. That's why you have sysadmins.
    - untog 12y ago
      Or use Nginx as a proxy, or something. It's frustrating, but I can see the argument for Solr to just delegate this kind of task to other projects that do it better.
  - Tharkun 12y ago
    You would be right if the Solr admin page were the only administrative interface. It's not. You could send delete queries, create additional indexes, etc all without using the admin page, simply by sending http requests to the relevant Solr components. Instead of adding overhead by securing each individual call, they leave it up to you. Or to whichever friendly consultant you decide to hire to help out with that. Wink wink. Nudge nudge.
- BonoboBoner 12y ago
  While you are technically right, that kind of thinking gave people access to a ton of MongoDB databases last week.
- gregors 12y ago
  The admin is a web application I can tell this because I'm looking at my browser. ;) Not wanting to deal with the boring parts of web apps -- that's understandable but not business/user savvy.
gchanan 12y ago
Disclaimer: I work at Cloudera on Solr and related technologies. I don't think there's anything out of the box in Solr 5.0 that changes that. SOLR-4470[0] should be able to do that, but it hasn't been committed. Apache Sentry[1] adds role based access control to Solr, but it's only been tested up to Solr 4.10 and with kerberos (not basic password protection). It comes nicely integrated out-of-the-box with Solr as part of Cloudera Search[2]; otherwise, you'll have to do some manual setup to get it to work. [0] https://issues.apache.org/jira/browse/SOLR-4470 https://issues.apache.org/jira/browse/SOLR-4470 [1] https://sentry.incubator.apache.org/ https://sentry.incubator.apache.org/ [2] http://www.cloudera.com/content/cloudera/en/products-and-services/cdh/search.html http://www.cloudera.com/content/cloudera/en/products-and-ser...
jonbaer 12y ago
If you need a solid easy-to-config security wrapped around Solr I would recommend taking a peek @ Fusion from Lucidworks, http://lucidworks.com/product/solr-enterprise/ http://lucidworks.com/product/solr-enterprise/
ankit-singh 12y ago
Make your solr listen to localhost, and put a reverse-proxy system like nginx to access it via web and control authentication from there.

sagivo 12y ago

Any reasons to prefer Solr over elastic-search?

chatman 12y ago
Solr is backed by the Apache community while ES is backed by a private entity. Is that a good reason?
- matthewmacleod 12y ago
  Not really, no. If ElasticsearchCo were to go belly-up, the technology would almost certainly be backed by someone else.
- capkutay 12y ago
  I don't know what you mean when you say backed by the Apache Community, but Elasticsearch is also an open source, apache-licensed project that has a commercial, private counterpart. I think every major open source project has this now (Hadoop has Cloudera/Hortonworks/MapR, Spark has databricks).
  - TallGuyShort 12y ago
    >> I don't know what you mean when you say backed by the Apache Community It's a matter of who does the releasing (i.e. curating of patches, responsibility for building community, etc.): Solr is governed by the Apache Software Foundation and associated volunteers, ElasticSearch is governed by the private entity. IMO (and this is an increasingly rare opinion) the license is the most important piece and they're both under the ASL 2.0.
    - nemothekid 12y ago
      Governing alone - I don't that thats a good reason. There are plenty of Apache projects that are effectively governed by a private entity (Cassandra, Samza, Kafka, and these are all in the "big data" space), that would most likely be as useful as Elasticsearch if their supporting companies disappeared overnight.

deleted 12y ago

[deleted]

krat0sprakhar 12y ago

At my company, we've been beating our heads to the wall[0] in getting multi-term synonyms to work correctly in SOLR. e.g. fruit extractor => fruit juicer, citrus juicer Does anyone experienced enough have a clue if SOLR 5 can help with that? [0] - http://opensourceconnections.com/blog/2013/10/27/why-is-multi-term-synonyms-so-hard-in-solr/ http://opensourceconnections.com/blog/2013/10/27/why-is-mult...

elchief 12y ago
This guy fixed it for 3 and 4 https://github.com/healthonnet/hon-lucene-synonyms https://github.com/healthonnet/hon-lucene-synonyms The Solr guys don't give a flying F about this issue though
- detnavillus 12y ago
  As I said I'm one Solr guy that does 'effin care about this issue - I should have replied on this sub thread, sorry. Will have a submission ready soon for my AutophrasingTokenFilter - when I get the JIRA number, I'll let you know.
AznHisoka 12y ago
I've yet to find a NLP library that can give you multi-term synonyms as accurate that.. let alone a SOLR module/fix.
oomkiller 12y ago
You could possibly change the text before indexing to solve this problem.

mikeblum 12y ago

looking at the solr downloads page, I can't seem to find the Solr 5 tarball... Clicking downloads: http://lucene.apache.org/solr/mirrors-solr-latest-redir.html http://lucene.apache.org/solr/mirrors-solr-latest-redir.html redirects to the 4. tarball, i added 5.0.0.to the url and got the latest tarbal: http://www.apache.org/dyn/closer.cgi/lucene/solr/5.0.0 http://www.apache.org/dyn/closer.cgi/lucene/solr/5.0.0

AznHisoka 12y ago

Still no percolator/reverse search. ElasticSearch still is my go-to search technology.

johnx123-up 12y ago
I've checked http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-percolate.html http://www.elasticsearch.org/guide/en/elasticsearch/referenc... But, couldn't grasp the usecase. Can anyone share some thoughts, where will it be helpful?
- altcognito 12y ago
  Your customers pay you to let them know when new data is available.
- AznHisoka 12y ago
  Think Google Alerts or Think "Tell me when price reaches $X"

detnavillus 12y ago

krat0sprakhar: Check out my blogs on this https://lucidworks.com/blog/solution-for-multi-term-synonyms-in-lucenesolr-using-the-auto-phrasing-tokenfilter/ https://lucidworks.com/blog/solution-for-multi-term-synonyms... I also did a Meetup on this just this week http://www.slideshare.net/detnavillus/the-well-tempered-search-application http://www.slideshare.net/detnavillus/the-well-tempered-sear... Check out slide 18 - autophrasing + synonyms: Precision 100% recall 100% Bag of words OOTB Solr/Lucene NOT so! The code is on github and is a Lucene TokenFilter so it should work. I used 4.10.3 for the Meetup demo

detnavillus 12y ago

elchief: I am a Solr guy and I do give a flying F about this. I've been remiss in submitting this to Solr though - my bad. Working on this now. Nolan's fix is also good. I referenced this work in my blog post