4 ms·

Or you could actually use something designed for indexing data and searches, like Elasticsearch or Solr. Either solution would have no problem indexing all the

Or you could actually use something designed for indexing data and searches, like Elasticsearch or Solr. Either solution would have no problem indexing all their data, rather than having to limit it to a subset to fit in a in-memory table.

herge 13y ago
Does Elasticsearch or Solr work with tabular data, can you search across multiple columns?
- Tomdarkness 13y ago
  Yes you can, and a whole lot more.
- Xylakant 13y ago
  I can't speak well for Solr, but with elasticsearch the answer depends on what you exactly mean with "search across multiple columns", but it's probably yes.
- arethuza 13y ago
  Lucene models documents as a collection of fields, each with a textual value. http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/document/Document.html http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/d... At search time you can use the default field or specify the fields to be searched: http://lucene.apache.org/core/2_9_4/queryparsersyntax.html#Fields http://lucene.apache.org/core/2_9_4/queryparsersyntax.html#F...
- waterlion 13y ago
  Yes. Read the example SOLR schema, it's very easy to understand.
webstartupper 13y ago
Thanks for the info. I haven't previously come across Elastisearch or Solr. Will definitely check these out.
jcampbell1 13y ago
Those solutions assume text is tokenizable, and suck at regex. I think they are the wrong tool for anything having to do with domain search. Furthermore, they don't play nice with continuous cron indexing.
- dalore 13y ago
  solr has real time indexing and batch indexing
randomnumber314 13y ago
>Elasticsearch >Register to watch Any site that requires me to register an account before I can even look at their product is going to have a bad time.
- steinnes 13y ago
  As far as I know you can download ElasticSearch from here without registering: http://www.elasticsearch.org/download/ http://www.elasticsearch.org/download/ ... what better way to check out the product than play around with it?
- Tomdarkness 13y ago
  It is a open source project. They sell support, but elasticsearch itself is licensed under the Apache license. Just click overview if you want to kniw what it is, I'm not sure what you are clicking on that requires registration. Perhaps their training sessions?
  - Spoom 13y ago
    Their videos require registration but the docs do not.
al2o3cr 13y ago
+1 for Solr - looking at the search on domcop.com, it seems like a perfect fit for the faceting stuff. There might be some messiness with the regex search (looking for user-defined patterns of consonants, vowels, etc) as I've never needed to set that up, but the rest would be really clean. See also this blog post that discusses how RoomKey uses read-only Solr instances that are repopulated daily to speed up searching for latency-insensitive data (think "does this hotel have a pool?" etc): http://www.colinsteele.org/post/23103789647/against-the-grain-aws-clojure-startup http://www.colinsteele.org/post/23103789647/against-the-grai...
beersigns 13y ago
This was initial thought as well, seems like both would fit the needs specified, multiple column search etc. I've had good experiences with both ES and Solr. They both have pretty healthy user bases and are well documented. Only hang-up is if you use primarily reg-ex style searching; turning on n-grams could work there, but it might be too slow.
tyw 13y ago
I haven't used either elasticsearch or solr, so I don't have firsthand experience on how they compare to Sphinx search (http://sphinxsearch.com http://sphinxsearch.com) but I really love Sphinx. Also integrates nicely with MySQL if you're already running that. I think Solr and Sphinx pretty much tick the same boxes in terms of features and performance though, so use what you know and like.

herge 13y ago

Does Elasticsearch or Solr work with tabular data, can you search across multiple columns?

Tomdarkness 13y ago
Yes you can, and a whole lot more.
Xylakant 13y ago
I can't speak well for Solr, but with elasticsearch the answer depends on what you exactly mean with "search across multiple columns", but it's probably yes.
arethuza 13y ago
Lucene models documents as a collection of fields, each with a textual value. http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/document/Document.html http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/d... At search time you can use the default field or specify the fields to be searched: http://lucene.apache.org/core/2_9_4/queryparsersyntax.html#Fields http://lucene.apache.org/core/2_9_4/queryparsersyntax.html#F...
waterlion 13y ago
Yes. Read the example SOLR schema, it's very easy to understand.

webstartupper 13y ago

Thanks for the info. I haven't previously come across Elastisearch or Solr. Will definitely check these out.

jcampbell1 13y ago

Those solutions assume text is tokenizable, and suck at regex. I think they are the wrong tool for anything having to do with domain search. Furthermore, they don't play nice with continuous cron indexing.

dalore 13y ago
solr has real time indexing and batch indexing

randomnumber314 13y ago

>Elasticsearch >Register to watch Any site that requires me to register an account before I can even look at their product is going to have a bad time.

steinnes 13y ago
As far as I know you can download ElasticSearch from here without registering: http://www.elasticsearch.org/download/ http://www.elasticsearch.org/download/ ... what better way to check out the product than play around with it?
Tomdarkness 13y ago
It is a open source project. They sell support, but elasticsearch itself is licensed under the Apache license. Just click overview if you want to kniw what it is, I'm not sure what you are clicking on that requires registration. Perhaps their training sessions?
- Spoom 13y ago
  Their videos require registration but the docs do not.

al2o3cr 13y ago

+1 for Solr - looking at the search on domcop.com, it seems like a perfect fit for the faceting stuff. There might be some messiness with the regex search (looking for user-defined patterns of consonants, vowels, etc) as I've never needed to set that up, but the rest would be really clean. See also this blog post that discusses how RoomKey uses read-only Solr instances that are repopulated daily to speed up searching for latency-insensitive data (think "does this hotel have a pool?" etc): http://www.colinsteele.org/post/23103789647/against-the-grain-aws-clojure-startup http://www.colinsteele.org/post/23103789647/against-the-grai...

beersigns 13y ago

This was initial thought as well, seems like both would fit the needs specified, multiple column search etc. I've had good experiences with both ES and Solr. They both have pretty healthy user bases and are well documented. Only hang-up is if you use primarily reg-ex style searching; turning on n-grams could work there, but it might be too slow.

tyw 13y ago

I haven't used either elasticsearch or solr, so I don't have firsthand experience on how they compare to Sphinx search (http://sphinxsearch.com http://sphinxsearch.com) but I really love Sphinx. Also integrates nicely with MySQL if you're already running that. I think Solr and Sphinx pretty much tick the same boxes in terms of features and performance though, so use what you know and like.