3 ms·

Bit offtopic, but does anyone have suggestions for simple full text search engines? I basically want something for names, just a couple of thousand, nothing fan

by huskyr 12y ago

Bit offtopic, but does anyone have suggestions for simple full text search engines? I basically want something for names, just a couple of thousand, nothing fancy. Setting up something like ElasticSearch seems like overkill (and is quite hungry for specs as well). I was thinking about simply hacking something together with Redis and Python, but i suppose someone might have a better solution.

ddorian43 12y ago
Python has a search-engine: whoosh
- huskyr 12y ago
  Whoosh sounds pretty much what i'm looking for. Thanks!
  - mhuffman 12y ago
    Also, if you are using Python, sqlite3 has some pretty advanced text searching capabilities that will scale past your "few thousand" with no problem.
YorickPeterse 12y ago
If you're using PostgreSQL you can take advantage of its full text search support. When doing so, make sure to save the text search vectors in a physical column as otherwise queries will be quite slow.
- huskyr 12y ago
  I would definitely take that option if we would use PostgreSQL, but we don't. PostgreSQL sounds like a wonderful piece of software, but they need to work on making it more accessible and user friendly. I tried getting something up and running, but given that there isn't a proper native client for Mac i gave up :(
  - cglace 12y ago
    It is really easy to install using http://brew.sh/ http://brew.sh/
    - huskyr 12y ago
      The command-line part and executables are indeed, but it's lacking in visual tools. Say, the equivalents of Sequel Pro and Phpmyadmin.
      - ewams 12y ago
        http://www.pgadmin.org/ http://www.pgadmin.org/ http://phppgadmin.sourceforge.net/doku.php http://phppgadmin.sourceforge.net/doku.php
ignoramous 12y ago
Sorry if come off as naive (as I don't really understand what a 'full text search' is), but for a couple thousand names wouldn't grep with regex wildcards suffice?
- huskyr 12y ago
  That doesn't sound like a bad suggestion at all. I think it might get bigger over time, or i might have aliases for the names, and in the end using just regex i'll probably hit a ceiling sooner than later.
mtrn 12y ago
I've worked a bit with a nice search library in Go called bleve[1]. That said, it is a library and you would have to implement a server component yourself. Bleve comes with a few command line utils: https://github.com/blevesearch/bleve/tree/master/utils https://github.com/blevesearch/bleve/tree/master/utils [1] http://www.blevesearch.com/ http://www.blevesearch.com/
- huskyr 12y ago
  Thanks!
richardbrevig 12y ago
What exactly are you trying to do? ElasticSearch isn't that difficult and the new guide they published a few months ago walks you through getting it set up without going too in-depth [1]. As a beginner I had it running on a $5 digitalocean droplet within a couple hours and indexed way over "a couple thousand" documents before the end of the day. But if your needs are really that simple, MySQL does support full-text search [2]. [1] http://www.elastic.co/guide/en/elasticsearch/guide/current/intro.html http://www.elastic.co/guide/en/elasticsearch/guide/current/i... [2] https://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html https://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
- huskyr 12y ago
  Yes, i've looked into ES before, and if my usecase would be anything more complicated i would definitely go for that route. However, it's still quite some work to setup, and they recommend at least 8GB of RAM for a working instance [1]. I didn't know MySQL had fulltext search as well. I'll look into that. Thanks. [1]: http://www.elastic.co/guide/en/elasticsearch/guide/master/hardware.html http://www.elastic.co/guide/en/elasticsearch/guide/master/ha...

by huskyr 12y ago

ddorian43 12y ago

Python has a search-engine: whoosh

huskyr 12y ago
Whoosh sounds pretty much what i'm looking for. Thanks!
- mhuffman 12y ago
  Also, if you are using Python, sqlite3 has some pretty advanced text searching capabilities that will scale past your "few thousand" with no problem.

YorickPeterse 12y ago

If you're using PostgreSQL you can take advantage of its full text search support. When doing so, make sure to save the text search vectors in a physical column as otherwise queries will be quite slow.

huskyr 12y ago
I would definitely take that option if we would use PostgreSQL, but we don't. PostgreSQL sounds like a wonderful piece of software, but they need to work on making it more accessible and user friendly. I tried getting something up and running, but given that there isn't a proper native client for Mac i gave up :(
- cglace 12y ago
  It is really easy to install using http://brew.sh/ http://brew.sh/
  - huskyr 12y ago
    The command-line part and executables are indeed, but it's lacking in visual tools. Say, the equivalents of Sequel Pro and Phpmyadmin.
    - ewams 12y ago
      http://www.pgadmin.org/ http://www.pgadmin.org/ http://phppgadmin.sourceforge.net/doku.php http://phppgadmin.sourceforge.net/doku.php

ignoramous 12y ago

Sorry if come off as naive (as I don't really understand what a 'full text search' is), but for a couple thousand names wouldn't grep with regex wildcards suffice?

huskyr 12y ago
That doesn't sound like a bad suggestion at all. I think it might get bigger over time, or i might have aliases for the names, and in the end using just regex i'll probably hit a ceiling sooner than later.

mtrn 12y ago

I've worked a bit with a nice search library in Go called bleve[1]. That said, it is a library and you would have to implement a server component yourself. Bleve comes with a few command line utils: https://github.com/blevesearch/bleve/tree/master/utils https://github.com/blevesearch/bleve/tree/master/utils [1] http://www.blevesearch.com/ http://www.blevesearch.com/

huskyr 12y ago
Thanks!

richardbrevig 12y ago

What exactly are you trying to do? ElasticSearch isn't that difficult and the new guide they published a few months ago walks you through getting it set up without going too in-depth [1]. As a beginner I had it running on a $5 digitalocean droplet within a couple hours and indexed way over "a couple thousand" documents before the end of the day. But if your needs are really that simple, MySQL does support full-text search [2]. [1] http://www.elastic.co/guide/en/elasticsearch/guide/current/intro.html http://www.elastic.co/guide/en/elasticsearch/guide/current/i... [2] https://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html https://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html

huskyr 12y ago
Yes, i've looked into ES before, and if my usecase would be anything more complicated i would definitely go for that route. However, it's still quite some work to setup, and they recommend at least 8GB of RAM for a working instance [1]. I didn't know MySQL had fulltext search as well. I'll look into that. Thanks. [1]: http://www.elastic.co/guide/en/elasticsearch/guide/master/hardware.html http://www.elastic.co/guide/en/elasticsearch/guide/master/ha...