4 ms·

Apache Lucy - Lucene-like full-text search in C

12 points by ot 15y ago

exDM69 15y ago
The question is, what was the motivation behind re-writing Java Lucene in C?
- Avshalom 15y ago
  It says on the page: so it can be used by dynamic languages.
  - exDM69 15y ago
    That is a very good reason. This is also why I think that most system level libraries should be written in C, because it (unfortunately) is one of the only languages that can be called from pretty much any other language out there. If you go with JVM or CLR, your software will only run on that virtual machine. C++ may be possible to some degree but C++'s pass-by-value semantics make it a bit tricky to export to other languages (with reference based semantics), so if you have a good C++ API, it won't turn into a nice Python API. Exception handling and other issues make C++ also tricky.
hopeless 15y ago
No more Java dependencies in your Ruby webapp? Sounds promising!
ognyankulev 15y ago
Interesting quotes: From http://incubator.apache.org/lucy/faq.html http://incubator.apache.org/lucy/faq.html Are Lucy and Lucene compatible? No. Lucy is a "loose" port of Lucene designed to take full advantage of C's unique feature set, rather than a line-by-line translation from Java. The two libraries are not compatible in terms of either file format or API, and there are no plans to establish such compatibility. Does Lucy provide a search server like Solr? Lucy is a low-level library, like Lucene. We'd like to provide a search server eventually, but it will likely be a thin wrapper rather than a comprehensive application like Solr. The low-level capabilities are our core mission. Why don't you use Swig? A major design goal of Lucy is to present bindings which are as idiomatic as possible so that our users feel as though they are programming in their native language and not in C. Swig is a great tool, but it does not offer support for many of the features which make Lucy so user friendly: subclassing, named parameters, default argument values, etc. From http://wiki.apache.org/lucy/MinimizingObjectOverhead http://wiki.apache.org/lucy/MinimizingObjectOverhead Lucy must be designed so that native OO overhead is minimized. Algorithms which create and destroy as few native objects as possible are preferred. Native method calls should be avoided for high-volume, performance critical applications. Possible alternatives: Change the algo to use a for/while loop (either native or C). Use a C struct with a function pointer. Use a C function.
quadhome 15y ago
In the same class: xapian[1], Sphinx[2] and whistlepig?[3] ;-) [1] http://xapian.org/ http://xapian.org/ [2] http://sphinxsearch.com/ http://sphinxsearch.com/ [3] https://github.com/wmorgan/whistlepig https://github.com/wmorgan/whistlepig
dguaraglia 15y ago
Has anyone here played with Lucy? I'm looking to move some code from ElasticSearch to something, and still haven't decided on what. Sphinx looks promising, but their model for adding documents isn't my favorite. It makes integration with ElasticSearch seem trivial. Something like Lucy might be a really good middle ground.

12 points by ot 15y ago

exDM69 15y ago

The question is, what was the motivation behind re-writing Java Lucene in C?

Avshalom 15y ago
It says on the page: so it can be used by dynamic languages.
- exDM69 15y ago
  That is a very good reason. This is also why I think that most system level libraries should be written in C, because it (unfortunately) is one of the only languages that can be called from pretty much any other language out there. If you go with JVM or CLR, your software will only run on that virtual machine. C++ may be possible to some degree but C++'s pass-by-value semantics make it a bit tricky to export to other languages (with reference based semantics), so if you have a good C++ API, it won't turn into a nice Python API. Exception handling and other issues make C++ also tricky.

hopeless 15y ago

No more Java dependencies in your Ruby webapp? Sounds promising!

ognyankulev 15y ago

Interesting quotes: From http://incubator.apache.org/lucy/faq.html http://incubator.apache.org/lucy/faq.html Are Lucy and Lucene compatible? No. Lucy is a "loose" port of Lucene designed to take full advantage of C's unique feature set, rather than a line-by-line translation from Java. The two libraries are not compatible in terms of either file format or API, and there are no plans to establish such compatibility. Does Lucy provide a search server like Solr? Lucy is a low-level library, like Lucene. We'd like to provide a search server eventually, but it will likely be a thin wrapper rather than a comprehensive application like Solr. The low-level capabilities are our core mission. Why don't you use Swig? A major design goal of Lucy is to present bindings which are as idiomatic as possible so that our users feel as though they are programming in their native language and not in C. Swig is a great tool, but it does not offer support for many of the features which make Lucy so user friendly: subclassing, named parameters, default argument values, etc. From http://wiki.apache.org/lucy/MinimizingObjectOverhead http://wiki.apache.org/lucy/MinimizingObjectOverhead Lucy must be designed so that native OO overhead is minimized. Algorithms which create and destroy as few native objects as possible are preferred. Native method calls should be avoided for high-volume, performance critical applications. Possible alternatives: Change the algo to use a for/while loop (either native or C). Use a C struct with a function pointer. Use a C function.

quadhome 15y ago

In the same class: xapian[1], Sphinx[2] and whistlepig?[3] ;-) [1] http://xapian.org/ http://xapian.org/ [2] http://sphinxsearch.com/ http://sphinxsearch.com/ [3] https://github.com/wmorgan/whistlepig https://github.com/wmorgan/whistlepig

dguaraglia 15y ago

Has anyone here played with Lucy? I'm looking to move some code from ElasticSearch to something, and still haven't decided on what. Sphinx looks promising, but their model for adding documents isn't my favorite. It makes integration with ElasticSearch seem trivial. Something like Lucy might be a really good middle ground.