4 ms·

The announcement says: Lucy is a "loose C" port of Apache Lucene, a search engine library for Java -- it is similar in purpose to Lucene, but designed to take

by chuhnk 15y ago

The announcement says: Lucy is a "loose C" port of Apache Lucene, a search engine library for Java -- it is similar in purpose to Lucene, but designed to take advantage of C's unique capabilities. I'm wondering what these unique capabilities are. Speed? Smaller memory footprint? And I wonder what the reason is behind doing this. I'm all for C project but very curious as to why when Lucene was very well done.

hotdox 15y ago
>I'm wondering what these unique capabilities are. easy bindings for every dynamic language. they start with perl
nkurz 15y ago
I'm one of the developers, although not currently very active. The main "unique capability" is closer integration to the machine. Our approach has been "The OS is our VM". We use mmap() heavily, and when running on 64-bit systems take liberal advantage of the giant address space. Using the system to do more of the buffering also allows us to have lightweight processes that can start quickly. We think there will be both speed and memory advantages in the long run. The other main difference is that C is much easier to integrate with other languages than Java. We're starting out with Perl bindings, but have plans for Ruby, Python, Lua, Tcl, and others. The goal is to offer a truly native interface from the language of your choice. The degree of host language integration is wild. You'll be able to seamlessly subclass just about any part of the C library in any supported language. Nothing is ready beyond C and Perl, but eventually you'll be able to have your indexer in one language, and your customized searchers in a couple more, while all sharing the same shared system cache. As for why? Marvin, the main developer started the project as KinoSearch at a time when Lucene wasn't really ready for prime time. He's been very interested in real time indexing, and at the time Lucene didn't handle this well. I got interested because I was looking for something lighter weight than Lucene, where I could try to blend the boundaries between search and database retrieval. Lucene had too many layers of abstraction for my purposes. A parallel might be SQLite and Postgres. Both have their place, but Lucy is more on the SQLite side of things.

by chuhnk 15y ago

hotdox 15y ago

>I'm wondering what these unique capabilities are. easy bindings for every dynamic language. they start with perl

nkurz 15y ago

I'm one of the developers, although not currently very active. The main "unique capability" is closer integration to the machine. Our approach has been "The OS is our VM". We use mmap() heavily, and when running on 64-bit systems take liberal advantage of the giant address space. Using the system to do more of the buffering also allows us to have lightweight processes that can start quickly. We think there will be both speed and memory advantages in the long run. The other main difference is that C is much easier to integrate with other languages than Java. We're starting out with Perl bindings, but have plans for Ruby, Python, Lua, Tcl, and others. The goal is to offer a truly native interface from the language of your choice. The degree of host language integration is wild. You'll be able to seamlessly subclass just about any part of the C library in any supported language. Nothing is ready beyond C and Perl, but eventually you'll be able to have your indexer in one language, and your customized searchers in a couple more, while all sharing the same shared system cache. As for why? Marvin, the main developer started the project as KinoSearch at a time when Lucene wasn't really ready for prime time. He's been very interested in real time indexing, and at the time Lucene didn't handle this well. I got interested because I was looking for something lighter weight than Lucene, where I could try to blend the boundaries between search and database retrieval. Lucene had too many layers of abstraction for my purposes. A parallel might be SQLite and Postgres. Both have their place, but Lucy is more on the SQLite side of things.