4 ms·

Xapian has a long history starting in the early 80s: https://xapian.org/history https://xapian.org/history I've used Xapian extensively, but not this new Xapi

Xapian has a long history starting in the early 80s: https://xapian.org/history https://xapian.org/history I've used Xapian extensively, but not this new Xapiand tool, so I can only speak to the actual library. Xapian is a C++ library that accesses index data files directly on disk. There are bindings for various languages, say Python, let's you do 'import xapian' and get FFI bindings to the library, then you basically open your on disk index files and issue queries. Xapian supports many concurrent readers, but only one writer. It's not a server, there are no protocols. Maybe that's what this Xapiand tool adds. In general the overhead is very, very light, just enough ram to hold the library code, the OS takes care of all the filesystem level caching. Many of the very same concepts that are in Lucene, Documents, Terms, weights, flavors of BM25 relevance ranking, query parsing trees, relevancy operators, etc, all apply to Xapian as well.

_wmd 8y ago
I love Xapian, the quality of its recall is excellent and indexing performance very hard to find fault with. There's just a tiny problem - it's stuck with the GPL, despite a long effort to relicence the code going back years.
- Kronuz 8y ago
  Maybe xapian library just needs a little push from a larger community to make relicensing faster, Xapiand could help towards that end by helping brining in more people which can help. Xapiand source code is itself licensed as MIT (before compiling), and xapian community is already taking big steps towards relicensing.
- vkazanov 8y ago
  Remind me what's wrong with GPL, here? You can't repackage and resell it?
  - ddorian43 8y ago
    You can't include it in your code and sell your software without distributing the source.
    - vkazanov 8y ago
      Yeah, a pity
    - pedrocr 8y ago
      Is there actually precedent for a GPL library, clearly setup to be consumed with an API, to require GPL of the whole binary? On the one hand that's clearly not the case in the Linux API, on the other hand the LGPL exists. But was curious if this is actually settled or just too murky to live with.
      - ddorian43 8y ago
        There are many gpl/lgpl dbs with bsd drivers (ex: scylladb). Dont know about library though (you should be able to use the library externally I think just like you do with the db)
      - detaro 8y ago
        The Linux kernel has explicit rules for the cases where it allows it: An explicit license exception for the syscall interface, stating that calling it from userspace is freely allowed: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/LICENSES/exceptions/Linux-syscall-note?h=v5.0-rc2 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... And modules, e.g. drivers, have various statements from Linus on how they're not necessarily considered derived works of the kernel/linked to the kernel, e.g. http://linuxmafia.com/faq/Kernel/proprietary-kernel-modules.html http://linuxmafia.com/faq/Kernel/proprietary-kernel-modules.... or http://lkml.iu.edu/hypermail/linux/kernel/0312.0/0670.html http://lkml.iu.edu/hypermail/linux/kernel/0312.0/0670.html
        pedrocr 8y ago
        I know the situation of the Linux kernel. The discussion hinges on what is a "derived work". Kernel modules are very interlinked with the kernel, particularly since there are no stable APIs inside the kernel. It seems strange that just linking to a GPL library that defined an API makes you a derived work. But that is the FSF position: https://en.wikipedia.org/wiki/GNU_Readline#Choice_of_the_GPL_as_GNU_Readline's_license https://en.wikipedia.org/wiki/GNU_Readline#Choice_of_the_GPL... This is why I always do libs as LGPL but it seems strange to me that it's even needed. If I've defined a proper opaque API, to be consumed by external code I know nothing about, it's strange to then argue that library callers are derived works and LGPL is explicitly needed.
- deleted 8y ago
  [deleted]

_wmd 8y ago

I love Xapian, the quality of its recall is excellent and indexing performance very hard to find fault with. There's just a tiny problem - it's stuck with the GPL, despite a long effort to relicence the code going back years.

Kronuz 8y ago
Maybe xapian library just needs a little push from a larger community to make relicensing faster, Xapiand could help towards that end by helping brining in more people which can help. Xapiand source code is itself licensed as MIT (before compiling), and xapian community is already taking big steps towards relicensing.
vkazanov 8y ago
Remind me what's wrong with GPL, here? You can't repackage and resell it?
- ddorian43 8y ago
  You can't include it in your code and sell your software without distributing the source.
  - vkazanov 8y ago
    Yeah, a pity
  - pedrocr 8y ago
    Is there actually precedent for a GPL library, clearly setup to be consumed with an API, to require GPL of the whole binary? On the one hand that's clearly not the case in the Linux API, on the other hand the LGPL exists. But was curious if this is actually settled or just too murky to live with.
    - ddorian43 8y ago
      There are many gpl/lgpl dbs with bsd drivers (ex: scylladb). Dont know about library though (you should be able to use the library externally I think just like you do with the db)
    - detaro 8y ago
      The Linux kernel has explicit rules for the cases where it allows it: An explicit license exception for the syscall interface, stating that calling it from userspace is freely allowed: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/LICENSES/exceptions/Linux-syscall-note?h=v5.0-rc2 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... And modules, e.g. drivers, have various statements from Linus on how they're not necessarily considered derived works of the kernel/linked to the kernel, e.g. http://linuxmafia.com/faq/Kernel/proprietary-kernel-modules.html http://linuxmafia.com/faq/Kernel/proprietary-kernel-modules.... or http://lkml.iu.edu/hypermail/linux/kernel/0312.0/0670.html http://lkml.iu.edu/hypermail/linux/kernel/0312.0/0670.html
      - pedrocr 8y ago
        I know the situation of the Linux kernel. The discussion hinges on what is a "derived work". Kernel modules are very interlinked with the kernel, particularly since there are no stable APIs inside the kernel. It seems strange that just linking to a GPL library that defined an API makes you a derived work. But that is the FSF position: https://en.wikipedia.org/wiki/GNU_Readline#Choice_of_the_GPL_as_GNU_Readline's_license https://en.wikipedia.org/wiki/GNU_Readline#Choice_of_the_GPL... This is why I always do libs as LGPL but it seems strange to me that it's even needed. If I've defined a proper opaque API, to be consumed by external code I know nothing about, it's strange to then argue that library callers are derived works and LGPL is explicitly needed.
deleted 8y ago
[deleted]