5 ms·
BM25 – The Next Generation of Lucene Relevance
- deleted 11y ago[deleted]
- asjo 11y agoXapian has been using BM25 forever[1] - what made Lucene switch now? [1] https://xapian.org/docs/bm25.html https://xapian.org/docs/bm25.html
- apatry 11y agoBM25 has been available for Lucene since version 4 thanks to a refactoring that allowed anyone to plug in a custom relevance score if I remember correctly . The big news is that starting version 6, BM25 is the default.
- asjo 11y agoYes, so I am asking why is Lucene switching now?
- deleted 11y ago[deleted]
- nl 11y agoHere's the issue tracking[1]. Basically it looks like the typical conservative approach most large scale open source projects are forced to take because of cross dependencies. It's a (potentially) breaking change, so they waited for a major release. [1] https://issues.apache.org/jira/browse/LUCENE-6789 https://issues.apache.org/jira/browse/LUCENE-6789
- rpedela 11y agoBM25 has been available for awhile so is there a reason it is becoming default soon besides it is better in certain scenarios? Have there been improvements to the implementation recently?
- eva1984 11y agoI went to the elastic conf last month, there is a great talk about BM25. It mentions that BM25 is more robust against common words than plain tf*idf. https://www.elastic.co/elasticon/conf/2016/sf/improved-text-scoring-with-bm25 https://www.elastic.co/elasticon/conf/2016/sf/improved-text-...
- CephalopodMD 11y agoDoesn't lucene let you choose your own relevance algorithms, smoothing, metrics, etc. anyway?