4 ms·

Perhaps, but the point about divergence stands! Elastic won’t be copying the OpenDistro KNN API.

• _msw_ 5y ago
  It is pretty clear to me that Elastic is planning to build their ANN features differently than OpenDistro's k-NN implementation, or other plugins modules that extend Easticsearch in similar ways. They now will build on the Apache Lucene capabilities that were collaboratively built "upstream" by a number of individuals, some that work for Amazon and some that work for Elastic. From the linked issue, it seemed that they were originally planning to develop this as a proprietary feature of Elasticsearch, without contributing the functionality to Apache Lucene, but then changed direction when the Apache Lucene developers (some of which are currently employed to do such work by Amazon) started to build its approximate nearest neighbor (ANN) vector search capabilities. [1] It's great to see folks that work for Elastic collaborating and building on what is in Apache Lucene to extend the utility of ANN with Hierarchical Navigable Small World Graphs (HNSW) [2]! From this, I think it should be possible to implement an Open Source version of the functionality with a compatible API, if that is something that OpenSearch users seek. [1] https://issues.apache.org/jira/browse/LUCENE-9004 https://issues.apache.org/jira/browse/LUCENE-9004 [2] https://github.com/apache/lucene/pull/250 https://github.com/apache/lucene/pull/250
  • binarymax 5y ago
    "it should be possible to implement an Open Source version of the functionality". Elasticsearch is still open source...just not in a license that Amazon can take advantage of.