3 ms·
Am I reading this right? You’ve written a vector database in python? Are there any performance benchmark comparisons for me to look at?
by drawnwren 3y ago
Am I reading this right? You’ve written a vector database in python? Are there any performance benchmark comparisons for me to look at?
- dmezzetti 3y agotxtai is in Python but much of the underlying code isn't. For example, PyTorch/Transformers uses C/C++/CUDA for most operations. The approximate nearest neighbor (ANN) libraries for vector indexes are Faiss, Hnswlib and Annoy. They are all C/C++. Much of the sparse vectors code is NumPy and uses Python's array module for storing term vectors. There are no 3rd party benchmarks for txtai as of now. You would have to compare how it does on your own data to judge.