4 ms·
LMDB is a key-value database. The Euclides DB build on top of a existing key-value database, LevelDB (could be LMDB) as the persistent layer. Im not familiar w
by oscargrouch 8y ago
LMDB is a key-value database. The Euclides DB build on top of a existing key-value database, LevelDB (could be LMDB) as the persistent layer.
Im not familiar with EuclidesDB, but from what i've understand from the documented architecture it uses gRPC to expose a RPC interface over the internet, and abstract away the objects in the domain of machine learning (like images) tied to PyTorch.
Anyway, its a higher level concept and architecture modeled on top of a key-value storage like LMDB, which is a lower level building block for this product. LMDB being comparable to LevelDB, not EuclidesDB.
(It looks that the author is here, so please correct me if im mistaken in any way)
- perone 8y agoHi, you're correct, LevelDB is the lower level building block for EuclidesDB, it's the underlying storage for item's features/predictions/metadata. EuclidesDB uses gRPC (reasons for that design decision are described in the docs) as protocol communication and protobuf as serialization mechanism for its RPC communication with client APIs (i.e. Python client API). EuclidesDB is also tightly coupled with libtorch (PyTorch C++ backend) and it is EuclidesDB that is responsible to run inference (forward pass) on the models instead of the client, so it takes all the heavy burden from clients and adds it into the database engine itself. EuclidesDB also has querying capabilities (using LSH for performance) to query what was added into it, so again, the query (and feature extraction for it) is executed by the EuclidesDB. So the comparison between LevelDB/LMDB and EuclidesDB doesn't make much sense, they are low-level embeddable engines for key-value storage (that EuclidesDB uses).
- tixocloud 8y agoHi, can you share what your thoughts were when wanting to build something like this?