3 ms·
CEO of Neon here. After we built an in memory HNSW index for Postgres that allowed us to establish a baseline in performance and prove that it's the right appro
by nikita 3y ago
CEO of Neon here. After we built an in memory HNSW index for Postgres that allowed us to establish a baseline in performance and prove that it's the right approach to support vector search we now built it "the right way" and now it support restarts of Postgres, replication and the rest of the Postgres machinery.
- myth17 3y agoCan you explain: What does it mean to be constructed on disk?
- nikita 3y agoThis means that the index is persisted in Postgres pages which are durable against restarting Postgres. Our previous version what fully in-memory and wasn't "durable".
- nh2 3y agoCan you clarify: Does the the initial construction (creation) of the index need to fit into RAM? Also, should one expect the reading via Postgres's buffer cache to have better access patterns / need less memory / have other benefit vs having a non-Postgres HNSW index that's mmapped from disk?
- raoufchebri 3y agoThe index doesn't need to fit in the shared buffers to be built, but you can speed the queries if you load it to memory