2 ms·

Can you tell about how you write/read your data to/from disk? I‘m also exploring the incremental world using an approach like salsa+rocksdb. Similar like https:

by hacki11 1mo ago

Can you tell about how you write/read your data to/from disk? I‘m also exploring the incremental world using an approach like salsa+rocksdb. Similar like https://docs.rs/qbice/latest/qbice/ https://docs.rs/qbice/latest/qbice/. Would be great to learn about your approach.

popzxc 1mo ago
Well, it's hard to give a concise overview, but in short -- I use `wincode` for serde (without zero-copy deserialization though; I've tried zero-copy first with rkyv but it was not trivial at all so I abandoned this idea, plus FS does not dominate the costs so far anyway). Data is written at the end of indexing phase, during write we hold a lock, and the codebase is aware that offloaded data can be corrupted/outdated (mostly relevant for cache). Query processing works on top of transaction, and the transaction holds data loaded from files for the duration of the query, and frees once the query is dropped. There is a ton of tricky parts there, but I guess that's the gist of it. If you're interested, here are the relevant parts of the source code: https://github.com/rust-glancer/rust-glancer/tree/main/crates/engine/package-store https://github.com/rust-glancer/rust-glancer/tree/main/crate... https://github.com/rust-glancer/rust-glancer/tree/main/crates/engine/project/src/cache https://github.com/rust-glancer/rust-glancer/tree/main/crate... https://github.com/rust-glancer/rust-glancer/blob/main/crates/engine/project/src/project/loading.rs https://github.com/rust-glancer/rust-glancer/blob/main/crate... https://github.com/rust-glancer/rust-glancer/blob/main/crates/engine/project/src/project/txn.rs https://github.com/rust-glancer/rust-glancer/blob/main/crate...
popzxc 1mo ago
And regarding salsa+rocksdb, actually Alex Kladov advocated exactly for that in the post about Rust Glancer and rust-analyzer architecture: https://matklad.github.io/2026/08/21/rust-glancer.html https://matklad.github.io/2026/08/21/rust-glancer.html Though I believe it's more of a direction for rust-analyzer rather Rust Glancer, at least for now.

by hacki11 1mo ago

popzxc 1mo ago

Well, it's hard to give a concise overview, but in short -- I use `wincode` for serde (without zero-copy deserialization though; I've tried zero-copy first with rkyv but it was not trivial at all so I abandoned this idea, plus FS does not dominate the costs so far anyway). Data is written at the end of indexing phase, during write we hold a lock, and the codebase is aware that offloaded data can be corrupted/outdated (mostly relevant for cache). Query processing works on top of transaction, and the transaction holds data loaded from files for the duration of the query, and frees once the query is dropped. There is a ton of tricky parts there, but I guess that's the gist of it. If you're interested, here are the relevant parts of the source code: https://github.com/rust-glancer/rust-glancer/tree/main/crates/engine/package-store https://github.com/rust-glancer/rust-glancer/tree/main/crate... https://github.com/rust-glancer/rust-glancer/tree/main/crates/engine/project/src/cache https://github.com/rust-glancer/rust-glancer/tree/main/crate... https://github.com/rust-glancer/rust-glancer/blob/main/crates/engine/project/src/project/loading.rs https://github.com/rust-glancer/rust-glancer/blob/main/crate... https://github.com/rust-glancer/rust-glancer/blob/main/crates/engine/project/src/project/txn.rs https://github.com/rust-glancer/rust-glancer/blob/main/crate...

popzxc 1mo ago

And regarding salsa+rocksdb, actually Alex Kladov advocated exactly for that in the post about Rust Glancer and rust-analyzer architecture: https://matklad.github.io/2026/08/21/rust-glancer.html https://matklad.github.io/2026/08/21/rust-glancer.html Though I believe it's more of a direction for rust-analyzer rather Rust Glancer, at least for now.