3 ms·

> and the associated sqlite3_serialize and sqlite3_deserialize APIs So... I have to admit I missed the existence of these. So for many cases, using sqlite3_des

> and the associated sqlite3_serialize and sqlite3_deserialize APIs So... I have to admit I missed the existence of these. So for many cases, using sqlite3_deserialize is probably a very reasonable choice over sqlite-memory-vfs Still, as you say, sqlite-memory-vfs doesn't require the memory to be contiguous, unlike sqlite3_deserialize > You don't support concurrency at all (xLock/xUnlock/xCheckReservedLock are noops). Well, you can read concurrently, just not write

ncruces 3y ago
If you're interested, both SQLite's and my memdb VFSes implement safe locking. Depending on your familiarity with Go, mine maybe easier to follow, or not. https://github.com/ncruces/go-sqlite3/blob/f1b00a9944730eaa9da28a7c967658ed2426d48d/vfs/memdb/memdb.go#L181-L257 https://github.com/ncruces/go-sqlite3/blob/f1b00a9944730eaa9...
- michalc 3y ago
  Ah thanks for this (maybe not so much for the code itself admittedly, but for the push to implement locking) So after a few attempts, I _think_ I have something reasonable: https://github.com/michalc/sqlite-memory-vfs/blob/64a55c349173794f37230f1c101514e8964da669/sqlite_memory_vfs.py#L111-L140 https://github.com/michalc/sqlite-memory-vfs/blob/64a55c3491... (I'm still a but stuck on PENDING I have to admit... not really sure how that fits in)
  - ncruces 3y ago
    Pending exists to prevent writer starvation: if there's always at least one reader, no one can ever write. You take pending before exclusive (which should always be possible because you have reserved by then), and you should check pending before taking a shared lock (no need to check exclusive, because they got pending before exclusive). Shared forces writers to wait; Reserved prevents new readers from upgrading to write; Pending prevents new readers from starting; and exclusive forces everyone else to wait.
    - michalc 3y ago
      Have now implemented a version of this at https://github.com/michalc/sqlite-memory-vfs/blob/74a24ce567557d72eebf73fbdf6d8e609aedafe3/sqlite_memory_vfs.py#L112-L155 https://github.com/michalc/sqlite-memory-vfs/blob/74a24ce567... I think different to yours in two ways: - If doesn't go via PENDING if it doesn't need to. Not sure what's best really - but since RESERVED can already be skipped in some cases (recovering from a hot-journal) then so far happy with "skipping is a thing", and PENDING is private to the VFS anyway from what I can tell, so nothing else should care I think - It doesn't do the blocking in the lock function, but depends on SQLite's core code doing it via the busy handler/timeout

ncruces 3y ago

If you're interested, both SQLite's and my memdb VFSes implement safe locking. Depending on your familiarity with Go, mine maybe easier to follow, or not. https://github.com/ncruces/go-sqlite3/blob/f1b00a9944730eaa9da28a7c967658ed2426d48d/vfs/memdb/memdb.go#L181-L257 https://github.com/ncruces/go-sqlite3/blob/f1b00a9944730eaa9...

michalc 3y ago
Ah thanks for this (maybe not so much for the code itself admittedly, but for the push to implement locking) So after a few attempts, I _think_ I have something reasonable: https://github.com/michalc/sqlite-memory-vfs/blob/64a55c349173794f37230f1c101514e8964da669/sqlite_memory_vfs.py#L111-L140 https://github.com/michalc/sqlite-memory-vfs/blob/64a55c3491... (I'm still a but stuck on PENDING I have to admit... not really sure how that fits in)
- ncruces 3y ago
  Pending exists to prevent writer starvation: if there's always at least one reader, no one can ever write. You take pending before exclusive (which should always be possible because you have reserved by then), and you should check pending before taking a shared lock (no need to check exclusive, because they got pending before exclusive). Shared forces writers to wait; Reserved prevents new readers from upgrading to write; Pending prevents new readers from starting; and exclusive forces everyone else to wait.
  - michalc 3y ago
    Have now implemented a version of this at https://github.com/michalc/sqlite-memory-vfs/blob/74a24ce567557d72eebf73fbdf6d8e609aedafe3/sqlite_memory_vfs.py#L112-L155 https://github.com/michalc/sqlite-memory-vfs/blob/74a24ce567... I think different to yours in two ways: - If doesn't go via PENDING if it doesn't need to. Not sure what's best really - but since RESERVED can already be skipped in some cases (recovering from a hot-journal) then so far happy with "skipping is a thing", and PENDING is private to the VFS anyway from what I can tell, so nothing else should care I think - It doesn't do the blocking in the lock function, but depends on SQLite's core code doing it via the busy handler/timeout