4 ms·

Then you don't need WAL. I didn't say replication isn't possible, I said "that's not how it works." A bunch of different people solved this already (though non

Then you don't need WAL. I didn't say replication isn't possible, I said "that's not how it works." A bunch of different people solved this already (though none in a couple of weekends); common to all of them: they don't use WAL mode.

jauntywundrkind 2y ago
> The shadow WAL is a directory next to your SQLite database where WAL files are effectively recreated as a sequence. . . These WAL files contain the original WAL frames & checksums to ensure consistency. https://litestream.io/how-it-works/ https://litestream.io/how-it-works/ > (though none in a couple of weekends) As I said, getting sqlite in wasm to run in WAL mode such that we could start to implement replication, like for example how litestream does. Not this strawman you build, of implementing replication.
- ncruces 2y ago
  Seriously? Litestream works out-of-process. It needs shared memory WAL and file locking to work exactly like on "desktop" SQLite for it to even function and produce this "shadow WAL." No ifs, no buts, no inbetweens. You just don't build Litestream without shared memory WAL. That's my entire point. These pieces you seem to think can be made to simply work together can't, in fact, work together at all, much less simply. Exclusive locking mode WAL doesn't work with Litestream, and can't be made to work with Litestream, even with a little elbow grease.
  - jauntywundrkind 2y ago
    Litestreams how-we-did-this to says they take a lock on the whole db during checkpoints to work, so I don't see why shared would be needed. Seriously bro, take a massive chill pill. You have been way overcommitted to the "it's all impossible!!! How stupid to imagine! If only you were smart like me you'd see everything is impossible!" bit way too hard. Take a breather from your possessed absolute self certainty & derogatory belittling of others. There's also lots of different fs impls for wasm! Just because some limitations apply to emscriptens impl doesn't mean another impl has the same limitations, or that we can't improve or work around. Use some of your hard working loves-to-tax-itself big brain to find avenues of possibility, rather than just pissing on every possibility.
    - ncruces 2y ago
      I'm chill bro. Still waiting for your productive couple of weekends. But that'll require more than looking at docs reading the words "lock" and "shared" and assuming you know what those mean.
      - jauntywundrkind 2y ago
        You've been wrong about how some of these replications work, you've been wrong about why the WAL doesn't work, and you still insist on being a pompous rude brat here? You have not been chill.
        ncruces 2y ago
        Litestream is out of process, requires shared memory WAL, doesn't work with exclusive mode WAL, wouldn't work work on Wasm. LiteFS didn't support WAL mode initially, now does. The Wasm version doesn't support WAL mode at all. mvSQLite doesn't support shared memory WAL. SQLSync doesn't support WAL at all. Browser Wasm builds either don't have WAL enabled, or don't support shared memory WAL. These replications are VFSes or FUSE drivers, and for those, it's much easier to work with rollback mode than WAL, especially if you don't want concurrency (or intend to provide concurrency through other means, like optimistic concurrency). In open source, only Turso actually did a VFS to replicate the WAL. They had to break SQLite so much they call it libSQL. It also requires shared memory as is, would be hard (if possible) to port to Wasm. Please, do take the challenge and build this. It will be much appreciated. I've been asking the Turso guys to document their thing for a year, with little success. They have the interface (which just opens up undocumented internal SQLite interfaces), with no open source samples of how to use it (except a logger), and zero additional docs.
        CRConrad 2y ago
        TBF, to this bystander ncruces isn't the one of the two of you who is coming off as more of "a pompous rude brat".

jauntywundrkind 2y ago

> The shadow WAL is a directory next to your SQLite database where WAL files are effectively recreated as a sequence. . . These WAL files contain the original WAL frames & checksums to ensure consistency. https://litestream.io/how-it-works/ https://litestream.io/how-it-works/ > (though none in a couple of weekends) As I said, getting sqlite in wasm to run in WAL mode such that we could start to implement replication, like for example how litestream does. Not this strawman you build, of implementing replication.

ncruces 2y ago
Seriously? Litestream works out-of-process. It needs shared memory WAL and file locking to work exactly like on "desktop" SQLite for it to even function and produce this "shadow WAL." No ifs, no buts, no inbetweens. You just don't build Litestream without shared memory WAL. That's my entire point. These pieces you seem to think can be made to simply work together can't, in fact, work together at all, much less simply. Exclusive locking mode WAL doesn't work with Litestream, and can't be made to work with Litestream, even with a little elbow grease.
- jauntywundrkind 2y ago
  Litestreams how-we-did-this to says they take a lock on the whole db during checkpoints to work, so I don't see why shared would be needed. Seriously bro, take a massive chill pill. You have been way overcommitted to the "it's all impossible!!! How stupid to imagine! If only you were smart like me you'd see everything is impossible!" bit way too hard. Take a breather from your possessed absolute self certainty & derogatory belittling of others. There's also lots of different fs impls for wasm! Just because some limitations apply to emscriptens impl doesn't mean another impl has the same limitations, or that we can't improve or work around. Use some of your hard working loves-to-tax-itself big brain to find avenues of possibility, rather than just pissing on every possibility.
  - ncruces 2y ago
    I'm chill bro. Still waiting for your productive couple of weekends. But that'll require more than looking at docs reading the words "lock" and "shared" and assuming you know what those mean.
    - jauntywundrkind 2y ago
      You've been wrong about how some of these replications work, you've been wrong about why the WAL doesn't work, and you still insist on being a pompous rude brat here? You have not been chill.
      - ncruces 2y ago
        Litestream is out of process, requires shared memory WAL, doesn't work with exclusive mode WAL, wouldn't work work on Wasm. LiteFS didn't support WAL mode initially, now does. The Wasm version doesn't support WAL mode at all. mvSQLite doesn't support shared memory WAL. SQLSync doesn't support WAL at all. Browser Wasm builds either don't have WAL enabled, or don't support shared memory WAL. These replications are VFSes or FUSE drivers, and for those, it's much easier to work with rollback mode than WAL, especially if you don't want concurrency (or intend to provide concurrency through other means, like optimistic concurrency). In open source, only Turso actually did a VFS to replicate the WAL. They had to break SQLite so much they call it libSQL. It also requires shared memory as is, would be hard (if possible) to port to Wasm. Please, do take the challenge and build this. It will be much appreciated. I've been asking the Turso guys to document their thing for a year, with little success. They have the interface (which just opens up undocumented internal SQLite interfaces), with no open source samples of how to use it (except a logger), and zero additional docs.
      - CRConrad 2y ago
        TBF, to this bystander ncruces isn't the one of the two of you who is coming off as more of "a pompous rude brat".