3 ms·
Now I can't help but wonder if there should be a `waln` mode where the WAL files would round robin instead of alternate between just two potentially allowing fo
by tomashubelbauer 3y ago
Now I can't help but wonder if there should be a `waln` mode where the WAL files would round robin instead of alternate between just two potentially allowing for much more intense write cadence.
- rmbyrro 3y agoThat would make reading slower, since readers need to read from WAL as well.
- nbevans 3y agoThere is a .shm (shared memory) file that all SQLite connections to the database would use to determine which page ranges are in the WAL versus main DB file. So that overhead already exists when WAL is enabled.
- rmbyrro 3y agoI'm not familiar with its inner workings, but if there were n WAL files, I'd expect the overhead to be a function of n.
- dspillett 3y agoPossibly a bad function of n (linear or worse) on IO systems based on traditional drives or dirt cheap SSDs, once IO becomes your main bottleneck (i.e. when you data no longer fits easily in RAM).
- avereveard 3y agoIl give this thread two or three more replies before it reimplement logrotate from first principles
- dspillett 3y agoPresumably it was at least considered. I would guess that the negatives would be taking more disk space (and potentially imposing greater IO concurrency though that probably isn't a great issue with modern storage systems), and it being more complex (if not in the core design, then in having extra edge cases to make sure are all covered). Other databases do do similar to what you suggest, though obviously the trade-offs will differ because of other different internals and product priorities, so it would have been thought about. For instance MS SQL Server has multiple “virtual logs” in its log files, for at least some overlapping reasons.