3 ms·

I actually don't need strong consistency here - I'll only have a single instance of the container accessing the sqlite file at a single point, but they can be r

by 3np 6y ago

I actually don't need strong consistency here - I'll only have a single instance of the container accessing the sqlite file at a single point, but they can be running on any host. As long as I can ensure replication is synchronized before process start when a container gets rescheduled we good. It goes over my head a bit exactly why (related to POSIX lock mechanisms?) but sqlite doesn't work reliable on NFS/ceph/gluster/networked filesystems, even when only accessed from a single host, distributed or not. https://sqlite.org/forum/forumpost/4b340b81eb https://sqlite.org/forum/forumpost/4b340b81eb

slaymaker1907 6y ago
You could try implementing a small shim over the FS using Posix or Dokan which just goes to the other FS but make sure writes are synced correctly. It would probably be quite slow, but is sounds like you mostly care about having something that works.
jlokier 6y ago
> It goes over my head a bit exactly why (related to POSIX lock mechanisms?) but sqlite doesn't work reliable on NFS/ceph/gluster/networked filesystems, even when only accessed from a single host, distributed or not. > https://sqlite.org/forum/forumpost/4b340b81eb https://sqlite.org/forum/forumpost/4b340b81eb I don't see anything in that forum thread which says SQLite is unreliable on sufficiently POSIX-ish network filesystems. Single host or not. The linked post says it will be slower to commit write transactions than is possible using other methods, but I don't see anything there saying it's unreliable. (What looks unreliable to me is Joelmo's proposal to remove the WAL lock for better performance. But the lock is there for a reliability reason, you can't just remove it. To get the boost in performance Joelmo would like requires significantly different techniques.)

3 ms·

I actually don't need strong consistency here - I'll only have a single instance of the container accessing the sqlite file at a single point, but they can be r

by 3np 6y ago

slaymaker1907 6y ago
You could try implementing a small shim over the FS using Posix or Dokan which just goes to the other FS but make sure writes are synced correctly. It would probably be quite slow, but is sounds like you mostly care about having something that works.
jlokier 6y ago
> It goes over my head a bit exactly why (related to POSIX lock mechanisms?) but sqlite doesn't work reliable on NFS/ceph/gluster/networked filesystems, even when only accessed from a single host, distributed or not. > https://sqlite.org/forum/forumpost/4b340b81eb https://sqlite.org/forum/forumpost/4b340b81eb I don't see anything in that forum thread which says SQLite is unreliable on sufficiently POSIX-ish network filesystems. Single host or not. The linked post says it will be slower to commit write transactions than is possible using other methods, but I don't see anything there saying it's unreliable. (What looks unreliable to me is Joelmo's proposal to remove the WAL lock for better performance. But the lock is there for a reliability reason, you can't just remove it. To get the boost in performance Joelmo would like requires significantly different techniques.)