3 ms·

Seems there is a bit of vendor lock-in here with flyio, the way he's throwing custom http error codes for their infra. Can I just run LiteFS on a few virtual m

Seems there is a bit of vendor lock-in here with flyio, the way he's throwing custom http error codes for their infra. Can I just run LiteFS on a few virtual machines and forward writes back to master? With LiteFS, what happens if master goes down? Can a new one be elected?

mildbyte 4y ago
You can indeed run LiteFS by yourself, without Consul, as a sidecar / wrapper around your application. We do it in our project and have a Docker Compose example at [0]. In this case, you specify a specific known leader node. We haven't tried getting it running independently with Consul to do leader election / failover. [0] https://github.com/splitgraph/seafowl/blob/main/examples/litefs/configs/seafowl-writer/litefs.yml#L4 https://github.com/splitgraph/seafowl/blob/main/examples/lit...
benbjohnson 4y ago
> Seems there is a bit of vendor lock-in here with flyio, the way he's throwing custom http error codes for their infra. Can I just run LiteFS on a few virtual machines and forward writes back to master? LiteFS author here. You can run LiteFS anywhere -- even on a laptop with a few instances. There's no special Fly.io lock-in. The "fly-replay" is some syntactic sugar for our proxy that makes it easy to redirect requests but you could do the same with regular HTTP proxying. > With LiteFS, what happens if master goes down? Can a new one be elected? Yes, if you use Consul-based leasing then a new primary will come up automatically. Consul sessions have a minimum TTL of 10 seconds so you could lose write availability for up to that amount of time in the event of unexpectedly losing the primary. If you cleanly shutdown the primary then it will release its session and another node will pick up the lease right away. The other option is static leasing where a single node will always be primary. This is a simpler option but you do lose write availability if your primary node goes down.

mildbyte 4y ago

You can indeed run LiteFS by yourself, without Consul, as a sidecar / wrapper around your application. We do it in our project and have a Docker Compose example at [0]. In this case, you specify a specific known leader node. We haven't tried getting it running independently with Consul to do leader election / failover. [0] https://github.com/splitgraph/seafowl/blob/main/examples/litefs/configs/seafowl-writer/litefs.yml#L4 https://github.com/splitgraph/seafowl/blob/main/examples/lit...

benbjohnson 4y ago

> Seems there is a bit of vendor lock-in here with flyio, the way he's throwing custom http error codes for their infra. Can I just run LiteFS on a few virtual machines and forward writes back to master? LiteFS author here. You can run LiteFS anywhere -- even on a laptop with a few instances. There's no special Fly.io lock-in. The "fly-replay" is some syntactic sugar for our proxy that makes it easy to redirect requests but you could do the same with regular HTTP proxying. > With LiteFS, what happens if master goes down? Can a new one be elected? Yes, if you use Consul-based leasing then a new primary will come up automatically. Consul sessions have a minimum TTL of 10 seconds so you could lose write availability for up to that amount of time in the event of unexpectedly losing the primary. If you cleanly shutdown the primary then it will release its session and another node will pick up the lease right away. The other option is static leasing where a single node will always be primary. This is a simpler option but you do lose write availability if your primary node goes down.