5 ms·

> You don’t need to route every write to primary though, but only those writes that have dependencies on other writes. Thanks, can you give an example of how t

by nickjj 4y ago

> You don’t need to route every write to primary though, but only those writes that have dependencies on other writes. Thanks, can you give an example of how that works? Did you write your own fork of Postgres or are you using a third party solution like BDR? Also do you have a few use cases where you'd want writes being dependent on another write? > 6-7 probably do the job quite well You could, let's call it 5. For a 2gb set up would that be about $50 for the web app, $50 for the background workers, $160ish for postgres and then $50 for Redis? We're still at $300+? I was thinking maybe 5 background workers wasn't necessary but frameworks like Rails will put a bunch of things through a background worker where you would want low latency even if they're happening in the background because it's not only things like sending an email where it doesn't matter if it's delayed for 2 seconds behind the scenes. It's performing various Hotwire Turbo actions which render templates and modify records where you'd want to see those things reflected in the web UI as soon as possible.

nicoburns 4y ago
For low-latency workers like that it might make sense to just run them on the same instance as the web servers.
- nickjj 4y ago
  Does Fly let you run multiple commands in separate Docker images? That's usually the pattern on how to run a web app + worker with Docker, as opposed to creating an init system in Docker and running (2) processes in 1 container (this goes against best practices). The Fly docs only mention the approach of using an init system inside of your image and also tries to talk you into running a separate VM[0] to keep your web app + worker isolated. In either case I think the price still doubles because both your web app and worker need memory for a bunch of common set ups like Rails + Sidekiq, Flask / Django + Celery, etc.. [0]: https://fly.io/docs/app-guides/multiple-processes/ https://fly.io/docs/app-guides/multiple-processes/
  - tptacek 4y ago
    It sounds like you're asking if we offer some alternative between running multiple processes in a VM, and running multiple VMs for multiple processes. What's the third option you're looking for? Are you asking if you can run Docker inside a VM, and parcel that single VM out that way? You've got root in a full-fledged Linux VM, so you can do that.
    - nickjj 4y ago
      > Are you asking if you can run Docker inside a VM, and parcel that single VM out that way? You've got root in a full-fledged Linux VM, so you can do that. On a single server VPS I'd use Docker Compose and up the project to run multiple containers. On a multi-server set up I'd use Kubernetes and set up a deployment for each long running container. On Heroku I'd use a Procfile to spin up web / workers as needed. The Fly docs say if you have 1 Docker image you need to run an init system in the Docker image and manage that in your image, it also suggests not using 2 processes in 1 VM and recommends spinning up 1 VM per process. I suppose I was looking for an easy solution to run multiple processes in 1 VM (in this case multiple Docker containers). The other 3 solutions are IMO easy because once you learn how they work you depend on the happy path of those tools using the built in mechanisms they support. In the Fly case, not even the docs cover how to do it other than rolling your own init system in Docker. If you have root, can I run docker-compose up in a Fly VM? Will it respect things like graceful timeouts out of the box? Does it support everything Docker Compose supports in the context of that single VM?
      - tptacek 4y ago
        The document you cited (I wrote it!) is entirely about the different ways to run multiple processes in 1 VM. There's no reason I can see why you couldn't run a VM that itself ran Docker, and have docker-compose run at startup. I wouldn't recommend it? It's kind of a lot of mechanism for a simple problem. I'd just use a process supervisor instead. But you could do it, and maybe I'm wrong and docker-compose is good for this. What you can't do is use docker-compose to boot up a bunch of different containers in different VMs on Fly.io.
manmal 4y ago
> Thanks, can you give an example of how that works? I just noticed I formulated it wrong, my apologies. What I meant is that the replicating regions don’t need to wait for the primary writes to go through before they respond to clients. They will still be read-only Postgres replicas, and info could be shuttled to primary in a fire-and-forget manner, if that’s an option. Whenever an instance notices that it‘s not primary, but it is currently dealing with a critical write, it can refuse to handle the request, and return a 409 with the fly-replay header that specifies the primary region. Their infra will replay the original request in the specified region. > Did you write your own fork of Postgres or are you using a third party solution like BDR? When using fly.io, the best option would probably be to use their postgres cluster service which supports read-only replicas (can take a few seconds for updates to reach replicas): https://fly.io/docs/getting-started/multi-region-databases/ https://fly.io/docs/getting-started/multi-region-databases/ > For a 2gb set up would that be about $50 for the web app, $50 for the background workers, $160ish for postgres and then $50 for Redis? We're still at $300+? Maybe. A few thoughts: - Why would you need 5 web workers, would one running on primary not be ideal? If you need so much compute for background work, then that’s not fly‘s fault, I guess. - Not sure the Postgres read replicas would need to be as powerful as primary - Crazy idea: Use SQLite (replicated with Litestream) instead of Redis and save 50 bucks
- nickjj 4y ago
  > Why would you need 5 web workers, would one running on primary not be ideal? It's not ideal due to some frameworks using background jobs to handle pushing events through to your web UI, such as broadcasting changes over websockets with Hotwire Turbo. The UI would update when that job completes and if you only have 1 worker then it's back to waiting 100-350ms to reach the primary worker to see UI changes based on your location which loses the appeal of global distribution. You might as well consider running everything on 1 DigitalOcean server for 15x less at this point and bypass the idea of global distribution if your goal was to reduce latency for your visitors. > Crazy idea: Use SQLite (replicated with Litestream) instead of Redis and save 50 bucks A number of web frameworks let you use Redis as a session, cache and job queue back-end with no alternatives (or having to make pretty big compromises to use a SQL DB as an alternative). Also, Rails depends on Redis for Action Cable, swapping that for SQLite isn't an option.

by nickjj 4y ago

nicoburns 4y ago

For low-latency workers like that it might make sense to just run them on the same instance as the web servers.

nickjj 4y ago
Does Fly let you run multiple commands in separate Docker images? That's usually the pattern on how to run a web app + worker with Docker, as opposed to creating an init system in Docker and running (2) processes in 1 container (this goes against best practices). The Fly docs only mention the approach of using an init system inside of your image and also tries to talk you into running a separate VM[0] to keep your web app + worker isolated. In either case I think the price still doubles because both your web app and worker need memory for a bunch of common set ups like Rails + Sidekiq, Flask / Django + Celery, etc.. [0]: https://fly.io/docs/app-guides/multiple-processes/ https://fly.io/docs/app-guides/multiple-processes/
- tptacek 4y ago
  It sounds like you're asking if we offer some alternative between running multiple processes in a VM, and running multiple VMs for multiple processes. What's the third option you're looking for? Are you asking if you can run Docker inside a VM, and parcel that single VM out that way? You've got root in a full-fledged Linux VM, so you can do that.
  - nickjj 4y ago
    > Are you asking if you can run Docker inside a VM, and parcel that single VM out that way? You've got root in a full-fledged Linux VM, so you can do that. On a single server VPS I'd use Docker Compose and up the project to run multiple containers. On a multi-server set up I'd use Kubernetes and set up a deployment for each long running container. On Heroku I'd use a Procfile to spin up web / workers as needed. The Fly docs say if you have 1 Docker image you need to run an init system in the Docker image and manage that in your image, it also suggests not using 2 processes in 1 VM and recommends spinning up 1 VM per process. I suppose I was looking for an easy solution to run multiple processes in 1 VM (in this case multiple Docker containers). The other 3 solutions are IMO easy because once you learn how they work you depend on the happy path of those tools using the built in mechanisms they support. In the Fly case, not even the docs cover how to do it other than rolling your own init system in Docker. If you have root, can I run docker-compose up in a Fly VM? Will it respect things like graceful timeouts out of the box? Does it support everything Docker Compose supports in the context of that single VM?
    - tptacek 4y ago
      The document you cited (I wrote it!) is entirely about the different ways to run multiple processes in 1 VM. There's no reason I can see why you couldn't run a VM that itself ran Docker, and have docker-compose run at startup. I wouldn't recommend it? It's kind of a lot of mechanism for a simple problem. I'd just use a process supervisor instead. But you could do it, and maybe I'm wrong and docker-compose is good for this. What you can't do is use docker-compose to boot up a bunch of different containers in different VMs on Fly.io.

manmal 4y ago

> Thanks, can you give an example of how that works? I just noticed I formulated it wrong, my apologies. What I meant is that the replicating regions don’t need to wait for the primary writes to go through before they respond to clients. They will still be read-only Postgres replicas, and info could be shuttled to primary in a fire-and-forget manner, if that’s an option. Whenever an instance notices that it‘s not primary, but it is currently dealing with a critical write, it can refuse to handle the request, and return a 409 with the fly-replay header that specifies the primary region. Their infra will replay the original request in the specified region. > Did you write your own fork of Postgres or are you using a third party solution like BDR? When using fly.io, the best option would probably be to use their postgres cluster service which supports read-only replicas (can take a few seconds for updates to reach replicas): https://fly.io/docs/getting-started/multi-region-databases/ https://fly.io/docs/getting-started/multi-region-databases/ > For a 2gb set up would that be about $50 for the web app, $50 for the background workers, $160ish for postgres and then $50 for Redis? We're still at $300+? Maybe. A few thoughts: - Why would you need 5 web workers, would one running on primary not be ideal? If you need so much compute for background work, then that’s not fly‘s fault, I guess. - Not sure the Postgres read replicas would need to be as powerful as primary - Crazy idea: Use SQLite (replicated with Litestream) instead of Redis and save 50 bucks

nickjj 4y ago
> Why would you need 5 web workers, would one running on primary not be ideal? It's not ideal due to some frameworks using background jobs to handle pushing events through to your web UI, such as broadcasting changes over websockets with Hotwire Turbo. The UI would update when that job completes and if you only have 1 worker then it's back to waiting 100-350ms to reach the primary worker to see UI changes based on your location which loses the appeal of global distribution. You might as well consider running everything on 1 DigitalOcean server for 15x less at this point and bypass the idea of global distribution if your goal was to reduce latency for your visitors. > Crazy idea: Use SQLite (replicated with Litestream) instead of Redis and save 50 bucks A number of web frameworks let you use Redis as a session, cache and job queue back-end with no alternatives (or having to make pretty big compromises to use a SQL DB as an alternative). Also, Rails depends on Redis for Action Cable, swapping that for SQLite isn't an option.