4 ms·

> The giant feature is "start a VM in less than 1 second". Followed by things like suspending/resuming VMs. Is this for scaling existing apps up (eg flyctl reg

> The giant feature is "start a VM in less than 1 second". Followed by things like suspending/resuming VMs. Is this for scaling existing apps up (eg flyctl regions add ...) or for new deployments (flyctl deploy)? I suspect the former (scaling existing) as artifact distribution and deployment parameters like health checks seem like they prevent subsecond deployments (even ignoring the time for initial artifact uploading/building). Although my view is probably quite biased by Nomad having a much better chance at solving the former than the latter! (Also hi! I appreciate our chats in the past and feel free to reach out if you're willing to brain dump Nomad gripes on me. :) )

mrkurt 5y ago
It's very different than Nomad! The basic model is direct "machine" management with very little orchestration. Machines are kind of like allocations, but more permanent. Some machines operations are slow, some are fast. Create is slow, update is slow. Start and stop are fast. Future "suspend" and "resume" are very fast. For the regions case, people use these by having a couple of machines per region they care about. Then they start and stop as needed. A stopped machine can be migrated between nodes, but we ensure it's ready to start quickly if we do that. Some interesting things happen with this model. Create is slow, but we are able to make it fast _sometimes_ because we can target creates at hardware that already has the rootfs cached. Updates don't work like nomad, we update machines in place for a rolling deploy. This lets us do things like pull an image in parallel, then stop the machine, update, and restart very quickly. We're also doing things differently with health checks. We may not wait for an active health checks before sending a machine an HTTP request. Many workloads don't even care about health checks, they just want a VM running and streaming logs back as fast as possible. There are features I could ask for in Nomad to do what we'd need, but they'd make it not Nomad. :D

mrkurt 5y ago

It's very different than Nomad! The basic model is direct "machine" management with very little orchestration. Machines are kind of like allocations, but more permanent. Some machines operations are slow, some are fast. Create is slow, update is slow. Start and stop are fast. Future "suspend" and "resume" are very fast. For the regions case, people use these by having a couple of machines per region they care about. Then they start and stop as needed. A stopped machine can be migrated between nodes, but we ensure it's ready to start quickly if we do that. Some interesting things happen with this model. Create is slow, but we are able to make it fast _sometimes_ because we can target creates at hardware that already has the rootfs cached. Updates don't work like nomad, we update machines in place for a rolling deploy. This lets us do things like pull an image in parallel, then stop the machine, update, and restart very quickly. We're also doing things differently with health checks. We may not wait for an active health checks before sending a machine an HTTP request. Many workloads don't even care about health checks, they just want a VM running and streaming logs back as fast as possible. There are features I could ask for in Nomad to do what we'd need, but they'd make it not Nomad. :D