3 ms·
So fork time is actually O(1) with VM size, its 500ms even for 64gb + disk. We're using some pretty weird COW techniques to pull it off.
by benswerd 6mo ago
So fork time is actually O(1) with VM size, its 500ms even for 64gb + disk. We're using some pretty weird COW techniques to pull it off.
- ianberdin 6mo agoInsane. Does it possible to fork to another bare metal machine? Maybe multi region as fly io. If not, I bet you have huge disk sizes on your machines to store all the snapshots (you said, you store them and bill only for disk space).
- benswerd 6mo agoSo forking across multiple nodes in that speed is not possible — we run extremely beefy nodes in order to avoid moving VMs across nodes as much as possible. We are researching systems of hot moving VMs across VMs but it would have very different performance characteristics.
- ianberdin 6mo agoYeah, I see. Is it possible to get a corrupted state? Let’s say we had realtime database actively writing at that moment?
- benswerd 6mo agoIt is impossible. Our tech is not decades old so there is a chance we've missed something but our layer management is atomic so I'd be shocked if you'd be able to corrupt state across forks/snapshots.
- ghm2199 6mo agoO(1) What! What might bring it down to say 10's of ms? Looks like its some kind of optimizable wall that its 500 for everything. Like with 10ms then online replication/backup — analogus to litestream for sqlite — but for in memory processes becomes feasible, no?
- benswerd 6mo agoWe're actually median under 500ms — ~320ms median — I just didn't want to piss of hacker news with over estimatation. We have another set of optimizations that we believe can take us to ~200ms in the next few months but beyond that we're pretty much completely stuck. Realistically other sandboxes will be able to get there before us because we've chosen to support so much of Linux/if you don't run an operating system or don't support custom snapshots that is much easier.