8 ms·
It also sounds ideal for resuming memory intensive per-user programs, like LLMs with a large context window. You can basically have an executable (and its memor
by chatmasta 2y ago
It also sounds ideal for resuming memory intensive per-user programs, like LLMs with a large context window. You can basically have an executable (and its memory) attached to a user session, but only pay the cost for it while the user session has an open request.
- dividuum 2y agoYes > TinyKVM can fork itself into copies that use copy-on-write to allow for huge workloads like LLMs to share most memory. As an example, 6GB weights required only 260MB working memory per instance, making it highly scalable.
- daralthus 2y agoyes that's the durable objects ~ durable agents model that cloudflare is building