74 ms·

> For example Jai and Odin both seem to really want you to write code which can throw away a "per-frame" arena periodically You can generalize this far beyond

> For example Jai and Odin both seem to really want you to write code which can throw away a "per-frame" arena periodically You can generalize this far beyond per-frame semantics. Think per-http-request, per-pubsub-message. Per each, you can create a new virtual.Arena, set it as your context.temp_allocator and use it for the entirety of the request or message. Afterwards, throw it away.

tialaramex 6d ago
The nice thing about video game frames is that they're monotonous. We render exactly one at a time for ever. So all the memory we don't reserve for long-lived data forms a single Arena for our per-frame temporary storage and we can put all kinds of stuff in there "for free" so long as we don't fill the Arena in any one frame we're all good. Our HTTP server may have dozens, or hundreds, or thousands of simultaneous requests. But we've decided to give each a specific arena with a size chosen in advance so that we can use arenas. This means that the request to check a CSS file hasn't changed (it has not) and the request to modify the holiday of an employee whose manager is changing mid-way through their annual leave are both given, say, a 10MB Arena. No problem for that CSS check, some text parsing, one OS call, an ALU operation, done. But alas the SQL Transaction for that holiday change used up so much memory now the text parsing code "temporary" allocation in code to queue email for both managers fails and it unwinds the whole HTTP request. 500 error, rewrite your Odin program. Arenas make lots of sense for video games, but their applicability for these other applications is much more dubious.
- zbentley 6d ago
  > Arenas make lots of sense for video games, but their applicability for these other applications is much more dubious. It's a fairly common pattern in non-video contexts to preallocate arenas of different sizes for different workload pools (e.g. different routes for a server). It's also fairly common to allocate per-work-item (e.g. request/session/batch job) arenas for "general-purpose" scratch allocations that are small (your header parsing, auth context retrieval, etc.) and then default to a global manual/refcounting allocator for one-off large data actions like your holiday change. Since most business applications are returning summary/small aggregates over the large data action (in this example, something like "holiday updated successfully", not the entire history of the PTO table or the database connection's internal state), the copy cost of moving data between the global dynamic allocator and the arena for result transmission tends to be small. That's not a terrible approach in some situations. If the large majority of code only needs the scratch arena and/or some per-handle allocators stored on e.g. the database connection pool, it can work well. But if, over time, the amount of bookkeeping required to maintain data tagging/movement between arenas/allocators becomes severe, it's worth stopping, stepping back, and considering that you've kind of walked backwards into inventing a shitty generational GC.

tialaramex 6d ago

The nice thing about video game frames is that they're monotonous. We render exactly one at a time for ever. So all the memory we don't reserve for long-lived data forms a single Arena for our per-frame temporary storage and we can put all kinds of stuff in there "for free" so long as we don't fill the Arena in any one frame we're all good. Our HTTP server may have dozens, or hundreds, or thousands of simultaneous requests. But we've decided to give each a specific arena with a size chosen in advance so that we can use arenas. This means that the request to check a CSS file hasn't changed (it has not) and the request to modify the holiday of an employee whose manager is changing mid-way through their annual leave are both given, say, a 10MB Arena. No problem for that CSS check, some text parsing, one OS call, an ALU operation, done. But alas the SQL Transaction for that holiday change used up so much memory now the text parsing code "temporary" allocation in code to queue email for both managers fails and it unwinds the whole HTTP request. 500 error, rewrite your Odin program. Arenas make lots of sense for video games, but their applicability for these other applications is much more dubious.

zbentley 6d ago
> Arenas make lots of sense for video games, but their applicability for these other applications is much more dubious. It's a fairly common pattern in non-video contexts to preallocate arenas of different sizes for different workload pools (e.g. different routes for a server). It's also fairly common to allocate per-work-item (e.g. request/session/batch job) arenas for "general-purpose" scratch allocations that are small (your header parsing, auth context retrieval, etc.) and then default to a global manual/refcounting allocator for one-off large data actions like your holiday change. Since most business applications are returning summary/small aggregates over the large data action (in this example, something like "holiday updated successfully", not the entire history of the PTO table or the database connection's internal state), the copy cost of moving data between the global dynamic allocator and the arena for result transmission tends to be small. That's not a terrible approach in some situations. If the large majority of code only needs the scratch arena and/or some per-handle allocators stored on e.g. the database connection pool, it can work well. But if, over time, the amount of bookkeeping required to maintain data tagging/movement between arenas/allocators becomes severe, it's worth stopping, stepping back, and considering that you've kind of walked backwards into inventing a shitty generational GC.