6 ms·

Static Allocation with Zig

217 points by todsacerdoti 10mo ago

leumassuehtam 10mo ago
> All memory must be statically allocated at startup. No memory may be dynamically allocated (or freed and reallocated) after initialization. This avoids unpredictable behavior that can significantly affect performance, and avoids use-after-free. As a second-order effect, it is our experience that this also makes for more efficient, simpler designs that are more performant and easier to maintain and reason about, compared to designs that do not consider all possible memory usage patterns upfront as part of the design. > TigerStyle It's baffling that a technique known for 30+ years in the industry have been repackage into "tiger style" or whatever this guru-esque thing this is.
- addaon 10mo ago
  Yep. Those of us who write embedded code for a living call this “writing code.”
  - nickmonad 10mo ago
    Author here! That's totally fair. I did learn this is a common technique in the embedded world and I had a whole section in the original draft about how it's not a super well-known technique in the typical "backend web server" world, but I wanted to keep the length of the post down so I cut that out. I think there's a lot we can learn from embedded code, especially around performance.
    - titzer 10mo ago
      Back in 2005 Virgil I, which target MCUs like AVR, had static initialization and would generate a C program with all of the heap statically allocated, which was then compiled into the binary. C programmers for AVR are used to just declaring globals, but Virgil allowed arbitrary code to run which just initialized a heap. Virgil II and III inherited this. It's a standard part of a Virgil program that its components and top-level initializers run at compile time, and the resulting heap is then optimized and serialized into the binary. It doesn't require passing allocators around, it's just part of how the language works.
- testdelacc1 10mo ago
  It is known, but not widely used outside of embedded programming. The fact that they’re using it while writing a database when they didn’t need to makes people sit up and notice. So why did they make this conscious choice? It’s tempting to cut people down to size, but I don’t think it’s warranted here. I think TigerBeetle have created something remarkable and their approach to programming is how they’ve created it.
  - deleted 10mo ago
    [deleted]
- codys 10mo ago
  It seems it's just a part of a doc on style in tigerbeatle, in a similar way to the various "Google Style Guide" for code. These rarely have something new, but document what a particular project or organization does with respect to code style.
- jandrewrogers 10mo ago
  Static allocation has been around for a long time but few people consider it even in contexts where it makes a lot of sense. I’ve designed a few database engines that used pure static allocation and developers often chafe at this model because it seems easier to delegate allocation (which really just obscures the complexity). Allocation aside, many optimizations require knowing precisely how close to instantaneous resource limits the software actually is, so it is good practice for performance engineering generally. Hardly anyone does it (look at most open source implementations) so promoting it can’t hurt.
  - wahern 10mo ago
    I've always thought static allocation was why we got overcommit[1] in Linux and its infamous OOM killer. In the 1990s big boy commercial databases assumed specialized admins, and one of their tasks was to figure out the value for the memory allocation setting in the DB configuration, which the DB would immediately allocate on startup. As a magic value, the easiest path forward was just to specify most of your RAM. DBs used to run on dedicated machines, anyhow. But then Linux came along and democratized running servers, and people wanted to run big boy databases alongside other services like Apache. Without overcommit these databases wouldn't run as typically configured--"best practice" allocation advice used up too much memory, leaving nothing for the rest of the services, especially on the more memory-constrained machines people ran Linux. Because on a typical system most of the memory preallocated to the DB was never used anyhow (the figure wasn't actually carefully chosen as intended), or the DB was designed (or at least the manual's written) with bigger machines in mind, and Linus wanted things to Just Work, whether experienced admins or not, the easy fix was just to overcommit in the kernel, et voila, a pain point for people dabbling with Linux was solved, at least superficially. NB: I was just a newbie back then, so any older grey beards, please feel free to correct me. But I distinctly remember supporting commercial databases as being one of the justifications for overcommit, despite overcommit not being typical in the environments originally running those DBs, AFAIU. [1] Note that AFAIU the BSDs had overcommit, too, but just for fork + CoW. Though these days FreeBSD at least has overcommit more similar to Linux. Solaris actually does strict accounting even for fork, and I assume that was true back in the 90s. Did any commercial Unices actually do overcommit by default?
- mitchellh 10mo ago
  Snide and condescending (or at best: dismissive) comments like this help no one and can at the extremes stereotype an entire group in a bad light. I think the more constructive reality is discussing why techniques that are common in some industries such as gaming or embedded systems have had difficulty being adopted more broadly, and celebrating that this idea which is good in many contexts is now spreading more broadly! Or, sharing some others that other industries might be missing out on (and again, asking critically why they aren't present). Ideas in general require marketing to spread, that's literally what marketing is in the positive (in the negative its all sorts of slime!). If a coding standard used by a company is the marketing this idea needs to live and grow, then hell yeah, "tiger style" it is! Such is humanity.
  - pjmlp 10mo ago
    It was a common practice in 8 and 16 bit home computing.
  - brabel 10mo ago
    > had difficulty being adopted more broadly Most applications don’t need to bother the user with things like how much memory they think will be needed upfront. They just allocate how much and when necessary. Most applications today are probably servers that change all the time. You would not know upfront how much memory you’d need as that would keep changing on every release! Static allocation may work in a few domains but it certainly doesn’t work in most.
    - AlotOfReading 10mo ago
      It's best to think of it as an entire spectrum from "statically allocate everything with compile time parameters" to "directly call the system allocator for every new bit of memory". It's just a helpful way to separate the concerns of memory allocation from memory usage. What this article is talking about isn't all the way at the other end (compile time allocation), but has the additional freedom that you can decide allocation size based on runtime parameters. That frees the rest of the application from needing to worry about managing memory allocations. We can imagine taking another step and only allocating at the start of a connection/request, so the rest of the server code doesn't need to deal with managing memory everywhere. This is more popularly known as region allocation. If you've ever worked with Apache or Nginx, this is what they do ("pools"). So on and so forth down into the leaf functions of your application. Your allocator is already doing this internally to help you out, but it doesn't have any knowledge of what your code looks like to optimize its patterns. Your application's performance (and maintainability) will usually benefit from doing it yourself, as much as you reasonably can.
- publicdebates 10mo ago
  > a technique known for 30+ years in the industry have been Knowledge sharing with next generations is one of those very tricky things. For one thing, how would I know where to find this? What book? What teacher? There are so many books, must I read all of them? What if my coworkers awaren't aware of it, how can they share it with me? Also, an old saying goes, if you're good at something, never do it for free. This isn't exactly a trade secret, but how many people blog about every advanced technique and trick they know? I blogged about how to create real C function pointers from Lua closures, as a way to advertise for my product, but that could very well have been kept a trade secret (and probably should have, as I got 0 sales from that blog post still). Why would anyone want to share this "tiger style" knowledge with newer generations with no personal benefit? Aren't they incentivized to use it secretly, or maybe write it in a book, or blog about it for advertising?
  - Nevermark 10mo ago
    Trade secrets are not necessary to build up interest/visibility momentum. And likely "wasted" for external rewards, without pre-existing momentum. (External = a marketing/sales bump. vs. Internal = enjoy sharing, writing to self-clarify, writing practice.) Developing the abilities to repeatedly pick attractive topics (not as widely known as they are useful/interesting), and communicating (in a clear well-received voice/style), until those are validated by increasing visibility and sales/marketing impact, is where to start. After that, the choice to share a trade secret can be made based on a more predictable reward trade off. But every post already made, is a post to build on.
  - mytailorisrich 10mo ago
    I'd say usually you learn those techniques when you join a company as a junior dev and come in contact with engineers with decades of experience and systems that have been in production for years, too. I think it's when people consider that anything more than 5 years old is ancient and dismiss it that we lose established techniques and knowledge that we are then bound to rediscover again and again.
- matklad 10mo ago
  To add more context, TigerStyle is quite a bit more than just static allocation, and it indeed explicitly attributes earlier work: > NASA's Power of Ten — Rules for Developing Safety Critical Code will change the way you code forever. To expand: * https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/TIGER_STYLE.md#safety https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/TI... * https://spinroot.com/gerard/pdf/P10.pdf https://spinroot.com/gerard/pdf/P10.pdf
  - kibwen 10mo ago
    Those guidelines are quite clear that they're written specifically in the context of the C programming language, and may not make sense in other contexts: "For fairly pragmatic reasons, then, our coding rules primarily target C and attempt to optimize our ability to more thoroughly check the reliability of critical applications written in C." A version of this document targeting, say, Ada would look quite different.
    - AlotOfReading 10mo ago
      The JPL C rules are quite old, but avoiding dynamic allocation outside initialization is am considered best practice for spaceflight software regardless of language. Here's the recommendation from NASA's language-agnostic cFS: 4.2.4 Consolidate Resource Allocations It is generally recommended to consolidate resource allocations to the application initialization function(s). Allocations and setup of resources such as memory pools and child tasks should happen once during initialization in order to provide more determinism during run time. From: https://github.com/nasa/cFE/blob/main/docs/cFE%20Application%20Developers%20Guide.md#424-consolidate-resource-allocations https://github.com/nasa/cFE/blob/main/docs/cFE%20Application... The ESA Ada standard also recommends all allocation occur at initialization, and requires exceptions to be justified.
      - kibwen 10mo ago
        > The JPL C rules are quite old, but avoiding dynamic allocation outside initialization is am considered best practice for spaceflight software regardless of language. The rules are written with the historical context of C making it too easy to leak heap-allocated memory. In the safety-critical Rust code that I've worked on, we tend not to dynamically allocate due to the usual constraints, and we're well aware of the "thou shalt not allocate" rules in the scripture, but we've already gotten clearance from the relevant certification authorities that Rust is exempt from the restriction against dynamic allocation specifically because of its ownership system.
- zozbot234 10mo ago
  > All memory must be statically allocated at startup. No memory may be dynamically allocated (or freed and reallocated) after initialization ... It's baffling that a technique known for 30+ years in the industry have been repackaged This is also how GPU shader programming works: there's no real equivalent to heap allocation or general pointers, you're expected to work as far as possible with local memory and sometimes preallocated shared buffers. So the technique may be quite relevant in the present day, even though it has a rather extensive history of its own.
- array_key_first 10mo ago
  I think it's more relevant than ever when most systems seem to dynamically allocate everything, some down to each individual object being its own runtime allocation, popularized by the likes of java. On the surface it seems great: infinite scale and perfectly generic. The system can handle anything. But does it need to handle everything? And, what's the cost in terms of complexity of handling every single possible scenario?
deleted 10mo ago
[deleted]
ivanjermakov 10mo ago
Related recent post from Tigerbeetle developer: https://matklad.github.io/2025/12/23/static-allocation-compilers.html https://matklad.github.io/2025/12/23/static-allocation-compi...
Zambyte 10mo ago
I'm doing pretty much this exact pattern with NATS right now instead of Redis. Cool to see other people following similar strategies. The fact that the Zig ecosystem follows the pattern set by the standard library to pass the Allocator interface around makes it super easy to write idiomatic code, and then decide on your allocation strategies at your call site. People have of course been doing this for decades in other languages, but it's not trivial to leverage existing ecosystems like libc while following this pattern, and your callees usually need to know something about the allocation strategy being used (even if only to avoid standard functions that do not follow that allocation strategy).
- nickmonad 10mo ago
  I have a few cases in this (proof of concept) codebase that require knowledge about allocation strategy, even in Zig, but that's on me and the design at this point. Something I wanted to touch on more in the post was the attempt to make the components of the system work with any kind of allocation strategy. I see a common thing in Zig projects today where something like `gpa: std.mem.Allocator` or even `arena: std.mem.Allocator` is used to signal intent, even though the allocator interface is generic.
d3ckard 10mo ago
Personally I believe static allocation has pretty huge consequences for theoretical computer science. It’s the only kind of program that can be actually reasoned about. Also, not exactly Turing complete in classic sense. Makes my little finitist heart get warm and fuzzy.
- dnautics 10mo ago
  i think you mean "exactly not Turing complete"
  - d3ckard 10mo ago
    Nice correction :) It’s actually quite tricky though. The allocation still happens and it’s not limited to, so you could plausibly argue both ways.
    - chongli 10mo ago
      I’m confused. How is a program that uses static allocation not Turing complete?
      - skybrian 10mo ago
        A Turing machine has an unlimited tape. You can’t emulate it with a fixed amount of memory. It’s mostly a theoretical issue, though, because all real computer systems have limits. It’s just that in languages that assume unlimited memory, the limits aren’t written down. It’s not “part of the language.”
        dnautics 10mo ago
        If we get REALLY nitpicky, zig currently (but not in the future) allows unbounded function recursion with "theoretically" assumes unlimited stack size, so it's potentially "still technically theoretically turing complete". For now.
        chongli 10mo ago
loeg 10mo ago
We do a lazier version of this with a service at work. All of the large buffers and caches are statically (runtime-configured) sized, but various internal data structures assumed to be approximately de minimis can use the standard allocator to add items without worrying about it.
nromiun 10mo ago
> All memory must be statically allocated at startup. But why? If you do that you are just taking memory away from other processes. Is there any significant speed improvement over just dynamic allocation?
- AnimalMuppet 10mo ago
  1. On modern OSes, you probably aren't "taking it away from other processes" until you actually use it. Statically allocated but untouched memory is probably just an entry in a page table somewhere. 2. Speed improvement? No. The improvement is in your ability to reason about memory usage, and about time usage. Dynamic allocations add a very much non-deterministic amount of time to whatever you're doing.
  - Ericson2314 10mo ago
    If you use it and stop using it, the OS cannot reclaim the pages, because it doesn't know that you've stopped. At best, it can offload the memory to disk, but this waste disk space, and also time for pointless writes.
    - norir 10mo ago
      This is true, whether it matters is context dependent. In an embedded program, this may be irrelevant since your program is the only thing running so there is no resource contention or need to swap. In multi-tenant, you could use arenas in an identical way as single static allocation and release the arena upon completion. I agree that allocating a huge amount of memory for a long running program on a multi-tenant os is a bad idea in general, but it could be ok if for example you are running a single application like a database on the server in which you are back to embedded programming only the embedding is a database on a beefy general purpose computer.
      - Ericson2314 10mo ago
        Yes it is context dependent, but the parent comment was acting as if it was just better. I wanted to correct that.
dale-cooper 10mo ago
Maybe I'm missing something, but two thoughts: 1. Doesn't the overcommit feature lessen the benefits of this? Your initial allocation works but you can still run out of memory at runtime. 2. For a KV store, you'd still be at risk of application level use-after-free bugs since you need to keep track of what of your statically allocated memory is in use or not?
- nickmonad 10mo ago
  Author here! Overcommit is definitely a thing to watch out for. I believe TigerBeetle calls this out in their documentation. I think you'd have to explicitly disable it on Linux. For the second question, yes, we have to keep track of what's in use. The keys and values are allocated via a memory pool that uses a free-list to keep track of what's available. When a request to add a key/value pair comes in, we first check if we have space (i.e. available buffers) in both the key pool and value pool. Once those are marked as "reserved", the free-list kind of forgets about them until the buffer is released back into the pool. Hopefully that helps!
- justincormack 10mo ago
  You can work around overcommit by writing a byte to every allocated page at allocation time, so that it has to be actually allocated.
  - dnautics 10mo ago
    out of curiosity, does that generally mean that (linux) OOM killer can't get you? IIRC the oom killer is only triggered on new page request, and only the requesting process is eligble for the murder?
    - chris- 10mo ago
      No, it does not. The oom killer acts on (mostly) the oom score and no process is exepmt, regardless of whether or not it allocates new memory. It may help you write correct programs in certain situations though, eg. if your program was running in a defined context, eg. a cgroup, and you would not allocate beyond your cgroup limits, and the system was configured sanely, you can handle allocation problems easier.
kristoff_it 10mo ago
A coincidental counterpart to this project is my zero allocation Redis client. If kv supports RESPv3 then it should work without issue :^) https://github.com/kristoff-it/zig-okredis https://github.com/kristoff-it/zig-okredis
- nickmonad 10mo ago
  Nice! Will definitely take a look :)
lazy-lambda 10mo ago
No mention of AI - such a good post.
MatthiasPortzel 10mo ago
One key thing to understand about TigerBeetle is that it's a file-system-backed database. Static allocation means they limit the number of resources in memory at once (number of connections, number of records that can be returned from a single query, etc). One of the points is that these things are limited in practice anyways (MySQL and Postgres have a simultaneous connection limit, applications should implement pagination). Thinking about and specifying these limits up front is better than having operations time out or OOM. On the other hand, TigerBeetle does not impose any limit on the amount of data that can be stored in the database. => https://tigerbeetle.com/blog/2022-10-12-a-database-without-dynamic-memory/ https://tigerbeetle.com/blog/2022-10-12-a-database-without-d... It's always bad to use O(N) memory if you don't have to. With a FS-backed database, you don't have to. (Whether you're using static allocation or not. I work on a Ruby web-app, and we avoid loading N records into memory at once, using fixed-sized batches instead.) Doing allocation up front is just a very nice way of ensuring you've thought about those limits, and making sure you don't slip up, and avoiding the runtime cost of allocations. This is totally different from OP's situation, where they're implementing an in-memory database. This means that 1) they've had to impose a limit on the number of kv-pairs they store, and 2) they're paying the cost for all kv-pairs at startup. This is only acceptable if you know you have a fixed upper bound on the number of kv-pairs to store.
- levkk 10mo ago
  That makes sense. For example, your redis instance will have fixed RAM, so might as well pre-allocate it at boot and avoid fragmentation. Memcached works similarly (slabs of fixed size), except they are not pre-allocated. If you're sharing hardware with multiple services, e.g. web, database, cache, the kind of performance this is targeting isn't a priority.
- matklad 10mo ago
  Yes, very good point, thanks! As a tiny nit, TigerBeetle isn't _file system_ backed database, we intentionally limit ourselves to a single "file", and can work with a raw block device or partition, without file system involvement.
  - 10mo ago
netik 10mo ago
Didn’t we solve this already with slab allocators in memcached? The major problem with fixed allocation like this is fragmentation in memory over time, which you then have to reinvent GC for.
TheMagicHorsey 10mo ago
In a strange coincidence (or maybe its actually inevitable given the timing) I also saw a podcast with Matklad of Tigerbeetle and had a similar idea--I've been working on a massively multiplayer game as a hobby project, also built in zig, also fully allocating all memory at startup, and also had an experience almost identical to OP's. In my case both my client and server are in Zig. Zig is pretty great at doing performant game code (rendering and physics on the client) ... it's less great on the server compared to Go (early days and fewer batteries included, fewer things just work out of the box ... but you can find pretty much everything you need for a game server with a little hunting and pecking and a debugging few build issues). Zig also works "okay" with vibe coding. Golang works much better (maybe a function of the models I use (primarily through Cursor) or maybe its that there's less Zig code out in the wild for models to scrape and learn from, or maybe idiomatic Zig just isn't a thing yet the way it is with Go. Not quite sure.
- dnautics 10mo ago
  claude is really good at zig. amount of examples on the web is not a good predictor of llm capability, since we know you can poison an llm with ~250 examples. it doesn't take much.
xthe 10mo ago
Nice write-up. Static allocation forcing you to think about limits up front feels like a real design win
ogogmad 10mo ago
This might be a silly thing to point out, but where do people draw the line between an allocation happening or not happening? You still need to track vacant/occupied memory even when there's no OS or other programs around. It's especially bewildering when people claim that some database program "doesn't allocate".
- nickmonad 10mo ago
  This is the fundamental question which motivated the post. :) I think there are a few different ways to approach the answer, and it kind of depends on what you mean by "draw the line between an allocation happening or not happening." At the surface level, Zig makes this relatively easy, since you can grep for all instances of `std.mem.Allocator` and see where those allocations are occurring throughout the codebase. This only gets you so far though, because some of those Allocator instances could be backed by something like a FixedBufferAllocator, which uses already allocated memory either from the stack or the heap. So the usage of the Allocator instance at the interface level doesn't actually tell you "this is for sure allocating memory from the OS." You have to consider it in the larger context of the system. And yes, we do still need to track vacant/occupied memory, we just do it at the application level. At that level, the OS sees it all as "occupied". For example, in kv, the connection buffer space is marked as vacant/occupied using a memory pool at runtime. But, that pool was allocated from the OS during initialization. As we use the pool we just have to do some very basic bookkeeping using a free-list. That determines if a new connection can actually be accepted or not. Hopefully that helps. Ultimately, we do allocate, it just happens right away during initialization and that allocated space is reused throughout program execution. But, it doesn't have to be nearly as complicated as "reinventing garbage collection" as I've seen some other comments mention.
  - deleted 10mo ago
    [deleted]
CashWasabi 10mo ago
This also works well for games. I use a FixedBufferAllocator that allocates everything except assets upfront (systems, entities, etc.). Tigerstyle is a good starting point for efficient and debuggable software.Thanks for the article!
CashWasabi 10mo ago
Great read! I'm doing something similar with my game engine. I use a FixedBufferAllocator for static allocation and initialize/allocate all my systems and entities with the necessary size at the start. The only exception currently is asset loading because this can be quite dynamic at times.

217 points by todsacerdoti 10mo ago

> All memory must be statically allocated at startup. No memory may be dynamically allocated (or freed and reallocated) after initialization. This avoids unpredictable behavior that can significantly affect performance, and avoids use-after-free. As a second-order effect, it is our experience that this also makes for more efficient, simpler designs that are more performant and easier to maintain and reason about, compared to designs that do not consider all possible memory usage patterns upfront as part of the design. > TigerStyle It's baffling that a technique known for 30+ years in the industry have been repackage into "tiger style" or whatever this guru-esque thing this is.

addaon 10mo ago
Yep. Those of us who write embedded code for a living call this “writing code.”
- nickmonad 10mo ago
  Author here! That's totally fair. I did learn this is a common technique in the embedded world and I had a whole section in the original draft about how it's not a super well-known technique in the typical "backend web server" world, but I wanted to keep the length of the post down so I cut that out. I think there's a lot we can learn from embedded code, especially around performance.
  - titzer 10mo ago
    Back in 2005 Virgil I, which target MCUs like AVR, had static initialization and would generate a C program with all of the heap statically allocated, which was then compiled into the binary. C programmers for AVR are used to just declaring globals, but Virgil allowed arbitrary code to run which just initialized a heap. Virgil II and III inherited this. It's a standard part of a Virgil program that its components and top-level initializers run at compile time, and the resulting heap is then optimized and serialized into the binary. It doesn't require passing allocators around, it's just part of how the language works.
testdelacc1 10mo ago
It is known, but not widely used outside of embedded programming. The fact that they’re using it while writing a database when they didn’t need to makes people sit up and notice. So why did they make this conscious choice? It’s tempting to cut people down to size, but I don’t think it’s warranted here. I think TigerBeetle have created something remarkable and their approach to programming is how they’ve created it.
- deleted 10mo ago
  [deleted]
codys 10mo ago
It seems it's just a part of a doc on style in tigerbeatle, in a similar way to the various "Google Style Guide" for code. These rarely have something new, but document what a particular project or organization does with respect to code style.
jandrewrogers 10mo ago
Static allocation has been around for a long time but few people consider it even in contexts where it makes a lot of sense. I’ve designed a few database engines that used pure static allocation and developers often chafe at this model because it seems easier to delegate allocation (which really just obscures the complexity). Allocation aside, many optimizations require knowing precisely how close to instantaneous resource limits the software actually is, so it is good practice for performance engineering generally. Hardly anyone does it (look at most open source implementations) so promoting it can’t hurt.
- wahern 10mo ago
  I've always thought static allocation was why we got overcommit[1] in Linux and its infamous OOM killer. In the 1990s big boy commercial databases assumed specialized admins, and one of their tasks was to figure out the value for the memory allocation setting in the DB configuration, which the DB would immediately allocate on startup. As a magic value, the easiest path forward was just to specify most of your RAM. DBs used to run on dedicated machines, anyhow. But then Linux came along and democratized running servers, and people wanted to run big boy databases alongside other services like Apache. Without overcommit these databases wouldn't run as typically configured--"best practice" allocation advice used up too much memory, leaving nothing for the rest of the services, especially on the more memory-constrained machines people ran Linux. Because on a typical system most of the memory preallocated to the DB was never used anyhow (the figure wasn't actually carefully chosen as intended), or the DB was designed (or at least the manual's written) with bigger machines in mind, and Linus wanted things to Just Work, whether experienced admins or not, the easy fix was just to overcommit in the kernel, et voila, a pain point for people dabbling with Linux was solved, at least superficially. NB: I was just a newbie back then, so any older grey beards, please feel free to correct me. But I distinctly remember supporting commercial databases as being one of the justifications for overcommit, despite overcommit not being typical in the environments originally running those DBs, AFAIU. [1] Note that AFAIU the BSDs had overcommit, too, but just for fork + CoW. Though these days FreeBSD at least has overcommit more similar to Linux. Solaris actually does strict accounting even for fork, and I assume that was true back in the 90s. Did any commercial Unices actually do overcommit by default?
mitchellh 10mo ago
Snide and condescending (or at best: dismissive) comments like this help no one and can at the extremes stereotype an entire group in a bad light. I think the more constructive reality is discussing why techniques that are common in some industries such as gaming or embedded systems have had difficulty being adopted more broadly, and celebrating that this idea which is good in many contexts is now spreading more broadly! Or, sharing some others that other industries might be missing out on (and again, asking critically why they aren't present). Ideas in general require marketing to spread, that's literally what marketing is in the positive (in the negative its all sorts of slime!). If a coding standard used by a company is the marketing this idea needs to live and grow, then hell yeah, "tiger style" it is! Such is humanity.
- pjmlp 10mo ago
  It was a common practice in 8 and 16 bit home computing.
- brabel 10mo ago
  > had difficulty being adopted more broadly Most applications don’t need to bother the user with things like how much memory they think will be needed upfront. They just allocate how much and when necessary. Most applications today are probably servers that change all the time. You would not know upfront how much memory you’d need as that would keep changing on every release! Static allocation may work in a few domains but it certainly doesn’t work in most.
  - AlotOfReading 10mo ago
    It's best to think of it as an entire spectrum from "statically allocate everything with compile time parameters" to "directly call the system allocator for every new bit of memory". It's just a helpful way to separate the concerns of memory allocation from memory usage. What this article is talking about isn't all the way at the other end (compile time allocation), but has the additional freedom that you can decide allocation size based on runtime parameters. That frees the rest of the application from needing to worry about managing memory allocations. We can imagine taking another step and only allocating at the start of a connection/request, so the rest of the server code doesn't need to deal with managing memory everywhere. This is more popularly known as region allocation. If you've ever worked with Apache or Nginx, this is what they do ("pools"). So on and so forth down into the leaf functions of your application. Your allocator is already doing this internally to help you out, but it doesn't have any knowledge of what your code looks like to optimize its patterns. Your application's performance (and maintainability) will usually benefit from doing it yourself, as much as you reasonably can.
publicdebates 10mo ago
> a technique known for 30+ years in the industry have been Knowledge sharing with next generations is one of those very tricky things. For one thing, how would I know where to find this? What book? What teacher? There are so many books, must I read all of them? What if my coworkers awaren't aware of it, how can they share it with me? Also, an old saying goes, if you're good at something, never do it for free. This isn't exactly a trade secret, but how many people blog about every advanced technique and trick they know? I blogged about how to create real C function pointers from Lua closures, as a way to advertise for my product, but that could very well have been kept a trade secret (and probably should have, as I got 0 sales from that blog post still). Why would anyone want to share this "tiger style" knowledge with newer generations with no personal benefit? Aren't they incentivized to use it secretly, or maybe write it in a book, or blog about it for advertising?
- Nevermark 10mo ago
  Trade secrets are not necessary to build up interest/visibility momentum. And likely "wasted" for external rewards, without pre-existing momentum. (External = a marketing/sales bump. vs. Internal = enjoy sharing, writing to self-clarify, writing practice.) Developing the abilities to repeatedly pick attractive topics (not as widely known as they are useful/interesting), and communicating (in a clear well-received voice/style), until those are validated by increasing visibility and sales/marketing impact, is where to start. After that, the choice to share a trade secret can be made based on a more predictable reward trade off. But every post already made, is a post to build on.
- mytailorisrich 10mo ago
  I'd say usually you learn those techniques when you join a company as a junior dev and come in contact with engineers with decades of experience and systems that have been in production for years, too. I think it's when people consider that anything more than 5 years old is ancient and dismiss it that we lose established techniques and knowledge that we are then bound to rediscover again and again.
matklad 10mo ago
To add more context, TigerStyle is quite a bit more than just static allocation, and it indeed explicitly attributes earlier work: > NASA's Power of Ten — Rules for Developing Safety Critical Code will change the way you code forever. To expand: * https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/TIGER_STYLE.md#safety https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/TI... * https://spinroot.com/gerard/pdf/P10.pdf https://spinroot.com/gerard/pdf/P10.pdf
- kibwen 10mo ago
  Those guidelines are quite clear that they're written specifically in the context of the C programming language, and may not make sense in other contexts: "For fairly pragmatic reasons, then, our coding rules primarily target C and attempt to optimize our ability to more thoroughly check the reliability of critical applications written in C." A version of this document targeting, say, Ada would look quite different.
  - AlotOfReading 10mo ago
    The JPL C rules are quite old, but avoiding dynamic allocation outside initialization is am considered best practice for spaceflight software regardless of language. Here's the recommendation from NASA's language-agnostic cFS: 4.2.4 Consolidate Resource Allocations It is generally recommended to consolidate resource allocations to the application initialization function(s). Allocations and setup of resources such as memory pools and child tasks should happen once during initialization in order to provide more determinism during run time. From: https://github.com/nasa/cFE/blob/main/docs/cFE%20Application%20Developers%20Guide.md#424-consolidate-resource-allocations https://github.com/nasa/cFE/blob/main/docs/cFE%20Application... The ESA Ada standard also recommends all allocation occur at initialization, and requires exceptions to be justified.
    - kibwen 10mo ago
      > The JPL C rules are quite old, but avoiding dynamic allocation outside initialization is am considered best practice for spaceflight software regardless of language. The rules are written with the historical context of C making it too easy to leak heap-allocated memory. In the safety-critical Rust code that I've worked on, we tend not to dynamically allocate due to the usual constraints, and we're well aware of the "thou shalt not allocate" rules in the scripture, but we've already gotten clearance from the relevant certification authorities that Rust is exempt from the restriction against dynamic allocation specifically because of its ownership system.
zozbot234 10mo ago
> All memory must be statically allocated at startup. No memory may be dynamically allocated (or freed and reallocated) after initialization ... It's baffling that a technique known for 30+ years in the industry have been repackaged This is also how GPU shader programming works: there's no real equivalent to heap allocation or general pointers, you're expected to work as far as possible with local memory and sometimes preallocated shared buffers. So the technique may be quite relevant in the present day, even though it has a rather extensive history of its own.
array_key_first 10mo ago
I think it's more relevant than ever when most systems seem to dynamically allocate everything, some down to each individual object being its own runtime allocation, popularized by the likes of java. On the surface it seems great: infinite scale and perfectly generic. The system can handle anything. But does it need to handle everything? And, what's the cost in terms of complexity of handling every single possible scenario?

deleted 10mo ago

[deleted]

ivanjermakov 10mo ago

Related recent post from Tigerbeetle developer: https://matklad.github.io/2025/12/23/static-allocation-compilers.html https://matklad.github.io/2025/12/23/static-allocation-compi...

Zambyte 10mo ago

I'm doing pretty much this exact pattern with NATS right now instead of Redis. Cool to see other people following similar strategies. The fact that the Zig ecosystem follows the pattern set by the standard library to pass the Allocator interface around makes it super easy to write idiomatic code, and then decide on your allocation strategies at your call site. People have of course been doing this for decades in other languages, but it's not trivial to leverage existing ecosystems like libc while following this pattern, and your callees usually need to know something about the allocation strategy being used (even if only to avoid standard functions that do not follow that allocation strategy).

nickmonad 10mo ago
I have a few cases in this (proof of concept) codebase that require knowledge about allocation strategy, even in Zig, but that's on me and the design at this point. Something I wanted to touch on more in the post was the attempt to make the components of the system work with any kind of allocation strategy. I see a common thing in Zig projects today where something like `gpa: std.mem.Allocator` or even `arena: std.mem.Allocator` is used to signal intent, even though the allocator interface is generic.

d3ckard 10mo ago

Personally I believe static allocation has pretty huge consequences for theoretical computer science. It’s the only kind of program that can be actually reasoned about. Also, not exactly Turing complete in classic sense. Makes my little finitist heart get warm and fuzzy.

dnautics 10mo ago
i think you mean "exactly not Turing complete"
- d3ckard 10mo ago
  Nice correction :) It’s actually quite tricky though. The allocation still happens and it’s not limited to, so you could plausibly argue both ways.
  - chongli 10mo ago
    I’m confused. How is a program that uses static allocation not Turing complete?
    - skybrian 10mo ago
      A Turing machine has an unlimited tape. You can’t emulate it with a fixed amount of memory. It’s mostly a theoretical issue, though, because all real computer systems have limits. It’s just that in languages that assume unlimited memory, the limits aren’t written down. It’s not “part of the language.”
      - dnautics 10mo ago
        If we get REALLY nitpicky, zig currently (but not in the future) allows unbounded function recursion with "theoretically" assumes unlimited stack size, so it's potentially "still technically theoretically turing complete". For now.
      - chongli 10mo ago

loeg 10mo ago

We do a lazier version of this with a service at work. All of the large buffers and caches are statically (runtime-configured) sized, but various internal data structures assumed to be approximately de minimis can use the standard allocator to add items without worrying about it.

nromiun 10mo ago

> All memory must be statically allocated at startup. But why? If you do that you are just taking memory away from other processes. Is there any significant speed improvement over just dynamic allocation?

AnimalMuppet 10mo ago
1. On modern OSes, you probably aren't "taking it away from other processes" until you actually use it. Statically allocated but untouched memory is probably just an entry in a page table somewhere. 2. Speed improvement? No. The improvement is in your ability to reason about memory usage, and about time usage. Dynamic allocations add a very much non-deterministic amount of time to whatever you're doing.
- Ericson2314 10mo ago
  If you use it and stop using it, the OS cannot reclaim the pages, because it doesn't know that you've stopped. At best, it can offload the memory to disk, but this waste disk space, and also time for pointless writes.
  - norir 10mo ago
    This is true, whether it matters is context dependent. In an embedded program, this may be irrelevant since your program is the only thing running so there is no resource contention or need to swap. In multi-tenant, you could use arenas in an identical way as single static allocation and release the arena upon completion. I agree that allocating a huge amount of memory for a long running program on a multi-tenant os is a bad idea in general, but it could be ok if for example you are running a single application like a database on the server in which you are back to embedded programming only the embedding is a database on a beefy general purpose computer.
    - Ericson2314 10mo ago
      Yes it is context dependent, but the parent comment was acting as if it was just better. I wanted to correct that.

dale-cooper 10mo ago

Maybe I'm missing something, but two thoughts: 1. Doesn't the overcommit feature lessen the benefits of this? Your initial allocation works but you can still run out of memory at runtime. 2. For a KV store, you'd still be at risk of application level use-after-free bugs since you need to keep track of what of your statically allocated memory is in use or not?

nickmonad 10mo ago
Author here! Overcommit is definitely a thing to watch out for. I believe TigerBeetle calls this out in their documentation. I think you'd have to explicitly disable it on Linux. For the second question, yes, we have to keep track of what's in use. The keys and values are allocated via a memory pool that uses a free-list to keep track of what's available. When a request to add a key/value pair comes in, we first check if we have space (i.e. available buffers) in both the key pool and value pool. Once those are marked as "reserved", the free-list kind of forgets about them until the buffer is released back into the pool. Hopefully that helps!
justincormack 10mo ago
You can work around overcommit by writing a byte to every allocated page at allocation time, so that it has to be actually allocated.
- dnautics 10mo ago
  out of curiosity, does that generally mean that (linux) OOM killer can't get you? IIRC the oom killer is only triggered on new page request, and only the requesting process is eligble for the murder?
  - chris- 10mo ago
    No, it does not. The oom killer acts on (mostly) the oom score and no process is exepmt, regardless of whether or not it allocates new memory. It may help you write correct programs in certain situations though, eg. if your program was running in a defined context, eg. a cgroup, and you would not allocate beyond your cgroup limits, and the system was configured sanely, you can handle allocation problems easier.

kristoff_it 10mo ago

A coincidental counterpart to this project is my zero allocation Redis client. If kv supports RESPv3 then it should work without issue :^) https://github.com/kristoff-it/zig-okredis https://github.com/kristoff-it/zig-okredis

nickmonad 10mo ago
Nice! Will definitely take a look :)

lazy-lambda 10mo ago

No mention of AI - such a good post.

MatthiasPortzel 10mo ago

One key thing to understand about TigerBeetle is that it's a file-system-backed database. Static allocation means they limit the number of resources in memory at once (number of connections, number of records that can be returned from a single query, etc). One of the points is that these things are limited in practice anyways (MySQL and Postgres have a simultaneous connection limit, applications should implement pagination). Thinking about and specifying these limits up front is better than having operations time out or OOM. On the other hand, TigerBeetle does not impose any limit on the amount of data that can be stored in the database. => https://tigerbeetle.com/blog/2022-10-12-a-database-without-dynamic-memory/ https://tigerbeetle.com/blog/2022-10-12-a-database-without-d... It's always bad to use O(N) memory if you don't have to. With a FS-backed database, you don't have to. (Whether you're using static allocation or not. I work on a Ruby web-app, and we avoid loading N records into memory at once, using fixed-sized batches instead.) Doing allocation up front is just a very nice way of ensuring you've thought about those limits, and making sure you don't slip up, and avoiding the runtime cost of allocations. This is totally different from OP's situation, where they're implementing an in-memory database. This means that 1) they've had to impose a limit on the number of kv-pairs they store, and 2) they're paying the cost for all kv-pairs at startup. This is only acceptable if you know you have a fixed upper bound on the number of kv-pairs to store.

levkk 10mo ago
That makes sense. For example, your redis instance will have fixed RAM, so might as well pre-allocate it at boot and avoid fragmentation. Memcached works similarly (slabs of fixed size), except they are not pre-allocated. If you're sharing hardware with multiple services, e.g. web, database, cache, the kind of performance this is targeting isn't a priority.
matklad 10mo ago
Yes, very good point, thanks! As a tiny nit, TigerBeetle isn't _file system_ backed database, we intentionally limit ourselves to a single "file", and can work with a raw block device or partition, without file system involvement.
- 10mo ago

netik 10mo ago

Didn’t we solve this already with slab allocators in memcached? The major problem with fixed allocation like this is fragmentation in memory over time, which you then have to reinvent GC for.

TheMagicHorsey 10mo ago

In a strange coincidence (or maybe its actually inevitable given the timing) I also saw a podcast with Matklad of Tigerbeetle and had a similar idea--I've been working on a massively multiplayer game as a hobby project, also built in zig, also fully allocating all memory at startup, and also had an experience almost identical to OP's. In my case both my client and server are in Zig. Zig is pretty great at doing performant game code (rendering and physics on the client) ... it's less great on the server compared to Go (early days and fewer batteries included, fewer things just work out of the box ... but you can find pretty much everything you need for a game server with a little hunting and pecking and a debugging few build issues). Zig also works "okay" with vibe coding. Golang works much better (maybe a function of the models I use (primarily through Cursor) or maybe its that there's less Zig code out in the wild for models to scrape and learn from, or maybe idiomatic Zig just isn't a thing yet the way it is with Go. Not quite sure.

dnautics 10mo ago
claude is really good at zig. amount of examples on the web is not a good predictor of llm capability, since we know you can poison an llm with ~250 examples. it doesn't take much.

xthe 10mo ago

Nice write-up. Static allocation forcing you to think about limits up front feels like a real design win

ogogmad 10mo ago

This might be a silly thing to point out, but where do people draw the line between an allocation happening or not happening? You still need to track vacant/occupied memory even when there's no OS or other programs around. It's especially bewildering when people claim that some database program "doesn't allocate".

nickmonad 10mo ago
This is the fundamental question which motivated the post. :) I think there are a few different ways to approach the answer, and it kind of depends on what you mean by "draw the line between an allocation happening or not happening." At the surface level, Zig makes this relatively easy, since you can grep for all instances of `std.mem.Allocator` and see where those allocations are occurring throughout the codebase. This only gets you so far though, because some of those Allocator instances could be backed by something like a FixedBufferAllocator, which uses already allocated memory either from the stack or the heap. So the usage of the Allocator instance at the interface level doesn't actually tell you "this is for sure allocating memory from the OS." You have to consider it in the larger context of the system. And yes, we do still need to track vacant/occupied memory, we just do it at the application level. At that level, the OS sees it all as "occupied". For example, in kv, the connection buffer space is marked as vacant/occupied using a memory pool at runtime. But, that pool was allocated from the OS during initialization. As we use the pool we just have to do some very basic bookkeeping using a free-list. That determines if a new connection can actually be accepted or not. Hopefully that helps. Ultimately, we do allocate, it just happens right away during initialization and that allocated space is reused throughout program execution. But, it doesn't have to be nearly as complicated as "reinventing garbage collection" as I've seen some other comments mention.
- deleted 10mo ago
  [deleted]

CashWasabi 10mo ago

This also works well for games. I use a FixedBufferAllocator that allocates everything except assets upfront (systems, entities, etc.). Tigerstyle is a good starting point for efficient and debuggable software.Thanks for the article!

CashWasabi 10mo ago

Great read! I'm doing something similar with my game engine. I use a FixedBufferAllocator for static allocation and initialize/allocate all my systems and entities with the necessary size at the start. The only exception currently is asset loading because this can be quite dynamic at times.