4 ms·

Allocating all required memory during startup has been idiomatic for database architectures as long as I can remember. The obvious benefits to doing things this

Allocating all required memory during startup has been idiomatic for database architectures as long as I can remember. The obvious benefits to doing things this way are wide-ranging. There is a less obvious benefit in more schedule-driven database architectures. By implication, the code knows exactly how much of which resources are instantaneously available at all times since it is directly managing those resources. This enables the database to dynamically adapt its behavior and scheduling decisions in response to specific resource pressures or excess resource capacity. This is basically a fine-grained form of internal micro-backpressure, dynamically prioritizing subsets of the workload that minimize impact on the resources under pressure at any moment in time. In some versions, you can switch internal execution strategies to take advantage of currently under-utilized resources. If you outsource this to the OS, you have no idea how close or far you are from the limits, it doesn't really work. This doesn't add much performance per se but is great for minimizing tail latencies. It helps to create a robust internal resource equilibrium that handles transients and concurrency more smoothly and consistently than if the database couldn't make a fine-grained decisions about what to do from a position of total resource awareness.

dilyevsky 4y ago
Not just databases but really any kind of latency sensitive software especially with “realtime” requirements (e.g telecom gear)
- josephg 4y ago
  Yep. After algorithmic improvements, tracking (and limiting) the use of malloc in a program can improve performance by an order of magnitude or more. Not only is allocating slow, but having data spread all over main memory is terrible for the CPU’s cache and branch predictor. I’ve gotten a lot of massive performance wins by rewriting code in rust because I can control memory allocations more closely. Some people I’ve talked to about this seem to just hear “rust = fast”. I’ve seen expensive rust rewrites which somehow end up running slower than the equivalent Javascript code. Turns out if you write rust like you write javascript, with boxes, vecs and .clone()s everywhere it’s going to be slow.
com2kid 4y ago
> Allocating all required memory during startup has been idiomatic for database architectures as long as I can remember. The obvious benefits to doing things this way are wide-ranging. Allocating all required memory is, IMHO, a great practice for almost any type of program. Heap memory allocators are a type of performance optimization, they allow re-use of existing memory, but only if you are careful to not run out of memory. Heap's of course also allow for more efficient utilization of memory, if some software module isn't using that bit of RAM right now, let another bit of code use it. But like all performance optimizations, they make code messier, and they also lead to less reliable and harder to follow code. In the embedded world, code statically allocates all memory it needs up front. This means that code that reads from the external world has a fixed buffer size, and also likely a related max throughput, regardless of system conditions (e.g. lots of free memory laying around). But, this is also a good thing! It forces programmers to think about their limits up front, and it also forces thinking about error handling when those limits are broken! Sure it can require creative coding, loading large blobs of binary data requires more work when you stop assuming malloc can return infinite memory, but, hot take here, programs should stop assuming malloc can return infinite memory anyway.
- _vvhw 4y ago
  Static allocation has also made TigerBeetle's code cleaner, by eliminating branching at call sites where before a message might not always have been available. With static allocation, there's no branch because a message is always guaranteed to be available. It's also made TigerBeetle's code more reliable, because tests can assert that limits are never exceeded. This has detected rare leaks that might otherwise have only been detected in production.
  - Twisol 4y ago
    I think the grandparent was saying that dynamic allocation is a form of optimization, which also makes the code harder to follow. Your anecdote seems exactly in line with their suggestion.
    - _vvhw 4y ago
parhamn 4y ago
> memory during startup has been idiomatic for database architectures as long as I can remember. How can this be true given most mainstream databases do not do this? Maybe a pattern sure, by idiomatic, not so much. I only add this comment in defense of the poster as this comment seemed to reduce their achievement/efforts.

dilyevsky 4y ago

Not just databases but really any kind of latency sensitive software especially with “realtime” requirements (e.g telecom gear)

josephg 4y ago
Yep. After algorithmic improvements, tracking (and limiting) the use of malloc in a program can improve performance by an order of magnitude or more. Not only is allocating slow, but having data spread all over main memory is terrible for the CPU’s cache and branch predictor. I’ve gotten a lot of massive performance wins by rewriting code in rust because I can control memory allocations more closely. Some people I’ve talked to about this seem to just hear “rust = fast”. I’ve seen expensive rust rewrites which somehow end up running slower than the equivalent Javascript code. Turns out if you write rust like you write javascript, with boxes, vecs and .clone()s everywhere it’s going to be slow.

com2kid 4y ago

> Allocating all required memory during startup has been idiomatic for database architectures as long as I can remember. The obvious benefits to doing things this way are wide-ranging. Allocating all required memory is, IMHO, a great practice for almost any type of program. Heap memory allocators are a type of performance optimization, they allow re-use of existing memory, but only if you are careful to not run out of memory. Heap's of course also allow for more efficient utilization of memory, if some software module isn't using that bit of RAM right now, let another bit of code use it. But like all performance optimizations, they make code messier, and they also lead to less reliable and harder to follow code. In the embedded world, code statically allocates all memory it needs up front. This means that code that reads from the external world has a fixed buffer size, and also likely a related max throughput, regardless of system conditions (e.g. lots of free memory laying around). But, this is also a good thing! It forces programmers to think about their limits up front, and it also forces thinking about error handling when those limits are broken! Sure it can require creative coding, loading large blobs of binary data requires more work when you stop assuming malloc can return infinite memory, but, hot take here, programs should stop assuming malloc can return infinite memory anyway.

_vvhw 4y ago
Static allocation has also made TigerBeetle's code cleaner, by eliminating branching at call sites where before a message might not always have been available. With static allocation, there's no branch because a message is always guaranteed to be available. It's also made TigerBeetle's code more reliable, because tests can assert that limits are never exceeded. This has detected rare leaks that might otherwise have only been detected in production.
- Twisol 4y ago
  I think the grandparent was saying that dynamic allocation is a form of optimization, which also makes the code harder to follow. Your anecdote seems exactly in line with their suggestion.
  - _vvhw 4y ago

parhamn 4y ago

> memory during startup has been idiomatic for database architectures as long as I can remember. How can this be true given most mainstream databases do not do this? Maybe a pattern sure, by idiomatic, not so much. I only add this comment in defense of the poster as this comment seemed to reduce their achievement/efforts.