4 ms·

Well, the issue really is that applications are not being designed correctly. To create reliable applications you need to design them to work within limits of

Well, the issue really is that applications are not being designed correctly. To create reliable applications you need to design them to work within limits of memory allocated to them. Unattended applications (like RDBMS or application container) should not allocate memory dynamically based on user input. By definition, reliable application will not fail because of external input. Allocating memory from OS cannot be done reliably unless that memory was set aside somehow. If the memory was not set aside then we say we are in overprovisioning situation and this means we accept that it might happen that a process wanting to allocate memory will fail to get it. So the solution (the simplest but not the only one) is to allocate the memory ahead of time for the maximum load the app can experience and to make a limit on the load (number of concurrent connections, etc.) so that this limit is never exceeded. Some systems do it halfway decently. For example Oracle will explicitly allocate all its memory spaces and then work within those. The OS really is a big heap of compromises. It does not know anything about the processes it runs but it is somehow expected to do the right thing. We see it is not behaving gracefully when memory runs out but the truth is, OS is built for the situation where it is being used mostly correctly. If memory is running out it means the user already made a mistake in desiging their use of resources (or didn't think about it at all) and there is not really much the OS can do to help it.

lclarkmichalek 7y ago
> Unattended applications (like RDBMS or application container) should not allocate memory dynamically based on user input. This is incredibly easy to say, and incredibly hard to do for any application of even moderate complexity. Getting people to accurately tune thread counts or concurrency limits is painful when working with diverse workloads, and getting a guaranteed upper bound on memory usage would be a lot harder. It's much more common to apply higher level techniques such as concurrency limits and load shedding to avoid OOMs and other resource starvation issues. The other thing we can do is reduce the impact of OOMs, and accept that they exist as a tail event. There is software running on your system that doesn't have hard memory limits, and you will always have to consider OOMs and other similar failure cases when ensuring your system is resilient. As long as we can prevent those tail events becoming a correlated failure across our fleet, we can get a pretty reliable system.
- lmilcin 7y ago
  > This is incredibly easy to say, and incredibly hard to do for any application of even moderate complexity. I agree. But this is what it takes to build really reliable applications. For example, MISRA rules (C and C++ standards for automotive applications) forbid dynamic memory allocation completely after the application started. It also, if I remember well, forbids anything that could make it impossible to statically calculate stack requirements like recurrency. This is to make sure it is possible to calculate memory requirements statically.
  - noselasd 7y ago
    Sure, but not all applications can favorly be designed like that. It makes little sense for Excel to pre-allocate 4GB of memory to account for the user wanting to create a gigantic analysis in their spreadsheet over the next weeks, if all that the user wanted was to make a small sum of 100 narrow rows.
    - deleted 7y ago
      [deleted]
    - yason 7y ago
      There is no problem for Excel to allocate memory dynamically for rows and columns as the user edits the sheet as long as it will also handle an allocation failure gracefully. That generally means the user input events call functions that fail and become no-ops until memory can be allocated again. That's how we used to write software until Linux ended up running everywhere, and that was the only way to write software for platforms without a MMU. As recently as a decade ago mobile phones, before Android, often had strict memory management. Each allocation could fail, and these failure paths had to be tested. Explicitly committing after a fork() would just make fork() cost a bit more but it would work. The fork() call would fail if there's no memory to duplicate all anonymous pages for the child process. But once it would succeed the child process would know it won't cause an OOM merely by writing into a memory location.
      - 7y ago
Nursie 7y ago
I worked briefly on the firmware for a range of enterprise storage products that had the same sort of constraints. To ensure total knowledge of memory use by the system at all times, dynamic allocation was not allowed. All buffers, queues, everything that could be wanted were allocated at the beginning of time. There was no possibility of running out of memory. There was the possibility of turning down an IO request if the system got flooded, but that wasn't going to be because a malloc failed, it was going to be because a known condition had been reached and no more requests could be accepted until the system cleared itself down a little. Computer resources were managed similarly - the most important and most CPU-heavy resources would be tied to a core and given exclusive use of that core. The only caveat here is that doing this all takes engineering effort and fairly deep knowledge of your OS. If you don't have mission-critical applications like this to write it's probably overkill. (One kinda-cool side effect of the memory all being pre-allocated was that you could just browse the memory space of the process, everything was at a predictable address even without debug symbols.)
- lmilcin 7y ago
  I worked on similar applications in my past. My understanding this requires "arcane knowledge" is because most frameworks and developers simply don't care. All new languages and frameworks are built to supposedly make things easy and fun and counting your objects or calculating your buffers isn't exactly fun. Making things dynamic and hiding underlying memory layout is seen as a tool to write software faster but it disconnects developers from understanding what actually happens and makes it even more difficult to write reliable software. If writing software the correct way was prevalent this would not be arcane knowledge but common sense.
  - diroussel 7y ago
    There is possibility that you could write software at a high level and the compiler and tool chain could compute all the static allocation at build time. It could even save the plan as an output to be reused. You could run an optimiser yo optiate cache and memory usage. If a human can do it, then an advance tool chain could do it. Probably too expensive for most use cases.
imtringued 7y ago
The downside to this is that if your application ever needs less than the configured amount of memory then that memory cannot be used by other applications. This is a big reason why java is such a huge memory hog even if the application itself isn't demanding nearly as much memory.
- deleted 7y ago
  [deleted]
- majewsky 7y ago
  I believe that Linux has all the tools to solve this in practice: 1. When you allocate memory, but do not write into it yet, those pages not mapped in RAM and thus don't occupy actual space. 2. When you're done with a specific page of memory, you can madvise(MADV_FREE) it, which means that the kernel can discard these pages from RAM and use it for caches and buffers. But you still hold on to the virtual memory allocation, so you can just start writing into the page again when you need more memory and the kernel will map it again. If I understand all that correctly, you can have your allocator work in such a way that it keeps a large reserve of preallocated memory pages, but the corresponding amount of RAM can be used for caches and buffers when it doesn't need everything. An interesting question would be how that scenario appears in ps(1) and top(1), i.e. whether those MADV_FREE'd pages would count towards RSS.
  - AstralStorm 7y ago
    And 3. When kernel cannot supply you with the allocated page what happens? (Mapped but inaccessible.) Currently, OOM killing.
dbmem 7y ago
The problem with that is that you're pushing the resource management problem onto the administrator (who now has to statically partition their system's resources) and the database (which has to reimplement a lot of the resource management that the OS would do, like I/O management and I/O caching). There are many reasons why this is not a good trade-off for the vast majority of use cases. Unless you are running something like a traditional Oracle deployment where you're just giving over the entire host to the database anyway and the database is enough of a cash cow that they can afford to re-implement much of the functionality of an operating system. It generally makes a lot more sense to let the OS manage the buffer pool using the available free memory on the system (this is what Postgres does, for example). It also makes more sense for the vast majority of deployments to allow process memory consumption to be dynamic and deploy the appropriate limits and monitoring to keep things healthy - e.g. by terminating malfunctioning processes and reconfiguring and migrating applications as needed. This doesn't meet some traditional ideals about how programs should be written, but actually seems to result in better systems in practice.
- dbmem 7y ago
  I also have the heretical belief that for the vast majority of applications, the best way to handle malloc() returning NULL is to abort the process. The main reason is that the error handling code has a high probability of being buggy because allocating memory is pervasive in most code and it's unlikely that you're going to have tests that provide adequate coverage of all the interesting code paths. The consequences of incorrect behaviour are often worse than the consequences of a process crash.

lclarkmichalek 7y ago

> Unattended applications (like RDBMS or application container) should not allocate memory dynamically based on user input. This is incredibly easy to say, and incredibly hard to do for any application of even moderate complexity. Getting people to accurately tune thread counts or concurrency limits is painful when working with diverse workloads, and getting a guaranteed upper bound on memory usage would be a lot harder. It's much more common to apply higher level techniques such as concurrency limits and load shedding to avoid OOMs and other resource starvation issues. The other thing we can do is reduce the impact of OOMs, and accept that they exist as a tail event. There is software running on your system that doesn't have hard memory limits, and you will always have to consider OOMs and other similar failure cases when ensuring your system is resilient. As long as we can prevent those tail events becoming a correlated failure across our fleet, we can get a pretty reliable system.

lmilcin 7y ago
> This is incredibly easy to say, and incredibly hard to do for any application of even moderate complexity. I agree. But this is what it takes to build really reliable applications. For example, MISRA rules (C and C++ standards for automotive applications) forbid dynamic memory allocation completely after the application started. It also, if I remember well, forbids anything that could make it impossible to statically calculate stack requirements like recurrency. This is to make sure it is possible to calculate memory requirements statically.
- noselasd 7y ago
  Sure, but not all applications can favorly be designed like that. It makes little sense for Excel to pre-allocate 4GB of memory to account for the user wanting to create a gigantic analysis in their spreadsheet over the next weeks, if all that the user wanted was to make a small sum of 100 narrow rows.
  - deleted 7y ago
    [deleted]
  - yason 7y ago
    There is no problem for Excel to allocate memory dynamically for rows and columns as the user edits the sheet as long as it will also handle an allocation failure gracefully. That generally means the user input events call functions that fail and become no-ops until memory can be allocated again. That's how we used to write software until Linux ended up running everywhere, and that was the only way to write software for platforms without a MMU. As recently as a decade ago mobile phones, before Android, often had strict memory management. Each allocation could fail, and these failure paths had to be tested. Explicitly committing after a fork() would just make fork() cost a bit more but it would work. The fork() call would fail if there's no memory to duplicate all anonymous pages for the child process. But once it would succeed the child process would know it won't cause an OOM merely by writing into a memory location.
    - 7y ago

Nursie 7y ago

I worked briefly on the firmware for a range of enterprise storage products that had the same sort of constraints. To ensure total knowledge of memory use by the system at all times, dynamic allocation was not allowed. All buffers, queues, everything that could be wanted were allocated at the beginning of time. There was no possibility of running out of memory. There was the possibility of turning down an IO request if the system got flooded, but that wasn't going to be because a malloc failed, it was going to be because a known condition had been reached and no more requests could be accepted until the system cleared itself down a little. Computer resources were managed similarly - the most important and most CPU-heavy resources would be tied to a core and given exclusive use of that core. The only caveat here is that doing this all takes engineering effort and fairly deep knowledge of your OS. If you don't have mission-critical applications like this to write it's probably overkill. (One kinda-cool side effect of the memory all being pre-allocated was that you could just browse the memory space of the process, everything was at a predictable address even without debug symbols.)

lmilcin 7y ago
I worked on similar applications in my past. My understanding this requires "arcane knowledge" is because most frameworks and developers simply don't care. All new languages and frameworks are built to supposedly make things easy and fun and counting your objects or calculating your buffers isn't exactly fun. Making things dynamic and hiding underlying memory layout is seen as a tool to write software faster but it disconnects developers from understanding what actually happens and makes it even more difficult to write reliable software. If writing software the correct way was prevalent this would not be arcane knowledge but common sense.
- diroussel 7y ago
  There is possibility that you could write software at a high level and the compiler and tool chain could compute all the static allocation at build time. It could even save the plan as an output to be reused. You could run an optimiser yo optiate cache and memory usage. If a human can do it, then an advance tool chain could do it. Probably too expensive for most use cases.

imtringued 7y ago

The downside to this is that if your application ever needs less than the configured amount of memory then that memory cannot be used by other applications. This is a big reason why java is such a huge memory hog even if the application itself isn't demanding nearly as much memory.

deleted 7y ago
[deleted]
majewsky 7y ago
I believe that Linux has all the tools to solve this in practice: 1. When you allocate memory, but do not write into it yet, those pages not mapped in RAM and thus don't occupy actual space. 2. When you're done with a specific page of memory, you can madvise(MADV_FREE) it, which means that the kernel can discard these pages from RAM and use it for caches and buffers. But you still hold on to the virtual memory allocation, so you can just start writing into the page again when you need more memory and the kernel will map it again. If I understand all that correctly, you can have your allocator work in such a way that it keeps a large reserve of preallocated memory pages, but the corresponding amount of RAM can be used for caches and buffers when it doesn't need everything. An interesting question would be how that scenario appears in ps(1) and top(1), i.e. whether those MADV_FREE'd pages would count towards RSS.
- AstralStorm 7y ago
  And 3. When kernel cannot supply you with the allocated page what happens? (Mapped but inaccessible.) Currently, OOM killing.

dbmem 7y ago

The problem with that is that you're pushing the resource management problem onto the administrator (who now has to statically partition their system's resources) and the database (which has to reimplement a lot of the resource management that the OS would do, like I/O management and I/O caching). There are many reasons why this is not a good trade-off for the vast majority of use cases. Unless you are running something like a traditional Oracle deployment where you're just giving over the entire host to the database anyway and the database is enough of a cash cow that they can afford to re-implement much of the functionality of an operating system. It generally makes a lot more sense to let the OS manage the buffer pool using the available free memory on the system (this is what Postgres does, for example). It also makes more sense for the vast majority of deployments to allow process memory consumption to be dynamic and deploy the appropriate limits and monitoring to keep things healthy - e.g. by terminating malfunctioning processes and reconfiguring and migrating applications as needed. This doesn't meet some traditional ideals about how programs should be written, but actually seems to result in better systems in practice.

dbmem 7y ago
I also have the heretical belief that for the vast majority of applications, the best way to handle malloc() returning NULL is to abort the process. The main reason is that the error handling code has a high probability of being buggy because allocating memory is pervasive in most code and it's unlikely that you're going to have tests that provide adequate coverage of all the interesting code paths. The consequences of incorrect behaviour are often worse than the consequences of a process crash.