3 ms·

Oh hello, thanks for the clarification! Having enjoyed writing some embedded Rust, I'm familiar with the core/alloc/std split. IIUC you're saying that the user-

Oh hello, thanks for the clarification! Having enjoyed writing some embedded Rust, I'm familiar with the core/alloc/std split. IIUC you're saying that the user-space Linux malloc API itself does not provide a reliable way for the application to think about hard memory limits? Which would fuel my pet theory about "infinite virtual memory" being a significant factor in the ever growing software bloat.

steveklabnik 2y ago
> I'm familiar with the core/alloc/std split. Ah, okay. So yeah, it's not a new standard library, it's "things like Vec are adding .push_within_capacity() that's like push except it returns a Result and errors instead of reallocating" more than "bespoke standard library." > IIUC you're saying that the user-space Linux malloc API itself does not provide a reliable way for the application to think about hard memory limits? It's not the user-space malloc API, it's lower than that. See " /proc/sys/vm/overcommit_memory" in https://man7.org/linux/man-pages/man5/proc_sys_vm.5.html https://man7.org/linux/man-pages/man5/proc_sys_vm.5.html The default is "heuristic overcommit." This page does a better job of explaining what that means: https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou... So, unless you've specifically configured this to 2, there are many circumstances where you simply will not get an error from the kernel, even if you've requested more memory than available. What happens in this case is that your program will continue to run. At some point, it will access the bad allocation. The kernel will notice that there's not actually enough memory, and the "oom killer" will decide to kill a process to make space. It might be your process! It also might not be. Just depends. But this happens later, and asynchronously from your program. You cannot handle this error from inside your program. So even if these APIs existed, they wouldn't change the behavior: they would faithfully report what the kernel reported to them: that the allocation succeeded.

steveklabnik 2y ago

> I'm familiar with the core/alloc/std split. Ah, okay. So yeah, it's not a new standard library, it's "things like Vec are adding .push_within_capacity() that's like push except it returns a Result and errors instead of reallocating" more than "bespoke standard library." > IIUC you're saying that the user-space Linux malloc API itself does not provide a reliable way for the application to think about hard memory limits? It's not the user-space malloc API, it's lower than that. See " /proc/sys/vm/overcommit_memory" in https://man7.org/linux/man-pages/man5/proc_sys_vm.5.html https://man7.org/linux/man-pages/man5/proc_sys_vm.5.html The default is "heuristic overcommit." This page does a better job of explaining what that means: https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou... So, unless you've specifically configured this to 2, there are many circumstances where you simply will not get an error from the kernel, even if you've requested more memory than available. What happens in this case is that your program will continue to run. At some point, it will access the bad allocation. The kernel will notice that there's not actually enough memory, and the "oom killer" will decide to kill a process to make space. It might be your process! It also might not be. Just depends. But this happens later, and asynchronously from your program. You cannot handle this error from inside your program. So even if these APIs existed, they wouldn't change the behavior: they would faithfully report what the kernel reported to them: that the allocation succeeded.