7 ms·

> three issues: "How big is it", "Who owns it", and "Who locks it" The issue with this kind of thinking is the belief that there is a "it", not "they". Say, y

> three issues: "How big is it", "Who owns it", and "Who locks it" The issue with this kind of thinking is the belief that there is a "it", not "they". Say, you are writing a HTTP server [1]. A beginner mindset (what seems to be demonstrated in this post) with start allocating left and right, for every HTTP header, for every piece of string, for every piece of metadata record, etc. Thus, "how big is it" become no bigger than it needs to be. The lifetime of these allocations and deallocations will be made as "narrow" as possible, meaning that the programmer will try to deallocate as soon the they stop needing the data stored; "who owns it" becomes the user of the data itself. And so on. However, think of an alternative strategy, something which game and embedded developers have been using for decades. You allocate a memory arena when the HTTP request comes. From that point on, every "allocation" is equivalent to bumping a pointer in that arena. If the arena gets full, we allocate another and chain it to the previous one, as a linked list. No deallocations are performed (or could possible be performed) until the Request is parsed, relevant processing is done, and Response is constructed and sent. At this point, the entire arena chain associated with that particular request-response cycle is deallocated in one go. How big is it? We don't care, smaller than the arena. Who owns it? That particular request-response cycle. Who locks it? No one, if different request-response are parallelised wrt each other; otherwise, depends on the nature of concurrency/parallelism. [1] Usually, I would have given a gamedev example, and then people would have said that it only works in gamedev. That's why I have tried to give a webdev example, considering the majority demographic on this website. Some reference: 1. Mike Acton's CPPCon Talk: https://m.youtube.com/watch?v=rX0ItVEVjHc https://m.youtube.com/watch?v=rX0ItVEVjHc 2. Rant by Casey Muratori (Brusqueness Allergy Warning): https://www.youtube.com/watch?v=f4ioc8-lDc0&t=4407s https://www.youtube.com/watch?v=f4ioc8-lDc0&t=4407s

samus 5y ago
Another optimization would be to keep the request buffer around and use it as backing storage for all the strings objects that result. The weakness of this strategy is that it doesn't make sense to keep it around if you only need a small part of it. It's a common source of memory leaks in Java where `String.substring()` just creates a new object with different start and end indexes for the same backing buffer. A possible disadvantage of arena allocators is that everything is now tied to the lifetime of the arena, even though it might not be required that long. You'd need to practice a strategy like subarenas, RAAI, garbage collection or reference counting within the request as well. The occurrence and advantages of all these strategies unfortunately are heavily workload-dependent. Most strategies work well for 80% of the workload. Different workloads of course.
notacoward 5y ago
> You allocate a memory arena when the HTTP request comes. The per-request-arena concept is one of the most powerful and IMO underappreciated tools out there for dealing with memory-lifetime issues. I've used it or advocated for its use at multiple jobs and projects, quite successfully, even for kernel code. Windows NT uses it in IRPs. Network code in both BSD and SysV did something pretty similar, at least in the long-before-Linux days. It's tried and true. Having a clear moment when things should be freed and an equally clear description of what those things are is wonderful. That said, even games and network servers and storage drivers often have other complex data structures not related to individual requests. It's usually not hard to come up with rules for each of these, but the rules forced on you by something like STL might not be as convenient, performant, and/or verifiable as those you come up with yourself. I hope that some day programmers will stop using memory-unsafe languages except when they really need to, and that when it is necessary those languages will give them the tools to solve those problems themselves instead of pretending they're already solved. They're not. Rust's borrow checker plus Zig's arena concept plus a couple more pieces might actually get us there. C++'s feature salad never will.
- HexDecOctBin 5y ago
  Maybe I didn't express myself properly (English is not my native language), but the point wasn't that everyone should use Memory Arena™ instead of Smart Pointers™ or Borrow Checker™. I am not trying to sell a conference here :) The point was, to put it in Mike Acton's words, "when there is one, there are many". Usually, things are grouped, perhaps semantically, and it make sense to think of them as a group rather than breaks them in pieces. Arenas are one rather simple grouping mechanism, but they are not the only one. Of course, explaining complex grouping will require explaining the associated problem-solution pair, which is obviously out of scope of a forum reply. All I would say is, if one steps out of one-at-a-time thinking, and statically plans the dynamically happening data transformation, and therefore exploit patterns and relationships inherent in that data flow, programming in low level languages becomes much easier. But you need to know the problem (really really know it), and you need to know the solution. Temet Nosce.
  - deleted 5y ago
tejohnso 5y ago
Wow, Casey doesn't hold back. Clearly, without hesitating says that 100% of code written in RAII* style sucks. Interesting considering Stroustrup considers RAII to be "the basis of some of the most effective modern C++ design techniques." I'd like to hear Casey's comments on TDD. *RAII style is lumped together with try/catch, smart pointers, tons of malloc/free new/delete
- HexDecOctBin 5y ago
  I wonder, has Bjarne Stroustrup even shipped any industrial-grade software? Because he's mostly an academic AFAIK.
  - PaulDavisThe1st 5y ago
    I have no idea what "industrial grade software is", but I ship a cross-platform DAW written in C++, and RAII is tremendously awesome.
    - HexDecOctBin 5y ago
      Are you saying that it is better than the alternatives? Have you tried writing code from a grouped-resources perspective? What was your conclusion?
      - PaulDavisThe1st 5y ago
        I don't see grouped resources as an alternative to the way we use RAII. It doesn't mean that the approach doesn't have its own merits, but it doesn't cover the things we do with RAII. The architecture of a DAW isn't much like a server that responds to requests, even if there are elements of it that do correspond to that pattern. Alternatively, you are referring to something I'm not aware of.
  - ncmncm 5y ago
    Bjarne Stroustrup is a Director at Morgan-Stanley, a major investment bank in New York where software difficulties may cost millions of dollars per minute. He has described his workday as people coming to him with a difficult software engineering problem, about which he asks increasingly detailed questions until light dawns, and they go away ready to re-write the badly designed subsystem causing the trouble.
soedirgo 5y ago
While I agree with the sentiment, my impression is that the GP's point is about memory safety rather than performance. So yes, this applies for common patterns like per-frame memory in games, in which case the "it" is the arena. Otherwise, as a general rule, profile first, then optimize.
- HexDecOctBin 5y ago
  > this applies for common patterns like per-frame memory in games Even though I went to the trouble to give a non-gamedev example... It's not about optimization, it's about resource management, an alternative to GC or RAII.
nightowl_games 5y ago
Game dev here. Try reading the Godot code base. It's tiny allocations all the way through. It's really quite bad. BUT I'm using it professionally and it's useful. It's hard for me to mentally reconcile how good and bad the creator of Godot simulatenously was (is?). Casey also goes way too far in his take (as usual). RAII is a method to guarantee safety and correctness. It's useful in many many scenarios. Game dev and c++ is a great area to hone your performance skills. And it's awesome to really squeeze the power out of a CPU, but doing that is only a small small piece of the game development journey. Ultimately, you must make a product that is entertaining. PS: Mike Acton's talk is really only tangentially related. He's just talking about uber perf in general.
- HexDecOctBin 5y ago
  Well, I'd be surprised if the design/"entertainment" focussed part of the game is bring written in C or C++. Actually, you know what, I would not be surprised, just disappointed :) EDIT: Unless its a small indie studio, of course. In that case, do whatever works for you. Go nuts, have fun!
  - flohofwoe 5y ago
    A fairly common way is to first implement experimental gameplay logic in a scripting- or visual-language, and once it works, extract the performance-critical parts into "proper" C++. Basically, the common lower level gameplay building blocks are written in C++, glued together by some higher level mechanism (like noodle graphs or a scripting language, and common features move into C++ as needed). Unless it's Unity, in that case, replace C++ with C#.
- flohofwoe 5y ago
  As productive and useful as Godot is (it is an awesome engine), its design resembles game engines of the late 90s to early 2010s when OOP was all the rage. Back then, most game engines were written that way (lots of tiny heap-allocated objects, connected by shared pointers). It's really just since the 2010s that the CPU/memory gap is the main driving force of game engine design (which wasn't much of an issue in the late 90's). On the other hand, many games don't need to juggle more than a few dozen to a few hundred dynamic instances of one "thing" (outside of the particle- and animation-systems at least), so for most games a modern ECS design is definitely overkill (except for some specific parts of the game). Providing a good "game building workflow" in the editor is definitely more important.
viraptor 5y ago
Your comment reads like you disagreed in some way with the parent, but then you showed a scenario and defined it in a way you can answer the important questions. Just drop the "we don't care" part and they're all good answers that can be used to model the ownership and usage. The lack of locks in this scenario is important in itself and can be expressed in types. (It shows for example that as long as the whole process is migrated to another thread, you can have safe N:M scheduling) The process ownership of the arena can be well defined as well. The "alternative strategy" as you described it doesn't have different questions - just different answers.
- HexDecOctBin 5y ago
  No, the point wasn't a scenario. The point was to stop focussing on individual allocation-deallocation and start thinking about the data transformation pipeline. And every program is a data transformer — because data transformation is all a computer does or can possibly do. When one starts thinking in terms of data transformation pipeline, and start to relate lifetime of data with the lifetime of various phases of that pipeline, then one stops worrying about individual "objects" and start thinking in terms of aggregates. Suddenly, there is no need to track the ownership or size for each object, because those properties are now shared with the same (or isomorphic) properties of the phases of pipeline itself. As long as the position in the pipeline is tracked for (e.g., the call stack), all other lifetimes will automatically be tracked.
  - viraptor 5y ago
    I get the idea you're describing and use it for various purposes. But I don't agree "there is no need to track the ownership or size for each object". You can offload some of that thinking to the arenas, the same way you'd do it with GC, sure. But you can't just ignore ownership - sure request context is owned by the arena - but do you need to copy the values you pass to logging? do you need to wait for log flush before destroying request context? What about the more common cached data / process state? Arenas simplify the processing just like GC but they're not magic and don't solve everything.
    - 5y ago
klibertp 5y ago
> From that point on, every "allocation" is equivalent to bumping a pointer in that arena. I'm not a C++ developer, so I have a hard time imagining how would this be implemented in real code. I know how to allocate a blob of memory, but how do I redirect all the later allocations (that use `new` or that are hidden in std::string and other containers) to use parts of that blob? How do I know when the blob is going to overflow so that I need to allocate another one? Is it possible to give 5-10 lines example showing the basics of the technique?
- HexDecOctBin 5y ago
  Sure, here is a snippet: https://pastebin.com/XGUeKdyk https://pastebin.com/XGUeKdyk Disclaimer: Quickly typed it on phone, so might make your computer explode.
- kaba0 5y ago
  It’s not an either or thing, there is probably no need to allocate strings in the arena. Some cpp structures do allow for custom allocators, but you are more likely to have an arena instance and call some specific function on it and it will do the allocation for you. It usually operates on only a few types, not meant for arbitrary allocations.
  - klibertp 5y ago
    > Some cpp structures do allow for custom allocators, Yeah, that's what I was thinking about, but my only knowledge of allocators comes from gcc error messages which include all the template parameters - I have no idea how they work or how to switch to another one :( I get the idea, though, it's basically what Erlang does for its processes - each one has a "private heap" just for it. If the process exits, the whole chunk of memory can be reclaimed and there's no need to run GC on it anymore (there's no sharing of memory between processes in Erlang, at all, so it's safe).
- clddd 5y ago
  You either pass the allocator around to every stl container, and everything that uses an stl container. Or you override new/delete and have a sidechannel that defines what arena to allocate in.
bsenftner 5y ago
This is why I'm a strong advocate of hiring former game developers - they know what matters when it matters, not before, and not after. how? Pain and tears of being there, trying in the obvious wrong way first, and then fixing it during production while kiddies shout at you. Former game devs are the Marines of software.
barrkel 5y ago
A meta comment, not really related to your point, but I found it amusing in context. You're writing about how doing lots of little things in an HTTP server is inefficient; and you're replying to John Nagle, who contributed the Nagle TCP optimization that debounces tiny sends with a little bit of delay, in the hope of doing more work in one go! (On topic of your HTTP example: this is where generational GC can shine. Ideally one of your younger generations covers the full allocation cycle for a request & response, so that nothing survives the collection and it's ultra-cheap to collect since tracing the roots doesn't find anything. The nice thing about GC instead of an explicit arena is that it's global, so you don't need to contort all your library calls to ensure they're using the right allocator.)
ape4 5y ago
I made a special purpose http sever in C++. I had a function called request() and just allocated everything in the stack of that function. Only a few KB. eg a vector of the headers etc. Of course after the function was done the memory was freed.
pjmlp 5y ago
The same Mike Acton that now is an Unity employee, being one of the team leads on Burst compiler for HPC#, and not a big C++ fan anyway.
zoomablemind 5y ago
> You allocate a memory arena when the HTTP request comes. From that point on, every "allocation" is equivalent to bumping a pointer in that arena. If the arena gets full, we allocate another and chain it to the previous one, as a linked list. Didn't you just describe the regular memory management as applied by OS in a process scope? At the completion of the process, all memory allocated by the process is reclaimed. On some systems this approach is combined with granular quotas to allow multi-process coexistence. Shifting this sort of memory management over to application scope may yield benefits now, but ultimately it's an OS-level concern.
- samus 5y ago
  Asking the OS for memory is a low-level operation, which means it is not portable and should be done by a library or framework. Also, since it's a syscall (involving a context switch), you don't want to do in performance-sensitive code. Some applications might benefit from allocating huge pages. By default, the page size is something of the order of 4KB or 16KB. Large numbers of pages result in huge overhead. By allocating fewer, large pages on the order of MBs can help.
- thawkins 5y ago
  Interestingly it's exactly what the PHP memory model does, each request is a shared nothing VM space, nothing survives the end of the request, even if allocated memory objects don't get free'd.
pastenpasten 5y ago
What asiachick wrote below. Muratori has demonstrated to me one time too many that he hasn't a got a clue about the things he's talking about. To be precise: He may or may not have a clue about some things he's talking about, but I don't know enough about them to form an opinion. But in more than one occasions he went on to talk (sometimes at great length) about things where I know he's mostly or completely wrong in what he said. I don't have a way to know the scope of these things, the things he doesn't know or understand yet talks about. Maybe it's just when it comes to "systems programming". Maybe it's everything that isn't game design and programming. And maybe he's wrong about games too. As I said, I don't know enough to judge everything he says, but the things I was able to judge convinced me not to trust him and his opinions. And since I can't set the scope safely, I have to not trust him regarding everything. For example, while his post[1][2] on ETW being "the worst API ever made" is slightly amusing and has _some_ good points, it mostly demonstrates: (a) Inability or unwillingness to read the documentation. It would have saved most of his problems. But let's say that to criticize the _design_ of an API we can disregard that for a moment. (b) Inability or unwillingness to pick the right tool for the task or pure trolling. (c) Worst of all: Complete lack of understanding of the goals, purposes, limitation, design goals and tradeoffs of the system. You can't criticize the design as being too complicated, while offering an irrelevant alternative for a "glorified memcpy()" (his words), when you don't understand what it does or what it's supposed to do. (I emphasize again: I'm not saying the API is a paragon API design and that all is perfect there. I am saying the criticism is both grossly exaggerated and demonstrates misunderstanding of what the API actually does.) After a couple of those I don't trust him and you shouldn't either but you're welcome to. [1] https://caseymuratori.com/blog_0025 https://caseymuratori.com/blog_0025 [2] https://news.ycombinator.com/item?id=8146124 https://news.ycombinator.com/item?id=8146124 See the discussion there.
- HexDecOctBin 5y ago
  > You can't criticize the design as being too complicated... I hadn't finished reading the ETW rant; but I don't see anything wrong with it. All he seems to be saying is that for communicating with OS, ioctl-like all-in-one APIs are bad while epoll like APIs are better. Not really a controversial statement. > After a couple of those I don't trust him Who said anything about trust? I have made enough mistakes and learnt enough lessons, and it's nice to see other well-respected programmers (like Casey and Mike) come to the same conclusions like mine. To paraphrase your post: "You shouldn't repeat those mistakes but you're welcome to". EDIT: Actually, after finishing reading the blog post, wow! I thought Linux's perf_event API was shit, but this ETW crap takes the cake. Well done, Microsoft; you had one job.

samus 5y ago

Another optimization would be to keep the request buffer around and use it as backing storage for all the strings objects that result. The weakness of this strategy is that it doesn't make sense to keep it around if you only need a small part of it. It's a common source of memory leaks in Java where `String.substring()` just creates a new object with different start and end indexes for the same backing buffer. A possible disadvantage of arena allocators is that everything is now tied to the lifetime of the arena, even though it might not be required that long. You'd need to practice a strategy like subarenas, RAAI, garbage collection or reference counting within the request as well. The occurrence and advantages of all these strategies unfortunately are heavily workload-dependent. Most strategies work well for 80% of the workload. Different workloads of course.

notacoward 5y ago

> You allocate a memory arena when the HTTP request comes. The per-request-arena concept is one of the most powerful and IMO underappreciated tools out there for dealing with memory-lifetime issues. I've used it or advocated for its use at multiple jobs and projects, quite successfully, even for kernel code. Windows NT uses it in IRPs. Network code in both BSD and SysV did something pretty similar, at least in the long-before-Linux days. It's tried and true. Having a clear moment when things should be freed and an equally clear description of what those things are is wonderful. That said, even games and network servers and storage drivers often have other complex data structures not related to individual requests. It's usually not hard to come up with rules for each of these, but the rules forced on you by something like STL might not be as convenient, performant, and/or verifiable as those you come up with yourself. I hope that some day programmers will stop using memory-unsafe languages except when they really need to, and that when it is necessary those languages will give them the tools to solve those problems themselves instead of pretending they're already solved. They're not. Rust's borrow checker plus Zig's arena concept plus a couple more pieces might actually get us there. C++'s feature salad never will.

HexDecOctBin 5y ago
Maybe I didn't express myself properly (English is not my native language), but the point wasn't that everyone should use Memory Arena™ instead of Smart Pointers™ or Borrow Checker™. I am not trying to sell a conference here :) The point was, to put it in Mike Acton's words, "when there is one, there are many". Usually, things are grouped, perhaps semantically, and it make sense to think of them as a group rather than breaks them in pieces. Arenas are one rather simple grouping mechanism, but they are not the only one. Of course, explaining complex grouping will require explaining the associated problem-solution pair, which is obviously out of scope of a forum reply. All I would say is, if one steps out of one-at-a-time thinking, and statically plans the dynamically happening data transformation, and therefore exploit patterns and relationships inherent in that data flow, programming in low level languages becomes much easier. But you need to know the problem (really really know it), and you need to know the solution. Temet Nosce.
- deleted 5y ago

tejohnso 5y ago

Wow, Casey doesn't hold back. Clearly, without hesitating says that 100% of code written in RAII* style sucks. Interesting considering Stroustrup considers RAII to be "the basis of some of the most effective modern C++ design techniques." I'd like to hear Casey's comments on TDD. *RAII style is lumped together with try/catch, smart pointers, tons of malloc/free new/delete

HexDecOctBin 5y ago
I wonder, has Bjarne Stroustrup even shipped any industrial-grade software? Because he's mostly an academic AFAIK.
- PaulDavisThe1st 5y ago
  I have no idea what "industrial grade software is", but I ship a cross-platform DAW written in C++, and RAII is tremendously awesome.
  - HexDecOctBin 5y ago
    Are you saying that it is better than the alternatives? Have you tried writing code from a grouped-resources perspective? What was your conclusion?
    - PaulDavisThe1st 5y ago
      I don't see grouped resources as an alternative to the way we use RAII. It doesn't mean that the approach doesn't have its own merits, but it doesn't cover the things we do with RAII. The architecture of a DAW isn't much like a server that responds to requests, even if there are elements of it that do correspond to that pattern. Alternatively, you are referring to something I'm not aware of.
- ncmncm 5y ago
  Bjarne Stroustrup is a Director at Morgan-Stanley, a major investment bank in New York where software difficulties may cost millions of dollars per minute. He has described his workday as people coming to him with a difficult software engineering problem, about which he asks increasingly detailed questions until light dawns, and they go away ready to re-write the badly designed subsystem causing the trouble.

soedirgo 5y ago

While I agree with the sentiment, my impression is that the GP's point is about memory safety rather than performance. So yes, this applies for common patterns like per-frame memory in games, in which case the "it" is the arena. Otherwise, as a general rule, profile first, then optimize.

HexDecOctBin 5y ago
> this applies for common patterns like per-frame memory in games Even though I went to the trouble to give a non-gamedev example... It's not about optimization, it's about resource management, an alternative to GC or RAII.

nightowl_games 5y ago

Game dev here. Try reading the Godot code base. It's tiny allocations all the way through. It's really quite bad. BUT I'm using it professionally and it's useful. It's hard for me to mentally reconcile how good and bad the creator of Godot simulatenously was (is?). Casey also goes way too far in his take (as usual). RAII is a method to guarantee safety and correctness. It's useful in many many scenarios. Game dev and c++ is a great area to hone your performance skills. And it's awesome to really squeeze the power out of a CPU, but doing that is only a small small piece of the game development journey. Ultimately, you must make a product that is entertaining. PS: Mike Acton's talk is really only tangentially related. He's just talking about uber perf in general.

HexDecOctBin 5y ago
Well, I'd be surprised if the design/"entertainment" focussed part of the game is bring written in C or C++. Actually, you know what, I would not be surprised, just disappointed :) EDIT: Unless its a small indie studio, of course. In that case, do whatever works for you. Go nuts, have fun!
- flohofwoe 5y ago
  A fairly common way is to first implement experimental gameplay logic in a scripting- or visual-language, and once it works, extract the performance-critical parts into "proper" C++. Basically, the common lower level gameplay building blocks are written in C++, glued together by some higher level mechanism (like noodle graphs or a scripting language, and common features move into C++ as needed). Unless it's Unity, in that case, replace C++ with C#.
flohofwoe 5y ago
As productive and useful as Godot is (it is an awesome engine), its design resembles game engines of the late 90s to early 2010s when OOP was all the rage. Back then, most game engines were written that way (lots of tiny heap-allocated objects, connected by shared pointers). It's really just since the 2010s that the CPU/memory gap is the main driving force of game engine design (which wasn't much of an issue in the late 90's). On the other hand, many games don't need to juggle more than a few dozen to a few hundred dynamic instances of one "thing" (outside of the particle- and animation-systems at least), so for most games a modern ECS design is definitely overkill (except for some specific parts of the game). Providing a good "game building workflow" in the editor is definitely more important.

viraptor 5y ago

Your comment reads like you disagreed in some way with the parent, but then you showed a scenario and defined it in a way you can answer the important questions. Just drop the "we don't care" part and they're all good answers that can be used to model the ownership and usage. The lack of locks in this scenario is important in itself and can be expressed in types. (It shows for example that as long as the whole process is migrated to another thread, you can have safe N:M scheduling) The process ownership of the arena can be well defined as well. The "alternative strategy" as you described it doesn't have different questions - just different answers.

HexDecOctBin 5y ago
No, the point wasn't a scenario. The point was to stop focussing on individual allocation-deallocation and start thinking about the data transformation pipeline. And every program is a data transformer — because data transformation is all a computer does or can possibly do. When one starts thinking in terms of data transformation pipeline, and start to relate lifetime of data with the lifetime of various phases of that pipeline, then one stops worrying about individual "objects" and start thinking in terms of aggregates. Suddenly, there is no need to track the ownership or size for each object, because those properties are now shared with the same (or isomorphic) properties of the phases of pipeline itself. As long as the position in the pipeline is tracked for (e.g., the call stack), all other lifetimes will automatically be tracked.
- viraptor 5y ago
  I get the idea you're describing and use it for various purposes. But I don't agree "there is no need to track the ownership or size for each object". You can offload some of that thinking to the arenas, the same way you'd do it with GC, sure. But you can't just ignore ownership - sure request context is owned by the arena - but do you need to copy the values you pass to logging? do you need to wait for log flush before destroying request context? What about the more common cached data / process state? Arenas simplify the processing just like GC but they're not magic and don't solve everything.
  - 5y ago

klibertp 5y ago

> From that point on, every "allocation" is equivalent to bumping a pointer in that arena. I'm not a C++ developer, so I have a hard time imagining how would this be implemented in real code. I know how to allocate a blob of memory, but how do I redirect all the later allocations (that use `new` or that are hidden in std::string and other containers) to use parts of that blob? How do I know when the blob is going to overflow so that I need to allocate another one? Is it possible to give 5-10 lines example showing the basics of the technique?

HexDecOctBin 5y ago
Sure, here is a snippet: https://pastebin.com/XGUeKdyk https://pastebin.com/XGUeKdyk Disclaimer: Quickly typed it on phone, so might make your computer explode.
kaba0 5y ago
It’s not an either or thing, there is probably no need to allocate strings in the arena. Some cpp structures do allow for custom allocators, but you are more likely to have an arena instance and call some specific function on it and it will do the allocation for you. It usually operates on only a few types, not meant for arbitrary allocations.
- klibertp 5y ago
  > Some cpp structures do allow for custom allocators, Yeah, that's what I was thinking about, but my only knowledge of allocators comes from gcc error messages which include all the template parameters - I have no idea how they work or how to switch to another one :( I get the idea, though, it's basically what Erlang does for its processes - each one has a "private heap" just for it. If the process exits, the whole chunk of memory can be reclaimed and there's no need to run GC on it anymore (there's no sharing of memory between processes in Erlang, at all, so it's safe).
clddd 5y ago
You either pass the allocator around to every stl container, and everything that uses an stl container. Or you override new/delete and have a sidechannel that defines what arena to allocate in.

bsenftner 5y ago

This is why I'm a strong advocate of hiring former game developers - they know what matters when it matters, not before, and not after. how? Pain and tears of being there, trying in the obvious wrong way first, and then fixing it during production while kiddies shout at you. Former game devs are the Marines of software.

barrkel 5y ago

A meta comment, not really related to your point, but I found it amusing in context. You're writing about how doing lots of little things in an HTTP server is inefficient; and you're replying to John Nagle, who contributed the Nagle TCP optimization that debounces tiny sends with a little bit of delay, in the hope of doing more work in one go! (On topic of your HTTP example: this is where generational GC can shine. Ideally one of your younger generations covers the full allocation cycle for a request & response, so that nothing survives the collection and it's ultra-cheap to collect since tracing the roots doesn't find anything. The nice thing about GC instead of an explicit arena is that it's global, so you don't need to contort all your library calls to ensure they're using the right allocator.)

ape4 5y ago

I made a special purpose http sever in C++. I had a function called request() and just allocated everything in the stack of that function. Only a few KB. eg a vector of the headers etc. Of course after the function was done the memory was freed.

pjmlp 5y ago

The same Mike Acton that now is an Unity employee, being one of the team leads on Burst compiler for HPC#, and not a big C++ fan anyway.

zoomablemind 5y ago

> You allocate a memory arena when the HTTP request comes. From that point on, every "allocation" is equivalent to bumping a pointer in that arena. If the arena gets full, we allocate another and chain it to the previous one, as a linked list. Didn't you just describe the regular memory management as applied by OS in a process scope? At the completion of the process, all memory allocated by the process is reclaimed. On some systems this approach is combined with granular quotas to allow multi-process coexistence. Shifting this sort of memory management over to application scope may yield benefits now, but ultimately it's an OS-level concern.

samus 5y ago
Asking the OS for memory is a low-level operation, which means it is not portable and should be done by a library or framework. Also, since it's a syscall (involving a context switch), you don't want to do in performance-sensitive code. Some applications might benefit from allocating huge pages. By default, the page size is something of the order of 4KB or 16KB. Large numbers of pages result in huge overhead. By allocating fewer, large pages on the order of MBs can help.
thawkins 5y ago
Interestingly it's exactly what the PHP memory model does, each request is a shared nothing VM space, nothing survives the end of the request, even if allocated memory objects don't get free'd.

pastenpasten 5y ago

What asiachick wrote below. Muratori has demonstrated to me one time too many that he hasn't a got a clue about the things he's talking about. To be precise: He may or may not have a clue about some things he's talking about, but I don't know enough about them to form an opinion. But in more than one occasions he went on to talk (sometimes at great length) about things where I know he's mostly or completely wrong in what he said. I don't have a way to know the scope of these things, the things he doesn't know or understand yet talks about. Maybe it's just when it comes to "systems programming". Maybe it's everything that isn't game design and programming. And maybe he's wrong about games too. As I said, I don't know enough to judge everything he says, but the things I was able to judge convinced me not to trust him and his opinions. And since I can't set the scope safely, I have to not trust him regarding everything. For example, while his post[1][2] on ETW being "the worst API ever made" is slightly amusing and has _some_ good points, it mostly demonstrates: (a) Inability or unwillingness to read the documentation. It would have saved most of his problems. But let's say that to criticize the _design_ of an API we can disregard that for a moment. (b) Inability or unwillingness to pick the right tool for the task or pure trolling. (c) Worst of all: Complete lack of understanding of the goals, purposes, limitation, design goals and tradeoffs of the system. You can't criticize the design as being too complicated, while offering an irrelevant alternative for a "glorified memcpy()" (his words), when you don't understand what it does or what it's supposed to do. (I emphasize again: I'm not saying the API is a paragon API design and that all is perfect there. I am saying the criticism is both grossly exaggerated and demonstrates misunderstanding of what the API actually does.) After a couple of those I don't trust him and you shouldn't either but you're welcome to. [1] https://caseymuratori.com/blog_0025 https://caseymuratori.com/blog_0025 [2] https://news.ycombinator.com/item?id=8146124 https://news.ycombinator.com/item?id=8146124 See the discussion there.

HexDecOctBin 5y ago
> You can't criticize the design as being too complicated... I hadn't finished reading the ETW rant; but I don't see anything wrong with it. All he seems to be saying is that for communicating with OS, ioctl-like all-in-one APIs are bad while epoll like APIs are better. Not really a controversial statement. > After a couple of those I don't trust him Who said anything about trust? I have made enough mistakes and learnt enough lessons, and it's nice to see other well-respected programmers (like Casey and Mike) come to the same conclusions like mine. To paraphrase your post: "You shouldn't repeat those mistakes but you're welcome to". EDIT: Actually, after finishing reading the blog post, wow! I thought Linux's perf_event API was shit, but this ETW crap takes the cake. Well done, Microsoft; you had one job.