4 ms·
Depends how long running the process needs to be, and how allocation heavy it gets. My first large web app was old enough that we did it in C++ as CGI... One
by vidarh 9y ago
Depends how long running the process needs to be, and how allocation heavy it gets.
My first large web app was old enough that we did it in C++ as CGI...
One of the first performance optimizations we did was accept that most object lifetimes roughly coincided with the end of the request, so we simply didn't deallocate anything and let the OS just free the whole thing at once. You can do the same in most GC'd languages by just tuning GC limits.
(the second performance optimization we did was to to statically link everything; loading a static binary that's constantly in the cache is/was very fast; and it was enough to make CGI surprisingly competitive in terms of performance)
Preallocation certainly has its place too, but in this case we tried custom allocators and it simply didn't make much difference. So before rewriting your code, see if you can simply turn off deallocations/GC without too much memory pressure.
- CyberDildonics 9y agoIn your case you likely could have made your stack large and used that in some areas while making the data segment of your binary large and using that for other areas. Since the executable is memory mapped anyway the OS would allocate memory in large chunks without the overhead of trying to organize a heap.
- vidarh 9y agoDoing just straight allocations in C++ without de-allocations has pretty much same effect. It'll ask for memory from the OS in large chunks, and without anything ever being de-allocated, unless your allocator is too smart for its own good, allocation can be extremely cheap. I did actually try a custom allocator that'd just hand out sequential pieces of memory from a large buffer without keeping track of it at all, but if it made things any better, the effect was small enough that we couldn't reliably measure it.
- CyberDildonics 9y agoNot exactly. You save on deallocations, but you still have the allocator doing system calls and then fitting the sizes you want into the heap. Granted the heap probably doesn't have to work hard to make them fit, but it still allocates. I don't know if under the hood system calls to mmap or virtual_alloc block other processes calling the same functions.
- vidarh 9y agoTry measuring it with various allocators. Some will try too hard and be "too smart" and will add extra cost, but most will try to get out of your way until/unless there are deallocations leaving holes. They'll typically ask for a multiple of pages at a time, and some increase the number of pages requested each time. A quick check with strace right now shows modern glibc in my instance allocating about 2MB at the time when running "find" for example. It will do more if you ask for larger chunks of memory at the time, of course. In term of fitting in, some allocators will segment the heap it requests into separate free lists, but many will do this first on deallocations to combat fragmentation. E.g. here's a simple example that shows objects linearly allocated on the heap (on my system, anyway) despite different sizes: int main () { void * a, * b, *c, *d; a = malloc(4); b = malloc(8); c = malloc(12); d = malloc(16); printf("a=%p\nb=%p\nc=%p\nd=%p\n", a,b,c,d); } $ ./test a=0x1a7d010 b=0x1a7d030 c=0x1a7d050 d=0x1a7d070 Increasing the size of c and d to 64 and 128 bytes respectively to get over the alignment size, I get: $ ./test a=0x1678010 b=0x1678030 c=0x1678050 d=0x16780a0 So still allocating linearly at those, admittedly still small size allocations. In this case, the cost of doing the allocation typically boils down to seeing if there is space in one of the free lists, then if there's space at the high water mark of the current heap allocation, and only if there is not do another system call. When there is space, the cost tends to be a few comparisons, adding some book-keeping details, and increasing a pointer. It's extremely rare to even show up when profiling. So the cost of those syscalls tends to get amortized over enough allocations that the cost is negligible. E.g. with my example above, glibc does a single allocation to cover the 4 allocations I did. For a process that only ever allocates, the number of system calls will depend on how aggressive the allocator is about growing the size of the extra memory it requests, but in general the number of syscalls will be small. I appreciate the attention to syscalls - it's a pet peeve of mine that people tend to think library calls are cheap, without realising the cost difference between a mostly user-space one vs. ones that do a syscall and hence context switch every time (read() and write() are particularly persistently abused - add buffering, people), but malloc() or C++ new are rarely amongst them (doesn't mean you should do lots of tiny ones, though; if you do, profile and consider a pool allocator).
- nly 9y ago> One of the first performance optimizations we did was accept that most object lifetimes roughly coincided with the end of the request, so we simply didn't deallocate anything This'd speed up development but wouldn't help performance with CGI, since spawning a process is very expensive. This is why CGI is dead.
- nitrogen 9y agoThe parent post said they used static linking to remove much of the process spawning overhead.
- vidarh 9y agoOur actual measurements and server loads proved otherwise. Actually, in reality things were exactly opposite of what you suggest: it provided no noticeable benefits for development, but massively improved performance in production. Spawning a process on Linux is more expensive than spawning a thread or reusing the same process via a fastcgi type setup, but it's not all that expensive when you're spawning a statically linked app with no complex startup code and the app is guaranteed to be in the buffer cache at any moment. What killed CGI for most uses was not process creation overhead, but dynamic linking and applications not written to take advantage of the fact they'd die quickly. We knew that because we actually measured first, before deciding what strategy to take. We got performance very close to a fastcgi type setup with the above method, and without the complexity of ensuring we didn't have any resource leaks anywhere. Doing the above was enough to bring process startup overhead sufficiently far down our list of performance issues that it was not worth optimizing further.