4 ms·

Your original comment was trying to argue that allocations aren't a big performance problem. Anyone with experience optimizing knows this is not true and that

Your original comment was trying to argue that allocations aren't a big performance problem. Anyone with experience optimizing knows this is not true and that minimizing allocations is the first thing to look at because it will most likely have the largest payoff with the least effort. Incorrect preconceptions and misinformation doesn't help people. Saying 'allocations are fast' because you can make unnecessary allocations cheaper while ignoring deallocation is basically a lie, it's playing some sort of language game while telling people the opposite of what is actually true. > Most allocations are from the fast path... that's why they bother to make it fast. This is where you have a deep misconception. Doing millions of allocations that could be one allocation is not fast. If you want to add 1000000 to a variable do you loop through it one million times and increment it by 1 every time? > TLAB allocation is heterogenous (which solves your size question.) An array isn't. TLAB can be evacuated and fragmention-free (which solves your lifetime problem.) An array can't. That's like building a skyscraper out of legos because they fit together so well. It's nonsensical, especially due to pages and memory mapping. > I said 'Deallocation though - yes that's slow!' in my very first comment. Do you have a house with a kitchen and no bathroom? The discussion is that excessive memory allocation is a huge factor in performance. Trying to play language games to ignore the entire cycle doesn't change that and misleads people. Allocated memory needs to be deallocated. Lots of tiny allocations of a few bytes each are a mistake. They cause huge performance problems and should be obviously unnecessary. You wouldn't pick one piece of cereal out of the box, you would pour it in. Do you profile your programs? Do you work on optimizing them? I have a hard time believing you are doing something performance sensitive while trying to rationalize massive memory allocation waste.

chrisseaton 6y ago
> This is where you have a deep misconception. Doing millions of allocations that could be one allocation is not fast. If you want to add 1000000 to a variable do you loop through it one million times and increment it by 1 every time? Great example... because guess what happens with TLAB allocation during optimisation? It will do exactly what you say and combine multiple allocations into a simple bump, like it would with any other arithmetic! > Anyone with experience optimizing knows this is not true ... Do you profile your programs? Do you work on optimizing them? I have a hard time believing you are doing something performance sensitive I'm not going to keep arguing on this forever. But I have a PhD in language implementation and optimisation and I've be working professionally and publishing in the field for almost a decade. I'd be surprised to find out I have deep misconceptions on the topic.
- CyberDildonics 6y ago
  > Great example... because guess what happens with TLAB allocation during optimisation? It will do exactly what you say and combine multiple allocations into a simple bump, like it would with any other arithmetic! Before you were saying it isn't a problem at all, now you are saying it isn't a problem because a single java allocator tries to work around it? That's a bandaid over something that you are trying to say isn't even a problem. > I'm not going to keep arguing on this forever. But I have a PhD in language implementation and optimisation and I've be working professionally and publishing in the field for almost a decade. I'd be surprised to find out I have deep misconceptions on the topic. But are you profiling and optimizing actual software? That's the real question, because as I keep saying, anyone optimizing software knows that lots of small allocations are the first thing to look for. You still haven't addressed this, even though the whole thread is about decreasing allocations for optimization and you seem to be saying it isn't a problem, which is misinformation.
  - chrisseaton 6y ago
    > Before you were saying it isn't a problem at all It isn't a problem - I was responding to what you thought was a problem - having to bump multiple times in loop - that's what you asked me about? The bumps are tiny to start with, but even from that tiny start point they still get collapsed. You said collapsing them was important, so I showed you how that also happens. > as I keep saying, anyone optimizing software knows that lots of small allocations are the first thing to look for In some cases I've made code faster by putting allocations back in that other people removed without enough context on how allocation works and based on cargo culting of allocation being simply bad like you're pushing. Sometimes allocation is much better than reusing existing allocated space, due to reasons of spatial and temporal locality, cache obliviousness, escape analysis, publication semantics, coherence and NUMA effects, and more.
    - CyberDildonics 6y ago
      To be clear, you specifically brought up an optimization that you are now saying was done for no reason since it gains nothing. Now you are trying to say that you actually optimize programs by taking minimal memory allocation and instead using millions of tiny allocations of a few bytes each? Do I have this right? Feel free to link that example. > Sometimes allocation is much better than reusing existing allocated space, due to reasons of spatial and temporal locality, cache obliviousness, escape analysis, publication semantics, coherence and NUMA effects, and more. Are we to the part where you are just throwing out terminology? Are you really digging in this hard? Virtually everything written about optimizing memory allocations is about doing it less. Even this article is about decreasing memory allocations for performance. You made a claim that is very unsound in a pragmatic sense and anyone with experience would recognize that. You can just own up to that instead of going into a territory of wild claims and diversions.
kragen 6y ago
> Anyone with experience optimizing knows this is not true...This is where you have a deep misconception. …Do you profile your programs? There's Dunning–Kruger, and then there's Dunning–Kruger of the level "telling a guy whose Ph.D. dissertation, Specialising Dynamic Techniques for Implementing The Ruby Programming Language, is about code optimization on GraalVM, that he has a deep misconception, and questioning whether he has any experience optimizing". I repeat my complaint about "the climate of boastful intellectual vacuity this site fosters."
- CyberDildonics 6y ago
  You realize this person that you are calling an expert claimed that they optimize software by putting tiny memory allocations into their tight loops right? I'm not really sure how anything I'm saying is even controversial unless someone is desperate for it to be. Anyone who has profiled and optimized software has experience weeding out excessive memory allocations since it is almost always the lowest hanging fruit. No matter how fast allocating a few bytes is, doing what you are ultimately trying to do with those bytes is much faster. Allocating 8 bytes in 150 cycles might seem fast, until you realize that modern CPUs can deal with multiple integers or floats on every single cycle. A 12 year old CPU can allocate space for, and add together well over 850 million floats to 850 million other floats in a single second on a single core. You can download ISPC and verify this for yourself. By your own numbers, the allocation alone would take about a minute and a half. Neither of you has confronted this. I'm actually fascinated by the lengths you both have gone to specifically to avoid confronting this. Saying that lots of small allocations has no impact on speed is counter to basically all optimization advice and here I have explained why that is.
  - chrisseaton 6y ago
    > You realize this person that you are calling an expert claimed that they optimize software by putting tiny memory allocations into their tight loops right? You're either misunderstanding, or pretending to misunderstand for some reason. What I said was that allocating fresh objects and using those can be faster than re-using stale objects in some failed attempt to optimise by reducing allocations. Why would that be? For the reasons I explained: The newly allocated objects are guaranteed to already be in cache. Each new object is guaranteed to be close to the last object you used, because they're allocated next to each other. The new objects are not going to need any memory barriers, because they're guaranteed to not be published. The new objects are less likely to escape, so they're eligible for scalar replacement. You dismissed all that as 'throwing out terminology'. Here's a practical example: require 'benchmark/ips' def clamp_fresh(min, max, value) fresh_array = Array.new fresh_array[0] = min fresh_array[1] = max fresh_array[2] = value fresh_array.sort! fresh_array[1] end def clamp_cached(cached_array, min, max, value) cached_array[0] = min cached_array[1] = max cached_array[2] = value cached_array.sort! cached_array[1] end cached_array = Array.new Benchmark.ips do |x| x.report("use-fresh-objects") { clamp_fresh(10, 90, rand(0..100)) } x.report("use-cached-objects") { clamp_cached(cached_array, 10, 90, rand(0..100)) } x.compare! end Which would you think is faster? The one that allocates a new object each iteration of the inner loop? Or the one that re-uses an existing object each time and doesn't allocate anything? It's actually the one that allocates a new object each time. The cached one is 1.6x slower in an optimising implementation of Ruby. It's faster... but the only change I made was I added an object allocation instead of the custom object caching. I went from not allocating any objects to allocating an object and it became faster. This example is so clear because of the last factor I mentioned - scalar replacement. If you came along and 'optimised' my code based on a cargo cult idea of 'object allocation disastrously slow' you wouldn't be helping would you?

chrisseaton 6y ago

> This is where you have a deep misconception. Doing millions of allocations that could be one allocation is not fast. If you want to add 1000000 to a variable do you loop through it one million times and increment it by 1 every time? Great example... because guess what happens with TLAB allocation during optimisation? It will do exactly what you say and combine multiple allocations into a simple bump, like it would with any other arithmetic! > Anyone with experience optimizing knows this is not true ... Do you profile your programs? Do you work on optimizing them? I have a hard time believing you are doing something performance sensitive I'm not going to keep arguing on this forever. But I have a PhD in language implementation and optimisation and I've be working professionally and publishing in the field for almost a decade. I'd be surprised to find out I have deep misconceptions on the topic.

CyberDildonics 6y ago
> Great example... because guess what happens with TLAB allocation during optimisation? It will do exactly what you say and combine multiple allocations into a simple bump, like it would with any other arithmetic! Before you were saying it isn't a problem at all, now you are saying it isn't a problem because a single java allocator tries to work around it? That's a bandaid over something that you are trying to say isn't even a problem. > I'm not going to keep arguing on this forever. But I have a PhD in language implementation and optimisation and I've be working professionally and publishing in the field for almost a decade. I'd be surprised to find out I have deep misconceptions on the topic. But are you profiling and optimizing actual software? That's the real question, because as I keep saying, anyone optimizing software knows that lots of small allocations are the first thing to look for. You still haven't addressed this, even though the whole thread is about decreasing allocations for optimization and you seem to be saying it isn't a problem, which is misinformation.
- chrisseaton 6y ago
  > Before you were saying it isn't a problem at all It isn't a problem - I was responding to what you thought was a problem - having to bump multiple times in loop - that's what you asked me about? The bumps are tiny to start with, but even from that tiny start point they still get collapsed. You said collapsing them was important, so I showed you how that also happens. > as I keep saying, anyone optimizing software knows that lots of small allocations are the first thing to look for In some cases I've made code faster by putting allocations back in that other people removed without enough context on how allocation works and based on cargo culting of allocation being simply bad like you're pushing. Sometimes allocation is much better than reusing existing allocated space, due to reasons of spatial and temporal locality, cache obliviousness, escape analysis, publication semantics, coherence and NUMA effects, and more.
  - CyberDildonics 6y ago
    To be clear, you specifically brought up an optimization that you are now saying was done for no reason since it gains nothing. Now you are trying to say that you actually optimize programs by taking minimal memory allocation and instead using millions of tiny allocations of a few bytes each? Do I have this right? Feel free to link that example. > Sometimes allocation is much better than reusing existing allocated space, due to reasons of spatial and temporal locality, cache obliviousness, escape analysis, publication semantics, coherence and NUMA effects, and more. Are we to the part where you are just throwing out terminology? Are you really digging in this hard? Virtually everything written about optimizing memory allocations is about doing it less. Even this article is about decreasing memory allocations for performance. You made a claim that is very unsound in a pragmatic sense and anyone with experience would recognize that. You can just own up to that instead of going into a territory of wild claims and diversions.

kragen 6y ago

> Anyone with experience optimizing knows this is not true...This is where you have a deep misconception. …Do you profile your programs? There's Dunning–Kruger, and then there's Dunning–Kruger of the level "telling a guy whose Ph.D. dissertation, Specialising Dynamic Techniques for Implementing The Ruby Programming Language, is about code optimization on GraalVM, that he has a deep misconception, and questioning whether he has any experience optimizing". I repeat my complaint about "the climate of boastful intellectual vacuity this site fosters."

CyberDildonics 6y ago
You realize this person that you are calling an expert claimed that they optimize software by putting tiny memory allocations into their tight loops right? I'm not really sure how anything I'm saying is even controversial unless someone is desperate for it to be. Anyone who has profiled and optimized software has experience weeding out excessive memory allocations since it is almost always the lowest hanging fruit. No matter how fast allocating a few bytes is, doing what you are ultimately trying to do with those bytes is much faster. Allocating 8 bytes in 150 cycles might seem fast, until you realize that modern CPUs can deal with multiple integers or floats on every single cycle. A 12 year old CPU can allocate space for, and add together well over 850 million floats to 850 million other floats in a single second on a single core. You can download ISPC and verify this for yourself. By your own numbers, the allocation alone would take about a minute and a half. Neither of you has confronted this. I'm actually fascinated by the lengths you both have gone to specifically to avoid confronting this. Saying that lots of small allocations has no impact on speed is counter to basically all optimization advice and here I have explained why that is.
- chrisseaton 6y ago
  > You realize this person that you are calling an expert claimed that they optimize software by putting tiny memory allocations into their tight loops right? You're either misunderstanding, or pretending to misunderstand for some reason. What I said was that allocating fresh objects and using those can be faster than re-using stale objects in some failed attempt to optimise by reducing allocations. Why would that be? For the reasons I explained: The newly allocated objects are guaranteed to already be in cache. Each new object is guaranteed to be close to the last object you used, because they're allocated next to each other. The new objects are not going to need any memory barriers, because they're guaranteed to not be published. The new objects are less likely to escape, so they're eligible for scalar replacement. You dismissed all that as 'throwing out terminology'. Here's a practical example: require 'benchmark/ips' def clamp_fresh(min, max, value) fresh_array = Array.new fresh_array[0] = min fresh_array[1] = max fresh_array[2] = value fresh_array.sort! fresh_array[1] end def clamp_cached(cached_array, min, max, value) cached_array[0] = min cached_array[1] = max cached_array[2] = value cached_array.sort! cached_array[1] end cached_array = Array.new Benchmark.ips do |x| x.report("use-fresh-objects") { clamp_fresh(10, 90, rand(0..100)) } x.report("use-cached-objects") { clamp_cached(cached_array, 10, 90, rand(0..100)) } x.compare! end Which would you think is faster? The one that allocates a new object each iteration of the inner loop? Or the one that re-uses an existing object each time and doesn't allocate anything? It's actually the one that allocates a new object each time. The cached one is 1.6x slower in an optimising implementation of Ruby. It's faster... but the only change I made was I added an object allocation instead of the custom object caching. I went from not allocating any objects to allocating an object and it became faster. This example is so clear because of the last factor I mentioned - scalar replacement. If you came along and 'optimised' my code based on a cargo cult idea of 'object allocation disastrously slow' you wouldn't be helping would you?