5 ms·

Hardware Intrinsics in .NET 8

140 points by GordonS 3y ago

flumpcakes 3y ago
How do people reason about performance in GC'd languages? I mean specifically native performance: memory/cpu cycles rather than choosing the right algorithm/logical performance. In C, Zig, Rust, C++ (hardish) I can sit down with pen and paper and map out the layout of memory and take informed decisions on cache locality, access patterns, etc. In C#, Go, Python (which I am no expert, or any language really) I understand the logical performance impacts, for example in a web framework ORM you should filter in your DB call rather than return the data and filter in the language runtime (this can often look very similar in code). However, I can't reason about native performance. It surprises me that C# has intrinsics, I know where they would be used in a compiled language and can reason about how that works. It's not clear to me in a GC'd/JIT'd language. Again, perhaps that's just because I'm not an expert/programmer.
- lambda_garden 3y ago
  In C# you can create types where you control the layout and they are always stack allocated.
  - vrighter 3y ago
    This is not really true, as far as I know. You can control the memory layout of structs, they can be allocated on the stack, because they're value types, but not necessarily. If you, for example, create an array of said struct type, the array's memory will be on the heap. And the structs will be stored in it. You can't assume a struct will be on the stack.
    - vrighter 3y ago
      nevermind... I was just out of date. C3 now has ref structs
- xnorswap 3y ago
  There are memory management tools: You can layout a struct (which are value types not class types in c#/dotnet) with LayoutKind attributes: https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.layoutkind?view=net-8.0 https://learn.microsoft.com/en-us/dotnet/api/system.runtime.... You can also use things like Span<T> and other types for better control. But honestly C# performs generally so well that it tends to be you only reach for these tools after you spot a performance problem and profile it, which is actually quite rare. IME most performance problems and solutions still tend to be, "Oh this person took a naive approach here, we can use a specialised loop or data structure to fix that".
- dmytroi 3y ago
  C# has support for unsafe code / pointer arithmetic / memory pinning / unmanaged types [1]. Using them it's possible to write performant computations where needed. - [1] https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/unsafe-code https://learn.microsoft.com/en-us/dotnet/csharp/language-ref...
- PhilipRoman 3y ago
  Not familiar with .NET, but I've done this for Java. There is nothing magical about the JIT, you can let it warm up, suspend the process and look at the compiled code with a debugger (prepare to be disappointed though). When working with primitive arrays, its not very different from C. Lack of value types / reinterpreting memory is a big problem for Java, but not so much for C# IIRC. Fingers crossed for project Valhalla :) The only thing that is kind of opaque is the GC, you need to learn the basics of how the algorithm works. There are some surprising relevations, like time of GC being proportional to surviving objects, not dead ones. Not to mention the new generation of GCs which use all kinds of tricks inside their read/write barriers to trade a little bit of throughput for low latency.
  - doikor 3y ago
    > Not familiar with .NET, but I've done this for Java. Java only has intrinsics for core jdk APIs and some auto-vectorization (which in my experience are between very hard to impossible to use reliably). A proper Vector API is in the works but that is waiting for Valhalla (value objects). https://openjdk.org/jeps/460 https://openjdk.org/jeps/460
    - pjmlp 3y ago
      One advantage of having multiple implementations, is that there is Java the language (like ISO C), and then there are the JVM implementations (like GCC). Azul and OpenJ9 have intrisics for object layout, ART has intrisics for faster JNI, microJ has intrisics for SNI (a simpler JNI alternative), and so on.
- fenring 3y ago
  I would say, that beside logical performance as you called it, 99% of optimizations is profiling the memory allocations and minimizing it by using more performant (lower-lever) API's, or different techniques to reduce as many allocations as possible. I've never really saw any meaningful examples of performance optimizations from the list you provided: memory layout, cache locality, access patterns. If that would be needed, I would say that wrong language was selected for solving the problem. In my company, we use .NET for most of our applications, with Rust being used in places where we needed raw processing speed. But to this day, I'm not 100% convinced that was good choice - I don't know how much faster is Rust in our case vs .NET written with performance as priority, and overhead of interoperability, separate team, etc. is huge.
  - ahoka 3y ago
    I was working on performance critical .net applications. Reasoning about memory layout, cache locality and access patterns are exactly the same as in any other langauge. I don’t understand how that would be any different in a .net. Memory allocations are actually faster in a GC language, given a proper heap size. I’m assuming performance means mostly throughput here. You probably lose more performance with the IPC/FFI than you gain, so my guess would be it was not a good choice.
  - physicsguy 3y ago
    I've worked on HPC applications doing simulation software, and those things are definitely important for that sort of application. But I really don't think I'd recommend it for general purpose web type stuff - there, there are a ton of performance factors that you can optimise before going down that route and it should probably be a last resort for a smaller company to turn that way.
- Thaxll 3y ago
  Go memory model is similar to lower level language, so it's not very hard to reason about. https://go.dev/ref/mem https://go.dev/ref/mem
- hayley-patton 3y ago
  What's odd about using vector intrinsics with GC and/or JIT? I did my capstone project for university with intrinsics for SBCL - testing hand-vectorised code at the REPL is rather funny (as it's an interactive environment for almost assembly) but it works fine. So long as you have arrays of integers you can load and store vectors, and the rest is all the same.
- soundarana 3y ago
  When you run a tight hot loop, GC has no impact. Just like you wouldn't allocate smart pointers in a tight C++ loop. You can do dumb slow things in both languages, and yes, C# programmers might be less informed about these topics, but it's not a language problem.
  - charleslmunger 3y ago
    This is not always true - for many GCs, threads need to suspend at least some of the time. If the compiler can't prove a small bound on the iterations of a loop, it needs to insert safe point instructions. Also depending on the GC, read or write barriers may be required. Lastly there are circumstances in generational moving GCs where there is extra overhead due to checking to and from space - for example if you attempt a compare-and-set on a reference type, you need extra code to check if your CAS failed due to comparing the from-space and to-space addresses of the same object. I work primarily in garbage collected languages, and there are many benefits - safety, throughput, easy ABA avoidance in lock-free code - but using a GC carries tradeoffs.
- bob1029 3y ago
  I try to not think about it. GC'd languages have their own weird performance advantages. GC is not pure downside in performance terms (at least for .NET). It actually brings some upside to the table. Deferring memory management to a batched, asynchronous background process can scale better in some applications. Your physical hardware likes to work in batches of contiguous things wherever possible. The only case where GC burns me is by introducing latency to the application. In situations where I cannot tolerate long GC pauses, my solution is to go straight through hell - invoke GC on purpose every time my hot path iterates. This ensures that at any given moment, the amount of trash to be collected is minimal and should incur the smallest delay. This works especially well in things like game loops where you have a very obvious, well-compensated edge upon which to collect garbage.
- pjmlp 3y ago
  The same way as in any language when done properly, we use profilers. Plenty of languages have intrisics, it is a urban myth that only C derived languages do.
- pjc50 3y ago
  They've linked to some examples: https://source.dot.net/#System.Linq/System/Linq/MaxMin.cs,70 https://source.dot.net/#System.Linq/System/Linq/MaxMin.cs,70 It's a little hard to read (due to the use of "ref" everywhere for raw pointer arithmetic), but that implements the use of vectorized min/max instructions to find the max of elements in an array (or any span), comparing them N at a time rather than 1 at a time. If you don't call "new", and are careful to call non-allocating functions, and pass everything by reference, then the GC is irrelevant. Obviously you only bother to do this at the very bottom of your flame graph in the hottest spots.
- gwbas1c 3y ago
  > In C, Zig, Rust, C++ (hardish) I can sit down with pen and paper and map out the layout of memory and take informed decisions on cache locality, access patterns, etc. When direct control over your memory is important, you're better off using those languages. When working with idiomatic C#, you really don't think about direct memory management. The garbage collector is excellent, and in many cases, it can outperform systems like reference counting. The primary critical tradeoff with C#'s garbage collector is memory footprint. When you directly control your memory, you can free it as soon as it's not needed. (We can typically say that the extra RAM is cheaper than developer time... Until it's not, and then "map out the layout of memory" becomes worth it.) The secondary critical tradeoff with C#'s garbage collector is that you don't want to write a general-purpose library like SQLite in C#. (You wouldn't want to suck .Net into a Python or Java process.) But, assuming you're writing a typical web application, server-side batch processing system, desktop application, ect, these tradeoffs often work out in your favor with C#'s garbage collector. (But, if you're running your code on 1000s of memory-constrained servers, "map out the layout of memory" may be cheaper in the long run.) And, yes, in C# you can do direct memory management with things like unsafe code (pointers and pointer arithmetic,) and structs. In my experience, the one time I did a deep dive into those optimizations, they had no advantage over "idiomatic" C# when I hooked up a profiler.
  - pjmlp 3y ago
    Well, it is a matter of knowing how to take advantage of them. https://devblogs.microsoft.com/dotnet/grpc-performance-improvements-in-net-5/ https://devblogs.microsoft.com/dotnet/grpc-performance-impro... Or just browse the performance improvements blog posts, https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-6/ https://devblogs.microsoft.com/dotnet/performance-improvemen... https://devblogs.microsoft.com/dotnet/performance_improvements_in_net_7/ https://devblogs.microsoft.com/dotnet/performance_improvemen... https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-8/ https://devblogs.microsoft.com/dotnet/performance-improvemen...
    - gwbas1c 3y ago
      I did direct memory management in C# before those techniques were available. (It was a .Net Framework 4.x application.)
- jabart 3y ago
  C# needs them to use them in their own frameworks. GC'd languages has a benefit that Strings can work different then C strings and you get things like interning to reduce memory usage. Now C#(dotnet) took it a step further and put in a bunch of optimizations on strings called Spans which is like a block of memory but in a safe friendly way. Now back to intrinsics, you can take a Span, shove it into a vector and poof, you have string matching. You can also use a memory safe image processing library and get intrinsic optimization for scaling, conversion, etc. I think there is also a buffer pool of spans in asp.net that uses spans to manage all the data coming and going from headers then tosses out the exact same headers while using one copy of it then giving that memory back to the next without having it GC'd. Web servers need a fast response time to serve more requests per a second with less resources and it's all strings all the time dealing with it. Some fancy Spans and Intrinsics baked into the framework give those gains to every call in a C# app.
- ExoticPearTree 3y ago
  > How do people reason about performance in GC'd languages? I think there's a misconception about the fact that languages that have GC are slower than the ones that don't. It is universally true that you can write garbage code in any language, being either Java/C# or C/Rust. Java a while back had the problem that when GC would run, the application would freeze for a bit due to how the GC worked. But, on the other hand, at least for Java, once everything was loaded, it would consume pretty much the same resources for 100 requests or 10000 requests because the language itself was very good at concurrency. On the other hand, C++ with boost would have it's own particular issues with concurrency and after a certain number of concurrent threads it would severely impact performance. Those intrinsics are for people that know what they are doing, they know the architecture the code is going to run on and why using those is better than leaving to the compiler to optimize the resulting byte-code.
  - kaba0 3y ago
    I see what you say, but have to nitpick this: “Java a while back had the problem that when GC would run, the application would freeze for a bit due to how the GC worked” — Java has pretty much always had the very best garbage collector available, though it was historically mostly interested in throughput-oriented workloads (due to its wide-spread usage in server software). Other platforms may not have had it as noticeable, but they just probably traded off throughput for better latency (e.g. Go still does it, by blocking threads that would allocate when it can’t “make sure” it will be able to keep up).
- kaba0 3y ago
  > In C, Zig, Rust, C++ (hardish) I can sit down with pen and paper and map out the layout of memory and take informed decisions on cache locality, access patterns, etc s/informed decisions/educated guess at most Even a seemingly low-level C code can be transformed to such a random code by the compiler that every intuition you have is thrown out the window. Hell, even assembly can be very misleading, even expert compiler writers have to fall back to micro-benchmarks, as it can be very misleading - the longer code might just be faster.
- estebarb 3y ago
  It is almost the same with those languages. Malloc and free can be expensive if used wrong, same with allocations in a GC language. In some sense, I would even say that with a good GC is easier to improve the performance, as collection can be deferred out of the critical path.
- WorldMaker 3y ago
  In most of the GC languages, but I mostly have a C# performance perspective, there are two "planes" to consider: the stack and the heap. In very rough, general terms, the heap is GCed and subject to its own world of memory/cpu than you are used to and the stack is has cache locality/access patterns similar to what you would expect from non-GCed languages. C# today in stack space especially increasingly looks a lot like a Rust relative if you squint: Memory<T> and Span<T> (and some relatives) live only on the stack and form a sort of borrow checking mechanism that is safer (including and especially to/from GCed objects on the heap) than raw pointer manipulation but directly related to it. On the heap things obviously get more interesting and the rabbit hole can get quite deep. It's different from the unmanaged (non-GC) languages but similarities apply and eventually you start to map some common ground. Most GCs at this point are multi-generational and when you are talking about The Heap you are actually generally talking about a series of heaps (which is generally not just the name, but also vaguely resembles some of the internals of the data structure as well). In .NET these are generally called Gen 0, Gen 1, Gen 2, and the LOH (which may have multiple generations as well). Gen 0 is sometimes referred to as the "nursery" or the "short-lived generation", it's generally for fresh, new objects and gets garbage collected frequently. It can get quite fragmented and very chaotic. Objects that survive for long enough get promoted to Gen 1 and then again as they survive get promoted to Gen 2 which is meant for long-lived objects. Each generation is then in turn garbage collected with less frequency. The idea being that if an object in the heap appears to be long-lived it probably is. As that assumption holds (and it may not in non-well-behaved programs) the generations also get progressively less fragmented (higher cache locality in access patterns, in best cases). LOH stands for Large Object Heap and when objects get above a certain (sometimes configurable) memory size they get treated differently. In some cases (older GCs, GCs running in lower memory environments) the regular Gen 2 doubles as the LOH, so large objects are "assumed" to be long-lived. In more modern/more recent/more common cases the LOH has its own Gen 0/Gen 1/Gen 2 breakdown with a similar garbage collection pattern of decreasing frequency with each generation. A lot of the cache locality/access pattern balancing acts are figuring out the "best fit" heap and trying to optimize for that. You can build very performant apps with lots of small objects that preferably never escape the Gen 0 heap. Even though garbage collection is frequent it can be very fast with simple data structures (few to no reference cycles) and if most of the objects are tossed as garbage each collection the Gen 0 remains very chaotic, but not that fragmented. Most applications by default live in some version of this state, especially early in their development. Optimizing for it can sometimes be easy, too: figure out why references are living too long; often that means simplifying objects and removing reference cycles, and relying on data structures that work that way, which can be good refactors in general (often matching naive algorithm/logical performance analysis). In other cases, you may want to focus on pushing your objects to the long-lived Gen 2 for benefits like increased cache locality and generally less fragmented access patterns. There's all sorts of patterns to do this including the dull simple ones like just holding on to references as long as possible, to things like object pools (instead of building new objects, hold on to references to "dead"/"unused" ones and reuse/recycle them as new again). Knowing which kind of objects should never escape Gen 0 if you can help it and which ones should move all the way to Gen 2 can be a fascinating balance to strike. The LOH obviously has similar concerns, but they all grow in complexity because those objects are large and potentially take up a lot of memory. If you are building a lot of "Gen 0" short-lived large objects, that might cause memory pressure issues. Having some idea of why those objects are so large and also getting garbage collected so quickly can help a lot to stabilize performance. But that also isn't a reason to avoid large objects. Some types of large objects can overall improve performance as they can be ways to more easily build cache locality or improve access patterns in cases like buffers and "memory pools", especially when long-lived in an application ("Gen 2 LOH"). As for intrinsics, they mostly apply to stack space, but with some Memory<T> and Span<T> options you can "borrow check" data from heap references. GCs obviously move things around (compaction to reduce fragmentation, collection to remove garbage and promote to the next generation) so the short-lived nature of stack space is interestingly important to their safety. Most of the intrinsics work with Vector<T>, a struct (stack space) which (among other options) can be constructed from any ReadOnlySpan<T> of the right size (and layout). That ReadOnlySpan<T> could be borrowed from an object somewhere in heap space (such as an intentional memory pool in Gen 2 LOH). No "copying" happens to build the Vector<T> because the ReadOnlySpan<T> is a type-safe/GC-safe pointer at that point and the Vector<T> can't last longer than the ReadOnlySpan<T>. Cache locality of that Vector<T> and subsequent ones in an operation become based on the data structure of your objects, but especially in the case of an intentional memory pool (an array or buffered series of arrays of some form) may be however you need it to be structured (arrays of value types like integers are designed to be contiguous memory and you can arrange things how you like in arrays, just like any other language; even if the GC affects where that array lives/moves, the data within it remains contiguous in the way you write it/read it, just about as you would expect).
doikor 3y ago
I really wish JDK had some API for hardware intrinsics. Hopefully we get something soon as the Vector API is coming (7th incubator version rolling on) https://openjdk.org/jeps/460 https://openjdk.org/jeps/460 Though the Vector API is pending on Valhalla to get far enough (it won't leave incubator until the necessary parts of Valhalla are in preview stage)
- pjmlp 3y ago
  Some alternative implementations like PTC and Aicas do, for the Real Time Java implementation.
- Const-me 3y ago
  Vector API is very far from being hardware intrinsics. Unfortunately for Java programmers, it’s merely a least common denominator of SIMD instructions across different ISAs. This makes the feature very limited by design, IMO borderline useless.
  - doikor 3y ago
    Yeah while not great it is still a step in the right direction (giving control to the developer). Before this in the OpenJDK the developer had exactly 0 control. You just wrote code and hoped JDK would slap in vectorization etc.
  - hajile 3y ago
    This depends on the implementation under the hood and I'd say that the ability to use SIMD even at less than peak efficiency is better than not being able to use it at all.
    - Const-me 3y ago
      For some problems, sticking to the SIMD subset exposed to Java might be slower than no SIMD at all. Example: https://news.ycombinator.com/item?id=36624240 https://news.ycombinator.com/item?id=36624240 The linked C++ code is easily portable to C# because hardware intrinsics, but not to Java because nothing similar to _mm256_sad_epu8 is available there.
neonsunset 3y ago
I wonder if .NET related submissions get score penalty on HN, the posts almost never manage to reach front-page or stay there for any meaningful period of time despite the score.
- asp_hornet 3y ago
  I don't think so, it’s just a polarising ecosystem. I worked in the .net for over 12 years and im so happy to be out. There’s nothing they could add to their language or tooling to entice me back.
  - neonsunset 3y ago
    Which ecosystem are you switching to? (this better not be Go which is strictly worse)
  - Guillaume86 3y ago
    And if you had to defend your position with arguments, what would the big ones be?
mappu 3y ago
> AVX-512 is a new feature set provided for x86 and x64 computers. Is it really possible to use AVX512 instructions from x86_32 mode?
- colejohnson66 3y ago
  Yes. You're just limited to eight vector registers.

140 points by GordonS 3y ago

How do people reason about performance in GC'd languages? I mean specifically native performance: memory/cpu cycles rather than choosing the right algorithm/logical performance. In C, Zig, Rust, C++ (hardish) I can sit down with pen and paper and map out the layout of memory and take informed decisions on cache locality, access patterns, etc. In C#, Go, Python (which I am no expert, or any language really) I understand the logical performance impacts, for example in a web framework ORM you should filter in your DB call rather than return the data and filter in the language runtime (this can often look very similar in code). However, I can't reason about native performance. It surprises me that C# has intrinsics, I know where they would be used in a compiled language and can reason about how that works. It's not clear to me in a GC'd/JIT'd language. Again, perhaps that's just because I'm not an expert/programmer.

lambda_garden 3y ago
In C# you can create types where you control the layout and they are always stack allocated.
- vrighter 3y ago
  This is not really true, as far as I know. You can control the memory layout of structs, they can be allocated on the stack, because they're value types, but not necessarily. If you, for example, create an array of said struct type, the array's memory will be on the heap. And the structs will be stored in it. You can't assume a struct will be on the stack.
  - vrighter 3y ago
    nevermind... I was just out of date. C3 now has ref structs
xnorswap 3y ago
There are memory management tools: You can layout a struct (which are value types not class types in c#/dotnet) with LayoutKind attributes: https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.layoutkind?view=net-8.0 https://learn.microsoft.com/en-us/dotnet/api/system.runtime.... You can also use things like Span<T> and other types for better control. But honestly C# performs generally so well that it tends to be you only reach for these tools after you spot a performance problem and profile it, which is actually quite rare. IME most performance problems and solutions still tend to be, "Oh this person took a naive approach here, we can use a specialised loop or data structure to fix that".
dmytroi 3y ago
C# has support for unsafe code / pointer arithmetic / memory pinning / unmanaged types [1]. Using them it's possible to write performant computations where needed. - [1] https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/unsafe-code https://learn.microsoft.com/en-us/dotnet/csharp/language-ref...
PhilipRoman 3y ago
Not familiar with .NET, but I've done this for Java. There is nothing magical about the JIT, you can let it warm up, suspend the process and look at the compiled code with a debugger (prepare to be disappointed though). When working with primitive arrays, its not very different from C. Lack of value types / reinterpreting memory is a big problem for Java, but not so much for C# IIRC. Fingers crossed for project Valhalla :) The only thing that is kind of opaque is the GC, you need to learn the basics of how the algorithm works. There are some surprising relevations, like time of GC being proportional to surviving objects, not dead ones. Not to mention the new generation of GCs which use all kinds of tricks inside their read/write barriers to trade a little bit of throughput for low latency.
- doikor 3y ago
  > Not familiar with .NET, but I've done this for Java. Java only has intrinsics for core jdk APIs and some auto-vectorization (which in my experience are between very hard to impossible to use reliably). A proper Vector API is in the works but that is waiting for Valhalla (value objects). https://openjdk.org/jeps/460 https://openjdk.org/jeps/460
  - pjmlp 3y ago
    One advantage of having multiple implementations, is that there is Java the language (like ISO C), and then there are the JVM implementations (like GCC). Azul and OpenJ9 have intrisics for object layout, ART has intrisics for faster JNI, microJ has intrisics for SNI (a simpler JNI alternative), and so on.
fenring 3y ago
I would say, that beside logical performance as you called it, 99% of optimizations is profiling the memory allocations and minimizing it by using more performant (lower-lever) API's, or different techniques to reduce as many allocations as possible. I've never really saw any meaningful examples of performance optimizations from the list you provided: memory layout, cache locality, access patterns. If that would be needed, I would say that wrong language was selected for solving the problem. In my company, we use .NET for most of our applications, with Rust being used in places where we needed raw processing speed. But to this day, I'm not 100% convinced that was good choice - I don't know how much faster is Rust in our case vs .NET written with performance as priority, and overhead of interoperability, separate team, etc. is huge.
- ahoka 3y ago
  I was working on performance critical .net applications. Reasoning about memory layout, cache locality and access patterns are exactly the same as in any other langauge. I don’t understand how that would be any different in a .net. Memory allocations are actually faster in a GC language, given a proper heap size. I’m assuming performance means mostly throughput here. You probably lose more performance with the IPC/FFI than you gain, so my guess would be it was not a good choice.
- physicsguy 3y ago
  I've worked on HPC applications doing simulation software, and those things are definitely important for that sort of application. But I really don't think I'd recommend it for general purpose web type stuff - there, there are a ton of performance factors that you can optimise before going down that route and it should probably be a last resort for a smaller company to turn that way.
Thaxll 3y ago
Go memory model is similar to lower level language, so it's not very hard to reason about. https://go.dev/ref/mem https://go.dev/ref/mem
hayley-patton 3y ago
What's odd about using vector intrinsics with GC and/or JIT? I did my capstone project for university with intrinsics for SBCL - testing hand-vectorised code at the REPL is rather funny (as it's an interactive environment for almost assembly) but it works fine. So long as you have arrays of integers you can load and store vectors, and the rest is all the same.
soundarana 3y ago
When you run a tight hot loop, GC has no impact. Just like you wouldn't allocate smart pointers in a tight C++ loop. You can do dumb slow things in both languages, and yes, C# programmers might be less informed about these topics, but it's not a language problem.
- charleslmunger 3y ago
  This is not always true - for many GCs, threads need to suspend at least some of the time. If the compiler can't prove a small bound on the iterations of a loop, it needs to insert safe point instructions. Also depending on the GC, read or write barriers may be required. Lastly there are circumstances in generational moving GCs where there is extra overhead due to checking to and from space - for example if you attempt a compare-and-set on a reference type, you need extra code to check if your CAS failed due to comparing the from-space and to-space addresses of the same object. I work primarily in garbage collected languages, and there are many benefits - safety, throughput, easy ABA avoidance in lock-free code - but using a GC carries tradeoffs.
bob1029 3y ago
I try to not think about it. GC'd languages have their own weird performance advantages. GC is not pure downside in performance terms (at least for .NET). It actually brings some upside to the table. Deferring memory management to a batched, asynchronous background process can scale better in some applications. Your physical hardware likes to work in batches of contiguous things wherever possible. The only case where GC burns me is by introducing latency to the application. In situations where I cannot tolerate long GC pauses, my solution is to go straight through hell - invoke GC on purpose every time my hot path iterates. This ensures that at any given moment, the amount of trash to be collected is minimal and should incur the smallest delay. This works especially well in things like game loops where you have a very obvious, well-compensated edge upon which to collect garbage.
pjmlp 3y ago
The same way as in any language when done properly, we use profilers. Plenty of languages have intrisics, it is a urban myth that only C derived languages do.
pjc50 3y ago
They've linked to some examples: https://source.dot.net/#System.Linq/System/Linq/MaxMin.cs,70 https://source.dot.net/#System.Linq/System/Linq/MaxMin.cs,70 It's a little hard to read (due to the use of "ref" everywhere for raw pointer arithmetic), but that implements the use of vectorized min/max instructions to find the max of elements in an array (or any span), comparing them N at a time rather than 1 at a time. If you don't call "new", and are careful to call non-allocating functions, and pass everything by reference, then the GC is irrelevant. Obviously you only bother to do this at the very bottom of your flame graph in the hottest spots.
gwbas1c 3y ago
> In C, Zig, Rust, C++ (hardish) I can sit down with pen and paper and map out the layout of memory and take informed decisions on cache locality, access patterns, etc. When direct control over your memory is important, you're better off using those languages. When working with idiomatic C#, you really don't think about direct memory management. The garbage collector is excellent, and in many cases, it can outperform systems like reference counting. The primary critical tradeoff with C#'s garbage collector is memory footprint. When you directly control your memory, you can free it as soon as it's not needed. (We can typically say that the extra RAM is cheaper than developer time... Until it's not, and then "map out the layout of memory" becomes worth it.) The secondary critical tradeoff with C#'s garbage collector is that you don't want to write a general-purpose library like SQLite in C#. (You wouldn't want to suck .Net into a Python or Java process.) But, assuming you're writing a typical web application, server-side batch processing system, desktop application, ect, these tradeoffs often work out in your favor with C#'s garbage collector. (But, if you're running your code on 1000s of memory-constrained servers, "map out the layout of memory" may be cheaper in the long run.) And, yes, in C# you can do direct memory management with things like unsafe code (pointers and pointer arithmetic,) and structs. In my experience, the one time I did a deep dive into those optimizations, they had no advantage over "idiomatic" C# when I hooked up a profiler.
- pjmlp 3y ago
  Well, it is a matter of knowing how to take advantage of them. https://devblogs.microsoft.com/dotnet/grpc-performance-improvements-in-net-5/ https://devblogs.microsoft.com/dotnet/grpc-performance-impro... Or just browse the performance improvements blog posts, https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-6/ https://devblogs.microsoft.com/dotnet/performance-improvemen... https://devblogs.microsoft.com/dotnet/performance_improvements_in_net_7/ https://devblogs.microsoft.com/dotnet/performance_improvemen... https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-8/ https://devblogs.microsoft.com/dotnet/performance-improvemen...
  - gwbas1c 3y ago
    I did direct memory management in C# before those techniques were available. (It was a .Net Framework 4.x application.)
jabart 3y ago
C# needs them to use them in their own frameworks. GC'd languages has a benefit that Strings can work different then C strings and you get things like interning to reduce memory usage. Now C#(dotnet) took it a step further and put in a bunch of optimizations on strings called Spans which is like a block of memory but in a safe friendly way. Now back to intrinsics, you can take a Span, shove it into a vector and poof, you have string matching. You can also use a memory safe image processing library and get intrinsic optimization for scaling, conversion, etc. I think there is also a buffer pool of spans in asp.net that uses spans to manage all the data coming and going from headers then tosses out the exact same headers while using one copy of it then giving that memory back to the next without having it GC'd. Web servers need a fast response time to serve more requests per a second with less resources and it's all strings all the time dealing with it. Some fancy Spans and Intrinsics baked into the framework give those gains to every call in a C# app.
ExoticPearTree 3y ago
> How do people reason about performance in GC'd languages? I think there's a misconception about the fact that languages that have GC are slower than the ones that don't. It is universally true that you can write garbage code in any language, being either Java/C# or C/Rust. Java a while back had the problem that when GC would run, the application would freeze for a bit due to how the GC worked. But, on the other hand, at least for Java, once everything was loaded, it would consume pretty much the same resources for 100 requests or 10000 requests because the language itself was very good at concurrency. On the other hand, C++ with boost would have it's own particular issues with concurrency and after a certain number of concurrent threads it would severely impact performance. Those intrinsics are for people that know what they are doing, they know the architecture the code is going to run on and why using those is better than leaving to the compiler to optimize the resulting byte-code.
- kaba0 3y ago
  I see what you say, but have to nitpick this: “Java a while back had the problem that when GC would run, the application would freeze for a bit due to how the GC worked” — Java has pretty much always had the very best garbage collector available, though it was historically mostly interested in throughput-oriented workloads (due to its wide-spread usage in server software). Other platforms may not have had it as noticeable, but they just probably traded off throughput for better latency (e.g. Go still does it, by blocking threads that would allocate when it can’t “make sure” it will be able to keep up).
kaba0 3y ago
> In C, Zig, Rust, C++ (hardish) I can sit down with pen and paper and map out the layout of memory and take informed decisions on cache locality, access patterns, etc s/informed decisions/educated guess at most Even a seemingly low-level C code can be transformed to such a random code by the compiler that every intuition you have is thrown out the window. Hell, even assembly can be very misleading, even expert compiler writers have to fall back to micro-benchmarks, as it can be very misleading - the longer code might just be faster.
estebarb 3y ago
It is almost the same with those languages. Malloc and free can be expensive if used wrong, same with allocations in a GC language. In some sense, I would even say that with a good GC is easier to improve the performance, as collection can be deferred out of the critical path.
WorldMaker 3y ago
In most of the GC languages, but I mostly have a C# performance perspective, there are two "planes" to consider: the stack and the heap. In very rough, general terms, the heap is GCed and subject to its own world of memory/cpu than you are used to and the stack is has cache locality/access patterns similar to what you would expect from non-GCed languages. C# today in stack space especially increasingly looks a lot like a Rust relative if you squint: Memory<T> and Span<T> (and some relatives) live only on the stack and form a sort of borrow checking mechanism that is safer (including and especially to/from GCed objects on the heap) than raw pointer manipulation but directly related to it. On the heap things obviously get more interesting and the rabbit hole can get quite deep. It's different from the unmanaged (non-GC) languages but similarities apply and eventually you start to map some common ground. Most GCs at this point are multi-generational and when you are talking about The Heap you are actually generally talking about a series of heaps (which is generally not just the name, but also vaguely resembles some of the internals of the data structure as well). In .NET these are generally called Gen 0, Gen 1, Gen 2, and the LOH (which may have multiple generations as well). Gen 0 is sometimes referred to as the "nursery" or the "short-lived generation", it's generally for fresh, new objects and gets garbage collected frequently. It can get quite fragmented and very chaotic. Objects that survive for long enough get promoted to Gen 1 and then again as they survive get promoted to Gen 2 which is meant for long-lived objects. Each generation is then in turn garbage collected with less frequency. The idea being that if an object in the heap appears to be long-lived it probably is. As that assumption holds (and it may not in non-well-behaved programs) the generations also get progressively less fragmented (higher cache locality in access patterns, in best cases). LOH stands for Large Object Heap and when objects get above a certain (sometimes configurable) memory size they get treated differently. In some cases (older GCs, GCs running in lower memory environments) the regular Gen 2 doubles as the LOH, so large objects are "assumed" to be long-lived. In more modern/more recent/more common cases the LOH has its own Gen 0/Gen 1/Gen 2 breakdown with a similar garbage collection pattern of decreasing frequency with each generation. A lot of the cache locality/access pattern balancing acts are figuring out the "best fit" heap and trying to optimize for that. You can build very performant apps with lots of small objects that preferably never escape the Gen 0 heap. Even though garbage collection is frequent it can be very fast with simple data structures (few to no reference cycles) and if most of the objects are tossed as garbage each collection the Gen 0 remains very chaotic, but not that fragmented. Most applications by default live in some version of this state, especially early in their development. Optimizing for it can sometimes be easy, too: figure out why references are living too long; often that means simplifying objects and removing reference cycles, and relying on data structures that work that way, which can be good refactors in general (often matching naive algorithm/logical performance analysis). In other cases, you may want to focus on pushing your objects to the long-lived Gen 2 for benefits like increased cache locality and generally less fragmented access patterns. There's all sorts of patterns to do this including the dull simple ones like just holding on to references as long as possible, to things like object pools (instead of building new objects, hold on to references to "dead"/"unused" ones and reuse/recycle them as new again). Knowing which kind of objects should never escape Gen 0 if you can help it and which ones should move all the way to Gen 2 can be a fascinating balance to strike. The LOH obviously has similar concerns, but they all grow in complexity because those objects are large and potentially take up a lot of memory. If you are building a lot of "Gen 0" short-lived large objects, that might cause memory pressure issues. Having some idea of why those objects are so large and also getting garbage collected so quickly can help a lot to stabilize performance. But that also isn't a reason to avoid large objects. Some types of large objects can overall improve performance as they can be ways to more easily build cache locality or improve access patterns in cases like buffers and "memory pools", especially when long-lived in an application ("Gen 2 LOH"). As for intrinsics, they mostly apply to stack space, but with some Memory<T> and Span<T> options you can "borrow check" data from heap references. GCs obviously move things around (compaction to reduce fragmentation, collection to remove garbage and promote to the next generation) so the short-lived nature of stack space is interestingly important to their safety. Most of the intrinsics work with Vector<T>, a struct (stack space) which (among other options) can be constructed from any ReadOnlySpan<T> of the right size (and layout). That ReadOnlySpan<T> could be borrowed from an object somewhere in heap space (such as an intentional memory pool in Gen 2 LOH). No "copying" happens to build the Vector<T> because the ReadOnlySpan<T> is a type-safe/GC-safe pointer at that point and the Vector<T> can't last longer than the ReadOnlySpan<T>. Cache locality of that Vector<T> and subsequent ones in an operation become based on the data structure of your objects, but especially in the case of an intentional memory pool (an array or buffered series of arrays of some form) may be however you need it to be structured (arrays of value types like integers are designed to be contiguous memory and you can arrange things how you like in arrays, just like any other language; even if the GC affects where that array lives/moves, the data within it remains contiguous in the way you write it/read it, just about as you would expect).

doikor 3y ago

I really wish JDK had some API for hardware intrinsics. Hopefully we get something soon as the Vector API is coming (7th incubator version rolling on) https://openjdk.org/jeps/460 https://openjdk.org/jeps/460 Though the Vector API is pending on Valhalla to get far enough (it won't leave incubator until the necessary parts of Valhalla are in preview stage)

pjmlp 3y ago
Some alternative implementations like PTC and Aicas do, for the Real Time Java implementation.
Const-me 3y ago
Vector API is very far from being hardware intrinsics. Unfortunately for Java programmers, it’s merely a least common denominator of SIMD instructions across different ISAs. This makes the feature very limited by design, IMO borderline useless.
- doikor 3y ago
  Yeah while not great it is still a step in the right direction (giving control to the developer). Before this in the OpenJDK the developer had exactly 0 control. You just wrote code and hoped JDK would slap in vectorization etc.
- hajile 3y ago
  This depends on the implementation under the hood and I'd say that the ability to use SIMD even at less than peak efficiency is better than not being able to use it at all.
  - Const-me 3y ago
    For some problems, sticking to the SIMD subset exposed to Java might be slower than no SIMD at all. Example: https://news.ycombinator.com/item?id=36624240 https://news.ycombinator.com/item?id=36624240 The linked C++ code is easily portable to C# because hardware intrinsics, but not to Java because nothing similar to _mm256_sad_epu8 is available there.

neonsunset 3y ago

I wonder if .NET related submissions get score penalty on HN, the posts almost never manage to reach front-page or stay there for any meaningful period of time despite the score.

asp_hornet 3y ago
I don't think so, it’s just a polarising ecosystem. I worked in the .net for over 12 years and im so happy to be out. There’s nothing they could add to their language or tooling to entice me back.
- neonsunset 3y ago
  Which ecosystem are you switching to? (this better not be Go which is strictly worse)
- Guillaume86 3y ago
  And if you had to defend your position with arguments, what would the big ones be?

mappu 3y ago

> AVX-512 is a new feature set provided for x86 and x64 computers. Is it really possible to use AVX512 instructions from x86_32 mode?

colejohnson66 3y ago
Yes. You're just limited to eight vector registers.