8 ms·

Performance Tuning for .NET Core

182 points by benaadams 8y ago

zamalek 8y ago
> Reduce branching & branch misprediction I wrote a parser for a "formalized" URI (it looked somewhat like OData). This parser was being invoked millions of times and was adding minutes to an operation - it dominated the profile at something like 30% CPU time. It started off something like this: int state = State_Start; for (var i = 0; i < str.Length; i++) { var c = str[i]; switch (state) { case State_Start: /* Handle c for this state. */ /* Update state if a new state is reached. */ } } Hardly rocket science, a clear-as-day miniature state machine. VTune was screaming about the switch, so I changed it to this: for (var i = 0; i < str.Length; i++) { for (; i < str.Length; i++) { var c = str[i]; /* Handle c for this state. */ /* Break if a new state is reached. */ } for (; i < str.Length; i++) { var c = str[i]; /* Handle c for this state. */ /* Break if a new state is reached. */ } } The new profile put the function at < 0.1% of CPU time. This is something that the "premature optimization crowd" (who tend to partially quote Knuth concerning optimization) get wrong: death by a thousand cuts. A single branch in the source (it ends up being more in machine code) was costing 30% performance.
- chrisseaton 8y ago
  > This is something that the "premature optimization crowd" (who tend to partially quote Knuth concerning optimization) get wrong But this wasn't premature optimisation. - you found performance was actually a problem in practice - you used a tool to profile the application for a real workload - you isolated something to optimise - you came up with a way to optimise based on the data and your tools - you tested that the optimisation worked That isn't premature optimisation. Premature optimisation would have been writing this in the first place without checking anything first.
  - deleted 8y ago
    [deleted]
- pierrebai 8y ago
  I failed to see how your anecdote relate to your ax-grindy assertion that premature optimization is bad. You wrote your code, you profiled it, you found one function taking 30% of the time, you optimized that function.
  - gameswithgo 8y ago
    We can always end up with a tautology interpretation of premature optimization, in which case it cannot be bad, if it was bad, it wasn't premature! While this may no include you, a great many forums and advise areas on the internet, and elsewhere, would discourage you from even worrying about a detail like this. Asking questions about this will just get you a knuth quote. I sometimes wonder if the reason so much software today is inexplicably slow is because for 30 years so many young kids curious about how computers and compilers work were told not to worry about it. I think it is useful to build up some intuition about these things, so your default implementation can be the fast one when it isn't especially messy to do so. People don't always get the opportunity to come back and change things later, and some design choices don't just affect a single function that you can easily fix later in the development processes.
    - int_19h 8y ago
      It's because most of the time, people asking questions like these don't actually have a perf issue. This is very evident on StackOverflow. So when you do have such an issue, and you want to make it clear that it's not a premature optimization, you should mention that it's a hot path as determined by profiling (or whatever other technique). Then you won't get the Knuth quote.
      - neetdeth 8y ago
        On the other hand, it’s highly annoying to have to anticipate and ward off a whole range of uncharitable and/or unimaginative assumptions about why one would want to do such a thing before there is even an attempt made at a straight answer. If nobody knows the answer, I’d rather have no replies than yet another soliloquy on why nobody actually needs to optimize anything.
jermaustin1 8y ago
So in other words extreme tuning = do the opposite of what you probably did!
GordonS 8y ago
> Mark classes as sealed by default Please, no! This shouldn't be the default - it's a constant bugbear of mine where I want to extend a class from a library, and I can't because it's been sealed for no good reason.
- wvenable 8y ago
  I had the same thought when I saw that. There seems to be a trend to seal and lock down classes preventing any kind of extension. It sort of misses one of the main benefits of OOP and I know what I'm doing.
  - scarface74 8y ago
    Instead of my rehashing the old arguments about inheritance.... http://developer-interview.com/p/oop-ood/what-are-advantages-of-composition-and-aggregation-over-inheritance-14 http://developer-interview.com/p/oop-ood/what-are-advantages...
    - wvenable 8y ago
      I understand the argument for composition-of-inheritance but they rarely apply when you actually need to do it. Often it's to reach in and fix a bug or enhance the behaviour of an existing component. You simply can't do that with composition. Without inheritance (if the class is sealed) I often end up having to re-write the entire component or simply accept my fate. So that is a lose-lose situation.
      - scarface74 8y ago
        That's where delegation comes in. You wrap each public method of Foo in another class call MyFoo and then fix the one method you care about. With C# and R# it's a simple matter of: Create a new class Create a private variable: private Foo _foo Click on _foo Resharper menu -> Generate Code -> create Delegating members.
        8y ago
gameswithgo 8y ago
One of the tips is to avoid Linq, which many .NET developers are hesitant to do. I made a library that lets you use Linq style convenience functions without a performance hit in many cases: https://github.com/jackmott/LinqFaster https://github.com/jackmott/LinqFaster
- GordonS 8y ago
  This looks really interesting - are the benchmarks in the GitHub repo against .NET Framework? With all the performance work that's been done in .NET Core, it might be interesting to compare it with .NET Core. Also, I noticed there haven't been any commits for a while - is this more or less considered 'complete'?
  - gameswithgo 8y ago
    Yes it is more or less considered complete. I may do a pass over it to see if it can leverage or support span<t> better sometime soon. Performance deltas should be similar on .NET Core.
- darrenkopp 8y ago
  Also worth mentioning: https://github.com/kevin-montrose/LinqAF https://github.com/kevin-montrose/LinqAF Focused on reducing the allocations which makes linq heavy
- WorldMaker 8y ago
  So your library speeds up Array<T>/T[] and List<T> Linq operations? That's quite interesting, though usually in my experiences in Linq performance optimization the problem is usually prematurely using the wrong data type for your Linq operation. Given the immediate next bullet point in the article is about foreach operations over List<T>, that would be my top expectation why they were seeing so much allocation in Linq. ToList() allocates a lot, and to many projects I've seen reify everything with ToList() way too often in Linq usage. I've argued before that List<T> is very rarely the appropriate data structure for a lot of Linq work, and personally consider ToList() harmful. A successful strategy I've used to cleaning up Linq performance in projects is to simply start by remove all ToList() calls entirely and work to move API signatures to use smarter, more appropriate data types than List<T> and IList<T> everywhere.
rhinoceraptor 8y ago
It would be nice if .NET Core profiling was a bit easier on Linux, Microsoft has a shell script[1] to do profiling but it requires Windows only tools. They don't ship Crossgen with the Linux packages, and you have to manually generate the .NET runtime symbols. I've gotten things like FlameGraphs working using BCC profile[2], but it took quite a bit of work. [1]: https://raw.githubusercontent.com/dotnet/corefx-tools/master/src/performance/perfcollect/perfcollect https://raw.githubusercontent.com/dotnet/corefx-tools/master... [2]: https://github.com/iovisor/bcc/blob/master/tools/profile.py https://github.com/iovisor/bcc/blob/master/tools/profile.py
- tluyben2 8y ago
  The perf script from MS is what we use to profile and fix issues on linux. I do not have Windows; no issues so far with just Linux. We managed to diagnose and fix every perf issue so far. Not sure what you mean by windows only tools or manually generate symbols?
  - rhinoceraptor 8y ago
    I haven't tried the script, the documentation says you need PerfView to view the data so I didn't bother running it. For generating symbols, (I misspoke a bit, I mean downloading them) I'm talking about the native CLR runtime symbols. According to the docs if you want those, you need to use dotnet-symbol and manually download the symbols for the CLR alongside the CLR .so files.
    - NicoJuicy 8y ago
      Well, .net was for a long time only available on windows (since 2002 and development in late 1990's). I think making the tools cross-platform after such a long time is not that easy/fast.
    - tluyben2 8y ago
      I view in the Linux cli and I can drill down, sort etc everything there and see exactly what the issues are. No Windows or PerfView needed.
blinkingled 8y ago
> JIT won't inline functions that throw Seriously? Never had to worry about that in Java land. What would be the reason for this?
- nlawalker 8y ago
  Stack trace accuracy, maybe?
  - fgonzag 8y ago
    Shouldn't matter, since you're throwing at that particular location of the executing function, so the runtime still has to have a way of knowing that the code was part of an inline function along with its name.
- benaadams 8y ago
  Size generally; you can aggressive inline it, but you'll likely be pulling in some chunky stuff for the throw. Push the throw out of line and your inline will be more trim.
kevingadd 8y ago
A tip related to the throw inlining tip: One way to get more consistent/effective inlining is to split the complex 'slow paths' out of your functions into helper functions. For example, let's say you have a cached operation with cache hit and cache miss paths: void GetValue (string key, out SomeBigType result) { if (_cache.TryGetValue(key, out result)) return; result = new SomeBigType(key, ...); _cache[key] = result; } In most scenarios this function might not get inlined, because the cache miss path makes the function bigger. If you use the aggressive inlining attribute you might be able to convince the JIT to inline it, but once the function gets bigger it doesn't inline anymore. However, if you pull the cache miss out: void GetValue (string key, out SomeBigType result) { if (_cache.TryGetValue(key, out result)) return; GetValue_Slow(key, out result); } void GetValue_Slow (string key, out SomeBigType result) { result = new SomeBigType(key, ...); _cache[key] = result; } You will find that in most cases, GetValue is inlined and only GetValue_Slow produces a function call. This is especially true in release builds and you can observe it in the built-in Visual Studio profiler or by looking at method disassembly. (Keep in mind that many debuggers - including VS's - will disable JIT optimization if you start an application under the debugger or attach to it. You can disable this.) This tip applies to both desktop .NET Framework and .NET Core, in my testing (netcore is generally better at inlining, though!) If you're writing any performance-sensitive paths in a library I highly recommend doing this. It can make the code easier to read in some cases anyway.

182 points by benaadams 8y ago

> Reduce branching & branch misprediction I wrote a parser for a "formalized" URI (it looked somewhat like OData). This parser was being invoked millions of times and was adding minutes to an operation - it dominated the profile at something like 30% CPU time. It started off something like this: int state = State_Start; for (var i = 0; i < str.Length; i++) { var c = str[i]; switch (state) { case State_Start: /* Handle c for this state. */ /* Update state if a new state is reached. */ } } Hardly rocket science, a clear-as-day miniature state machine. VTune was screaming about the switch, so I changed it to this: for (var i = 0; i < str.Length; i++) { for (; i < str.Length; i++) { var c = str[i]; /* Handle c for this state. */ /* Break if a new state is reached. */ } for (; i < str.Length; i++) { var c = str[i]; /* Handle c for this state. */ /* Break if a new state is reached. */ } } The new profile put the function at < 0.1% of CPU time. This is something that the "premature optimization crowd" (who tend to partially quote Knuth concerning optimization) get wrong: death by a thousand cuts. A single branch in the source (it ends up being more in machine code) was costing 30% performance.

chrisseaton 8y ago
> This is something that the "premature optimization crowd" (who tend to partially quote Knuth concerning optimization) get wrong But this wasn't premature optimisation. - you found performance was actually a problem in practice - you used a tool to profile the application for a real workload - you isolated something to optimise - you came up with a way to optimise based on the data and your tools - you tested that the optimisation worked That isn't premature optimisation. Premature optimisation would have been writing this in the first place without checking anything first.
- deleted 8y ago
  [deleted]
pierrebai 8y ago
I failed to see how your anecdote relate to your ax-grindy assertion that premature optimization is bad. You wrote your code, you profiled it, you found one function taking 30% of the time, you optimized that function.
- gameswithgo 8y ago
  We can always end up with a tautology interpretation of premature optimization, in which case it cannot be bad, if it was bad, it wasn't premature! While this may no include you, a great many forums and advise areas on the internet, and elsewhere, would discourage you from even worrying about a detail like this. Asking questions about this will just get you a knuth quote. I sometimes wonder if the reason so much software today is inexplicably slow is because for 30 years so many young kids curious about how computers and compilers work were told not to worry about it. I think it is useful to build up some intuition about these things, so your default implementation can be the fast one when it isn't especially messy to do so. People don't always get the opportunity to come back and change things later, and some design choices don't just affect a single function that you can easily fix later in the development processes.
  - int_19h 8y ago
    It's because most of the time, people asking questions like these don't actually have a perf issue. This is very evident on StackOverflow. So when you do have such an issue, and you want to make it clear that it's not a premature optimization, you should mention that it's a hot path as determined by profiling (or whatever other technique). Then you won't get the Knuth quote.
    - neetdeth 8y ago
      On the other hand, it’s highly annoying to have to anticipate and ward off a whole range of uncharitable and/or unimaginative assumptions about why one would want to do such a thing before there is even an attempt made at a straight answer. If nobody knows the answer, I’d rather have no replies than yet another soliloquy on why nobody actually needs to optimize anything.

jermaustin1 8y ago

So in other words extreme tuning = do the opposite of what you probably did!

GordonS 8y ago

> Mark classes as sealed by default Please, no! This shouldn't be the default - it's a constant bugbear of mine where I want to extend a class from a library, and I can't because it's been sealed for no good reason.

wvenable 8y ago
I had the same thought when I saw that. There seems to be a trend to seal and lock down classes preventing any kind of extension. It sort of misses one of the main benefits of OOP and I know what I'm doing.
- scarface74 8y ago
  Instead of my rehashing the old arguments about inheritance.... http://developer-interview.com/p/oop-ood/what-are-advantages-of-composition-and-aggregation-over-inheritance-14 http://developer-interview.com/p/oop-ood/what-are-advantages...
  - wvenable 8y ago
    I understand the argument for composition-of-inheritance but they rarely apply when you actually need to do it. Often it's to reach in and fix a bug or enhance the behaviour of an existing component. You simply can't do that with composition. Without inheritance (if the class is sealed) I often end up having to re-write the entire component or simply accept my fate. So that is a lose-lose situation.
    - scarface74 8y ago
      That's where delegation comes in. You wrap each public method of Foo in another class call MyFoo and then fix the one method you care about. With C# and R# it's a simple matter of: Create a new class Create a private variable: private Foo _foo Click on _foo Resharper menu -> Generate Code -> create Delegating members.
      - 8y ago

gameswithgo 8y ago

One of the tips is to avoid Linq, which many .NET developers are hesitant to do. I made a library that lets you use Linq style convenience functions without a performance hit in many cases: https://github.com/jackmott/LinqFaster https://github.com/jackmott/LinqFaster

GordonS 8y ago
This looks really interesting - are the benchmarks in the GitHub repo against .NET Framework? With all the performance work that's been done in .NET Core, it might be interesting to compare it with .NET Core. Also, I noticed there haven't been any commits for a while - is this more or less considered 'complete'?
- gameswithgo 8y ago
  Yes it is more or less considered complete. I may do a pass over it to see if it can leverage or support span<t> better sometime soon. Performance deltas should be similar on .NET Core.
darrenkopp 8y ago
Also worth mentioning: https://github.com/kevin-montrose/LinqAF https://github.com/kevin-montrose/LinqAF Focused on reducing the allocations which makes linq heavy
WorldMaker 8y ago
So your library speeds up Array<T>/T[] and List<T> Linq operations? That's quite interesting, though usually in my experiences in Linq performance optimization the problem is usually prematurely using the wrong data type for your Linq operation. Given the immediate next bullet point in the article is about foreach operations over List<T>, that would be my top expectation why they were seeing so much allocation in Linq. ToList() allocates a lot, and to many projects I've seen reify everything with ToList() way too often in Linq usage. I've argued before that List<T> is very rarely the appropriate data structure for a lot of Linq work, and personally consider ToList() harmful. A successful strategy I've used to cleaning up Linq performance in projects is to simply start by remove all ToList() calls entirely and work to move API signatures to use smarter, more appropriate data types than List<T> and IList<T> everywhere.

rhinoceraptor 8y ago

It would be nice if .NET Core profiling was a bit easier on Linux, Microsoft has a shell script[1] to do profiling but it requires Windows only tools. They don't ship Crossgen with the Linux packages, and you have to manually generate the .NET runtime symbols. I've gotten things like FlameGraphs working using BCC profile[2], but it took quite a bit of work. [1]: https://raw.githubusercontent.com/dotnet/corefx-tools/master/src/performance/perfcollect/perfcollect https://raw.githubusercontent.com/dotnet/corefx-tools/master... [2]: https://github.com/iovisor/bcc/blob/master/tools/profile.py https://github.com/iovisor/bcc/blob/master/tools/profile.py

tluyben2 8y ago
The perf script from MS is what we use to profile and fix issues on linux. I do not have Windows; no issues so far with just Linux. We managed to diagnose and fix every perf issue so far. Not sure what you mean by windows only tools or manually generate symbols?
- rhinoceraptor 8y ago
  I haven't tried the script, the documentation says you need PerfView to view the data so I didn't bother running it. For generating symbols, (I misspoke a bit, I mean downloading them) I'm talking about the native CLR runtime symbols. According to the docs if you want those, you need to use dotnet-symbol and manually download the symbols for the CLR alongside the CLR .so files.
  - NicoJuicy 8y ago
    Well, .net was for a long time only available on windows (since 2002 and development in late 1990's). I think making the tools cross-platform after such a long time is not that easy/fast.
  - tluyben2 8y ago
    I view in the Linux cli and I can drill down, sort etc everything there and see exactly what the issues are. No Windows or PerfView needed.

blinkingled 8y ago

> JIT won't inline functions that throw Seriously? Never had to worry about that in Java land. What would be the reason for this?

nlawalker 8y ago
Stack trace accuracy, maybe?
- fgonzag 8y ago
  Shouldn't matter, since you're throwing at that particular location of the executing function, so the runtime still has to have a way of knowing that the code was part of an inline function along with its name.
benaadams 8y ago
Size generally; you can aggressive inline it, but you'll likely be pulling in some chunky stuff for the throw. Push the throw out of line and your inline will be more trim.

kevingadd 8y ago

A tip related to the throw inlining tip: One way to get more consistent/effective inlining is to split the complex 'slow paths' out of your functions into helper functions. For example, let's say you have a cached operation with cache hit and cache miss paths: void GetValue (string key, out SomeBigType result) { if (_cache.TryGetValue(key, out result)) return; result = new SomeBigType(key, ...); _cache[key] = result; } In most scenarios this function might not get inlined, because the cache miss path makes the function bigger. If you use the aggressive inlining attribute you might be able to convince the JIT to inline it, but once the function gets bigger it doesn't inline anymore. However, if you pull the cache miss out: void GetValue (string key, out SomeBigType result) { if (_cache.TryGetValue(key, out result)) return; GetValue_Slow(key, out result); } void GetValue_Slow (string key, out SomeBigType result) { result = new SomeBigType(key, ...); _cache[key] = result; } You will find that in most cases, GetValue is inlined and only GetValue_Slow produces a function call. This is especially true in release builds and you can observe it in the built-in Visual Studio profiler or by looking at method disassembly. (Keep in mind that many debuggers - including VS's - will disable JIT optimization if you start an application under the debugger or attach to it. You can disable this.) This tip applies to both desktop .NET Framework and .NET Core, in my testing (netcore is generally better at inlining, though!) If you're writing any performance-sensitive paths in a library I highly recommend doing this. It can make the code easier to read in some cases anyway.