4 ms·

Does anyone know if C#/dotnet is used in HFT to any extent? I'd imagine proper support for user-defined unboxed types (structs) and non-hacky methods for manual

Does anyone know if C#/dotnet is used in HFT to any extent? I'd imagine proper support for user-defined unboxed types (structs) and non-hacky methods for manual memory management (unsafe) could give it major advantage over Java with much of the QoL improvements java has over C++.

ed25519FUUU 6y ago
I don’t think Java was chosen because it had ways to do manual GC. I think the developers were comfortable with Java, so they wanted to find a way to fit it into something “fast”.
- dangerbird2 6y ago
  That was my assumption, considering how dominant Java is in finance. Also, as the article notes, Java has a pretty great runtime ecosystem, allowing devs to choose between hotspot, azul, graal, etc. depending on their runtime needs, while dotnet devs are more-or-less limited to Microsoft-controlled offerings of Mono, dotnet 4.x and dotnet core/5+.
  - muststopmyths 6y ago
    Here are my guesses (in addition to your points): - Until late in it's lifecycle, .net runtime performance was worse than JVM. - Also, the Windows TCP stack, re-architected with Vista, had severe latency issues, which probably led to wholesale abandonment of that platform for low-latency applications. Some of these problems were fixed in Windows 8, but it was probably too late by then. (These two points are based on my own experience with Windows/CLR/JVM in the mid/late 2000s and some anecdotes I heard second-hand. So I can't really cite any public sources for them) I'm sure the fact that you can customize Linux to the extreme and the lack of .Net on Linux in those days probably had a lot to do with the choice of Java, in addition to the above. However, based on my reading of blogs like Mechanical Sympathy [1] a while ago (~8-10 years), there does seem to have been some demand for .Net-based solutions in the HFT space. They had code for both Java and C# in their public code. [1]https://mechanical-sympathy.blogspot.com/ https://mechanical-sympathy.blogspot.com/
stzup7 6y ago
I've seen it on the client-side. HFT software, written in C or C++ is piloted by traders with C# / .NET apps via RPC
Grazester 6y ago
I had a conversation with someone who works at Credit Suisse in IT in some capacity. He said they experiment with C# but went back to C++ due to latency. This was about 12 years ago.
motives 6y ago
This is definitely an area of recent improvements, and I don't think modern .net (.net core 3.1 - .net 5) is comparable to older school .net here. Whilst GC isnt quite on JVM level yet (no serious custom GC's afaik), the .net team is clearly paying attention to performance(https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-5/#gc https://devblogs.microsoft.com/dotnet/performance-improvemen...). As you mention, increased ease of use for structs in the form of Span<T> is having quite significant impact in a lot of common .net libraries, aswell as the recent availability of hardware intrinsics for more performance sensitive tasks. I can certainly say F# has had a pretty decent impact in the finance sector in London and likely elsewhere (not sure about HFT however), mostly as a result of this attention to both performance and developer experience.
kamaitachi 6y ago
I work in a company that uses C# for HFT. However, we moved away from “pure C#” about 10 years ago, to use FPGAs, especially in the ultra low latency stuff. A lot of strategies can still be profitable with C#, colo’ed servers and kernel bypass NICs.
exhftdev 6y ago
I've worked for one of the big trading firms and they used C# including for their ultra-low-latency trading strategies. I'm a bit surprised by all the chatter about garbage collection because it's not as much an issue as it sounds. The thing is not to do allocations (i.e. GC) on the _critical path_. In a trading strategy you react to a stream of market data. It appears constant to the human eye, but in fact it's a series of "blips", sporadic events. When you get a new message you need to react as fast as possible. The fastest code is, of course, no code, so it doesn't really matter if it's not written in C++, C# or PHP. Here you're just going to do the very minimum required. You will find that generally there is no difference in the code compiled whether it' done by the C# JIT, the Java JIT or a C++ AOT compiler, because the code is simple: you read a few fields from the message and select a pre-computed outcome to decide whether or not to send a (buy|sell) order. Now, after that event has been handled (and perhaps you sent an order), you need to recompute stuff. You generally have plenty of time (a few milliseconds) before the next event arrives, and that's where you want to do all your fancy calculations, and here a high-productivity language like Java or C# helps, just because it's quicker to iterate and deploy modified trading strategies (typically deployed daily, or at least weekly). Many people assume that you need to do all your fancy calculations very rapidly right after a message arrives. But you don't have to because the message is not completely random. The order book is not going to change completely after one message. You can pre-compute a few likely scenarios, and when a message arrives and confirms one of those scenarios, you just mechanically go with the precomputed result.

ed25519FUUU 6y ago

I don’t think Java was chosen because it had ways to do manual GC. I think the developers were comfortable with Java, so they wanted to find a way to fit it into something “fast”.

dangerbird2 6y ago
That was my assumption, considering how dominant Java is in finance. Also, as the article notes, Java has a pretty great runtime ecosystem, allowing devs to choose between hotspot, azul, graal, etc. depending on their runtime needs, while dotnet devs are more-or-less limited to Microsoft-controlled offerings of Mono, dotnet 4.x and dotnet core/5+.
- muststopmyths 6y ago
  Here are my guesses (in addition to your points): - Until late in it's lifecycle, .net runtime performance was worse than JVM. - Also, the Windows TCP stack, re-architected with Vista, had severe latency issues, which probably led to wholesale abandonment of that platform for low-latency applications. Some of these problems were fixed in Windows 8, but it was probably too late by then. (These two points are based on my own experience with Windows/CLR/JVM in the mid/late 2000s and some anecdotes I heard second-hand. So I can't really cite any public sources for them) I'm sure the fact that you can customize Linux to the extreme and the lack of .Net on Linux in those days probably had a lot to do with the choice of Java, in addition to the above. However, based on my reading of blogs like Mechanical Sympathy [1] a while ago (~8-10 years), there does seem to have been some demand for .Net-based solutions in the HFT space. They had code for both Java and C# in their public code. [1]https://mechanical-sympathy.blogspot.com/ https://mechanical-sympathy.blogspot.com/

stzup7 6y ago

I've seen it on the client-side. HFT software, written in C or C++ is piloted by traders with C# / .NET apps via RPC

Grazester 6y ago

I had a conversation with someone who works at Credit Suisse in IT in some capacity. He said they experiment with C# but went back to C++ due to latency. This was about 12 years ago.

motives 6y ago

This is definitely an area of recent improvements, and I don't think modern .net (.net core 3.1 - .net 5) is comparable to older school .net here. Whilst GC isnt quite on JVM level yet (no serious custom GC's afaik), the .net team is clearly paying attention to performance(https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-5/#gc https://devblogs.microsoft.com/dotnet/performance-improvemen...). As you mention, increased ease of use for structs in the form of Span<T> is having quite significant impact in a lot of common .net libraries, aswell as the recent availability of hardware intrinsics for more performance sensitive tasks. I can certainly say F# has had a pretty decent impact in the finance sector in London and likely elsewhere (not sure about HFT however), mostly as a result of this attention to both performance and developer experience.

kamaitachi 6y ago

I work in a company that uses C# for HFT. However, we moved away from “pure C#” about 10 years ago, to use FPGAs, especially in the ultra low latency stuff. A lot of strategies can still be profitable with C#, colo’ed servers and kernel bypass NICs.

exhftdev 6y ago

I've worked for one of the big trading firms and they used C# including for their ultra-low-latency trading strategies. I'm a bit surprised by all the chatter about garbage collection because it's not as much an issue as it sounds. The thing is not to do allocations (i.e. GC) on the _critical path_. In a trading strategy you react to a stream of market data. It appears constant to the human eye, but in fact it's a series of "blips", sporadic events. When you get a new message you need to react as fast as possible. The fastest code is, of course, no code, so it doesn't really matter if it's not written in C++, C# or PHP. Here you're just going to do the very minimum required. You will find that generally there is no difference in the code compiled whether it' done by the C# JIT, the Java JIT or a C++ AOT compiler, because the code is simple: you read a few fields from the message and select a pre-computed outcome to decide whether or not to send a (buy|sell) order. Now, after that event has been handled (and perhaps you sent an order), you need to recompute stuff. You generally have plenty of time (a few milliseconds) before the next event arrives, and that's where you want to do all your fancy calculations, and here a high-productivity language like Java or C# helps, just because it's quicker to iterate and deploy modified trading strategies (typically deployed daily, or at least weekly). Many people assume that you need to do all your fancy calculations very rapidly right after a message arrives. But you don't have to because the message is not completely random. The order book is not going to change completely after one message. You can pre-compute a few likely scenarios, and when a message arrives and confirms one of those scenarios, you just mechanically go with the precomputed result.