9 ms·
as far as I've heard from HFT people Java's relatively popular in HFT what C# lacks here? is there huge gap between CLR and JVM?
by tester34 6y ago
as far as I've heard from HFT people Java's relatively popular in HFT
what C# lacks here?
is there huge gap between CLR and JVM?
- pjmlp 6y ago- Multiple implementations from various vendors - More support for UNIX and mainframe flavours - Given the multiple implementations, there are plenty of GC, JIT and tuning options to chose from - .NET Core is the Python 3 of .NET world, not everything from .NET Framework is available or portable to non-Windows platforms - Several JIT optimizations quite common in JVM world are only now coming to CLR, like dynamic rewrite of native code
- oaiey 6y agoJust that in difference to Python, everyone wants but not everyone can migrate to .NET Core. Also the source code is compatible.
- pjmlp 6y agoNot even Microsoft teams themselves have done it, plenty of MS stuff like SharePoint, Dynamics and SQL Server rely on .NET Framework. Source being compatible doesn't help when the libraries require code rewrites anyway.
- pionar 6y agoSQL Server itself does not rely on .NET Framework. SQL CLR, an optional component does. We run SQL Server in Linux containers now.
- moron4hire 6y agoAlso, .NET 5 is now here, and 6 is already in preview, which are much, much more backwards compatible than .NET Core 3. And unlike Python, we had .NET Standard to start migrating code between Framework and Core. A lot of extant code on NuGet has already migrated and there is really very little you can't do these days.
- pjmlp 6y agoUnless one is using third party controls, CMS like SiteCore, SharePoint and Dynamics, writing custom .NET code to be called by SQL Server stored procedures, having a working WCF infrastructure, using RDMS with EF 6 that isn't SQL Server, in-house stuff that isn't fully covered by the Windows Compatibility Pack, and a couple of other little things.
- hackerfromthefu 6y agoAlso asp.net webforms is not supported, webservice/soap are 'supported' but the tooling is broken and i don't think you could extend a system with it in modern VS. The winforms editor works but is buggier with each new version, and asp.net mvc projects are similar but changed, and I'm sure the list of issues continues. Also from my experience coaching and consulting there's a significant learning curve for .NET framework developers to work in the new core/.NET 5 systems.
- moron4hire 6y agoSo keep using .NET Framework. v4.8 is only 3 years old and there is no end date set on the support yet. Considering the last version of the v3 era will be supported until 2029, I'm sure the last version of the v4 era will get quite a long period of support as well. https://docs.microsoft.com/en-us/lifecycle/products/microsoft-net-framework https://docs.microsoft.com/en-us/lifecycle/products/microsof... I bit the bullet and transitioned from WebForms to ASP.NET Core. I think it's easier than WebForms ever was. You can replicate 90% of what WebForms did with Razor Pages, but you also get much easier support for building RESTful APIs. And with DI built-in, I don't have to worry about junior devs forgetting to close/dispose database connections ever again.
- hackerfromthefu 6y agoActually its not that easy, depending which parts of framework you used. It's certainly reasonable, but will take re-work for many applications beyond the project structure migration. I think in .NET 6/7/8 the .NET framework migration situation will improve to be mostly pure migration. For example I migrated about 75% of a moderately large winforms system directly, but 25% needed rewrites due to missing core implementations for framework libraries. e.g. MS chart component not implemented in core yet, soap services changing to grpc, and various others. And this system was originally built to minimise third party dependencies and use the core framework or microsoft provided libraries wherever possible.
- tasogare 6y agoPython 3 broke the language, .NET Framework and Core differ on the libraries, that a very different thing.
- pjmlp 6y agoWhen working code breaks in the new version it doesn't matter what broke, it still doesn't work.
- adamdusty 6y agoSure, but comparing to python is disingenuous. Python3 was a complete break from Python2. The first version of .net core (5ish? years ago) broke the standard library down to only libs that work cross platform. Is there anything in .NET 5 you're missing from .net 4.x?
- pjmlp 6y agoI already mentioned in a couple of replies, even Microsoft own products like SharePoint, Dynamics, SQL CLR aren't on Core.
- adamdusty 6y agoAnd on the other hand, Bing is running on the first preview version of .NET 6. MS is a big company, they aren't going to immediately migrate everything just like Google didn't migrate every single project to Go when they were heading it.
- Jamieee 6y agoDoes anyone know of any HFTs using C#?
- bob1029 6y agoI don't know of any, but I have been building prototypes in .NET Core 3+ that give me confidence in using this platform as a basis for an ultra-low-latency business system. The biggest issue would be avoiding heap alloc, but that is actually not too difficult if you manage your data well enough. Most transactions in this business can be defined in terms of structs that are <1kb in size. One other approach I have started to look at is sacrificing an entire high-priority thread to handling well-batched transactions (or extremely important timers) so that I never have to yield to the OS. In HFT, you always get an entire physical server to yourself, so eating 1 out of 32 threads is not a huge deal. In testing of this idea, I have found that I am able to reliably execute timers with accuracy of around 1/10th of a microsecond. Java probably has more optimizations around GC suppression, but I feel like avoiding allocation in the first place is the most important bit. I believe there are already some ways to trick .NET into not running GC.
- gher-shyu3i 6y agoOut of curiosity, what is the domain for the prototypes you're building?
- bob1029 6y agoFor now, I have been using this to test a toy/custom 2d client-server framework. All client events are piped to the server and processed in 1 gigantic ring buffer. This allows for ridiculous amounts of throughput due to batching effect. I am also using recurring, high-precision timers scheduled per client for purposes of triggering redraws and other important events as appropriate. This allows for complete decoupling of the event handling and client rendering pipelines. The breakdown is something like: 1 thread for ultra-low-latency timer execution 1 thread for processing the actual client event ring buffer, producing a consistent snapshot after each microbatch execution. 14+ threads for servicing HTTP requests (i.e. enqueuing client events), timers and redrawing client views using the near-real-time snapshots of business state. My thinking is that if I can build something in this domain that is satisfactory, I could consider bringing it to an HFT firm as well.
- gameswithgo 6y agoJava being big in HFT is an accident of history, not due to sound engineering choices.
- wealthyyy 6y agoAccenture used C# for a HFT shop but that shop went away from it to C++. Java is not popular in HFT other than 2-3 shops using it. C++ is best choice IMHO. It has become easy to use it and plus you can get compile time branch prediction with templates. RAII also amazing. RAII looks like a GC but it's not but it's works like a GC ..
- kaba0 6y agoWhat do you mean by compile time branch predictions?
- wealthyyy 6y agoYou pay cost for branch predictions at Runtime. You can avoid it at compile-time using Templates or even constexpr. .
- kaba0 6y agoI still don’t understand what you mean, please correct me where I’m wrong. Branch prediction is a CPU level thing, and mis-predictions has a cost. You can do things like partitioning data beforehand so that a given if condition will take the same branch each time in the partition, but I don’t see how does it apply to templates at all. You can’t avoid branches that depend on runtime infos, and those branches that depend on compile time known infos can be elided either automatically by the compiler if it can prove it is constant for example, or by things like constexpr, templates as you say. But it’s nothing too fancy, the dumb C-preprocessor macro can do similar things. JIT-compiled languages can sometimes elide branches based on runtime data, so there is that.
- usrusr 6y ago> what C# lacks here? Options. You are talking about the very tiny subset of Java users who actively worry about the relative differences in performance tweakability of different JVM implementations. Chances are that they routinely run multiple VMs and VM configuration sets in parallel on real world inputs to find the fastest configuration. "Look at this nice VM, trust us, it's very fast, rewrite all your code and see for yourselves!" just won't be very convincing to those. With that crowd, .NET isn't competing against the JVM, it's competing against against an entire market of competing JVMs.
- fnord123 6y ago> as far as I've heard from HFT people Java's relatively popular in HFT Java's popularity depends how HF in HFT is and the threshold for pain developers are willing to go through to develop Java in a way to avoid GC and reduce memory use (and hence reduce cache misses) Deploying to Linux boxen in colocation facilities is the norm and there is little/no culture of using dotNet on Linux. We all know it exists and it's open source now, but most Linux developers don't reach for dotNet.
- TeMPOraL 6y ago> but most Linux developers don't reach for dotNet On that note, how bad is .NET experience on Linux vs. on Windows? Besides the IDE (MonoDevelop is no Visual Studio). I was about to dig into this topic soon - I'm particularly interested in how useful PowerShell is on Linux these days. On Windows, I appreciate the deep .NET interop it offers; I think it's closest experience to Lisp Machines that you can find in mainstream computing. EDIT: Big thanks to the commenters who mentioned Rider, I didn't realize JetBrains had an IDE for .NET! That said, to clarify my comment above, I'm more interested in the experience with the platform itself - how does .NET Core work on Linux in terms of performance, fragility, access to first-party and third-party libraries? Can I expect non-UI .NET code to port well to Linux? And, in PowerShell, do I get to do things like Add-Type "<insert bunch of C# code here>", or $foo = [Some.dotNET.Type]::new()?
- tester34 6y agoI think people on Linux go for Rider/VS Code instead of MonoDevelop
- philliphaydon 6y agoI've been full time .NET Core on Ubuntu since Jan 2020. With the occasional booting a windows VM for a legacy project. JetBrains Rider is such a pleasure to work with.
- lol768 6y agoRider is great and surpasses Visual Studio in my opinion.