7 ms·

As someone who has implemented a complex system in C++ in this decade, I’d say he’s not wrong, but you need to carefully weight the pros and cons. In our case

As someone who has implemented a complex system in C++ in this decade, I’d say he’s not wrong, but you need to carefully weight the pros and cons. In our case latency and real time demands mattered a lot (NASDAQ feed parser), to the point of the (potential) slowdown of a garbage collector kicking in was enough to rule out Java and .NET. It runs entirely in memory and on 64+ cores. We implemented our own reference counting system to keep some of our sanity, and to at least try to avoid the worst memory leaks. This was an edge case, and for almost everything else you’re probably better off implementing it in something that handles memory for you. If performance is an issue, least try it in Go or Rust with a quick PoC before jumping the C++ wagon.

choeger 7y ago
But aren't the edge cases the things that satisfy your pay? The standard cases of today are the introductory examples of tomorrow and will be automated or abstracted away by the end of next week. In JavaScript.
CoolGuySteve 7y ago
If you're parsing market data then you shouldn't really be allocating at all once the initial setup is complete. So there shouldn't be a need for ref counts. This is because the allocator itself can have an unbounded runtime that takes milliseconds, causing you to drop. In the past I've replaced malloc with an implementation that asserts if called on certain threads after init time.
- tom_ 7y ago
  Milliseconds??! What causes that to occur? Though I suppose if they can take that long, avoiding them is an even better idea than it is normally... For constant time allocations, try TLSF: http://www.gii.upv.es/tlsf/ http://www.gii.upv.es/tlsf/
  - cogman10 7y ago
    I'm not the op, but I'm guessing large amounts of fragmented memory is what causes that. It's one benefit of a gced language with compaction, allocations are typically bounded (except when they trigger a gc).
    - CoolGuySteve 7y ago
      Except compaction can also take many milliseconds and come from different threads. Writing a trading system in Java is harder than C++ imo because where before you had an allocation problem, now you have a multithreaded randomly stalling allocation problem. Virtu did it but everything I’ve heard about it nullifies the benefits of using java in the first place.
      - cogman10 7y ago
        Newer Java GCs are very low latency (microsecond). You trade performance and memory for that low latency though. AFAIK, they are still compacting. Still though, probably makes sense to do it in a lower level language. It's just far easier in C++ to decide that "Hey, you know what, I just want a big memory block that I control". I've even heard of game devs doing things like having per frame allocators. They get super fast allocation because they just pointer bump and at the end they simply reset the pointer back to location 0. I'm sure trading systems could do something similar.
        nwallin 7y ago
        The point is that a one millisecond pause is unacceptable. Low latency Java GCs have average latencies of one millisecond, 99th percentile latencies of 10 milliseconds, and 99.9th percentile latencies are neither measured nor optimized for. I don't consider it realistic to think that garbage collected languages might ever be usable in the context of game engines or HFT.
        cogman10 7y ago
        > game engines Game engines are already written in GCed languages. Java in particular. You may be right that a 10ms pause is unacceptable for HFT. However, for a FPS, 10ms is more than acceptable. It translates to 1 or 2 missed frames in the worst case. A bigger issue with using Java in particular for games is it's lack of value types. Writing high performance code for java is just that much harder because the language gets in the way.
  - ycombobreaker 7y ago
    Most likely it is simply invoking a syscall (sbrk or mmap), which invokes the scheduler and may yield the time slice. Feed parsing code may not issue any other syscalls (entirely userspace), turning the allocation into a bit of russian roulette.
davidcbc 7y ago
Memory management in modern C++ is considerably easier than it used to be. Bespoke memory managers aren't really needed, you can do almost anything you need to without ever using new or delete.
- stevenwoo 7y ago
  Out of curiosity, what is the field for which you find this to be true?
  - wffurr 7y ago
    If you are writing C++, any field! Just use std::shared_ptr and std::unique_ptr from the standard library, along with std::make_shared and std::make_unique.
    - berkut 7y ago
      std::shared_ptr uses CAS atomics, which in heavy multithreaded code (multiple threads operating on the same pointers), can have surprising overhead in some situations.
      - usrnm 7y ago
        On the other hand, not using atomics in heavy multithreaded code is just a recipe for disaster. The problem with shared_ptr is the fact that it uses atomics even in single-threaded code, which is obviously an overkill.
        jcelerier 7y ago
        > The problem with shared_ptr is the fact that it uses atomics even in single-threaded code, which is obviously an overkill. On the other hand imagine the security issues if shared_ptr was not thread-safe - you could just not reliably destroy a shared_ptr in any thread. If you really know that you're going to get a graph of shared_ptr that do not move of a single thread, you can use boost::local_shared_ptr explicitely.
pnako 7y ago
I've seen state-of-the-art low-latency trading software written in C# or Java. You should not be doing any memory allocation in the critical path anyway, so garbage collection is hardly a relevant factor.
- pkolaczk 7y ago
  Having written lot of Java, Scala and C++ (and recently some Rust), I must say it is much easier to avoid heap allocations in C++ and Rust than in GCed languages, thanks to explicit allocation on the stack and pass by value + move semantics.
  - chusk3 7y ago
    A big push in .net core 2.x and 3.x was what we call the Span-ification of the base class library and the runtime. This means there are many new APIs for dealing with slices of memory in a non-allocating manner, and this combined with memory pooling has contributed greatly to an overall performance boost to the runtime by reducing copying and GC time. These same APIs are available to the developer so I'd imagine that it would be simple to build a non allocating network buffer reader. I have built a non-allocating video renderer before using the new APIs, for example.
    - pkolaczk 7y ago
      There exist libraries for native memory pooling in Java as well, and we're using them. I'm not saying low allocation code can't be done in C# or Java. But these languages don't give some nice tools that are present in C++ and Rust - in particular RAII and automatic reference counting.
      - masklinn 7y ago
        > I'm not saying low allocation code can't be done in C# or Java. With modern tools and APIs it should even be possible to write alloc-free work loops in C# (though you're probably writing pretty alien C# at this point). AFAIK it'll remain impossible in Java until the "value types" effort bears fruits. A few months back a study of sort made the rounds implementing a network driver in multiple languages (Ixy), C# did extremely well in it (better throughput than Go and near competitive with C and Rust at higher batch size though it was way behind on latency) while Java was pretty much in the dumps. In one of the reaction threads (don't remember if it was on HN or Reddit) one of the people involved explained the discrepancy between Java and C# by not being able to go under ~20 bytes of allocation per packet forwarded in Java. There were other odd / interesting results from the effort e.g. Rust was slightly slower than C, in investigating that they found out Rust executed way more instructions (especially significantly more stores) but had significantly higher IPC and much higher cache hit rates. ixy also tried just converting the C code to Rust (using C2Rust) then compiling it and that turned out to be very slightly faster than the C code, which was funny: https://github.com/ixy-languages/ixy-languages/blob/master/Rust-vs-C-performance.md https://github.com/ixy-languages/ixy-languages/blob/master/R...

choeger 7y ago

But aren't the edge cases the things that satisfy your pay? The standard cases of today are the introductory examples of tomorrow and will be automated or abstracted away by the end of next week. In JavaScript.

CoolGuySteve 7y ago

If you're parsing market data then you shouldn't really be allocating at all once the initial setup is complete. So there shouldn't be a need for ref counts. This is because the allocator itself can have an unbounded runtime that takes milliseconds, causing you to drop. In the past I've replaced malloc with an implementation that asserts if called on certain threads after init time.

tom_ 7y ago
Milliseconds??! What causes that to occur? Though I suppose if they can take that long, avoiding them is an even better idea than it is normally... For constant time allocations, try TLSF: http://www.gii.upv.es/tlsf/ http://www.gii.upv.es/tlsf/
- cogman10 7y ago
  I'm not the op, but I'm guessing large amounts of fragmented memory is what causes that. It's one benefit of a gced language with compaction, allocations are typically bounded (except when they trigger a gc).
  - CoolGuySteve 7y ago
    Except compaction can also take many milliseconds and come from different threads. Writing a trading system in Java is harder than C++ imo because where before you had an allocation problem, now you have a multithreaded randomly stalling allocation problem. Virtu did it but everything I’ve heard about it nullifies the benefits of using java in the first place.
    - cogman10 7y ago
      Newer Java GCs are very low latency (microsecond). You trade performance and memory for that low latency though. AFAIK, they are still compacting. Still though, probably makes sense to do it in a lower level language. It's just far easier in C++ to decide that "Hey, you know what, I just want a big memory block that I control". I've even heard of game devs doing things like having per frame allocators. They get super fast allocation because they just pointer bump and at the end they simply reset the pointer back to location 0. I'm sure trading systems could do something similar.
      - nwallin 7y ago
        The point is that a one millisecond pause is unacceptable. Low latency Java GCs have average latencies of one millisecond, 99th percentile latencies of 10 milliseconds, and 99.9th percentile latencies are neither measured nor optimized for. I don't consider it realistic to think that garbage collected languages might ever be usable in the context of game engines or HFT.
        cogman10 7y ago
        > game engines Game engines are already written in GCed languages. Java in particular. You may be right that a 10ms pause is unacceptable for HFT. However, for a FPS, 10ms is more than acceptable. It translates to 1 or 2 missed frames in the worst case. A bigger issue with using Java in particular for games is it's lack of value types. Writing high performance code for java is just that much harder because the language gets in the way.
- ycombobreaker 7y ago
  Most likely it is simply invoking a syscall (sbrk or mmap), which invokes the scheduler and may yield the time slice. Feed parsing code may not issue any other syscalls (entirely userspace), turning the allocation into a bit of russian roulette.

davidcbc 7y ago

Memory management in modern C++ is considerably easier than it used to be. Bespoke memory managers aren't really needed, you can do almost anything you need to without ever using new or delete.

stevenwoo 7y ago
Out of curiosity, what is the field for which you find this to be true?
- wffurr 7y ago
  If you are writing C++, any field! Just use std::shared_ptr and std::unique_ptr from the standard library, along with std::make_shared and std::make_unique.
  - berkut 7y ago
    std::shared_ptr uses CAS atomics, which in heavy multithreaded code (multiple threads operating on the same pointers), can have surprising overhead in some situations.
    - usrnm 7y ago
      On the other hand, not using atomics in heavy multithreaded code is just a recipe for disaster. The problem with shared_ptr is the fact that it uses atomics even in single-threaded code, which is obviously an overkill.
      - jcelerier 7y ago
        > The problem with shared_ptr is the fact that it uses atomics even in single-threaded code, which is obviously an overkill. On the other hand imagine the security issues if shared_ptr was not thread-safe - you could just not reliably destroy a shared_ptr in any thread. If you really know that you're going to get a graph of shared_ptr that do not move of a single thread, you can use boost::local_shared_ptr explicitely.

pnako 7y ago

I've seen state-of-the-art low-latency trading software written in C# or Java. You should not be doing any memory allocation in the critical path anyway, so garbage collection is hardly a relevant factor.

pkolaczk 7y ago
Having written lot of Java, Scala and C++ (and recently some Rust), I must say it is much easier to avoid heap allocations in C++ and Rust than in GCed languages, thanks to explicit allocation on the stack and pass by value + move semantics.
- chusk3 7y ago
  A big push in .net core 2.x and 3.x was what we call the Span-ification of the base class library and the runtime. This means there are many new APIs for dealing with slices of memory in a non-allocating manner, and this combined with memory pooling has contributed greatly to an overall performance boost to the runtime by reducing copying and GC time. These same APIs are available to the developer so I'd imagine that it would be simple to build a non allocating network buffer reader. I have built a non-allocating video renderer before using the new APIs, for example.
  - pkolaczk 7y ago
    There exist libraries for native memory pooling in Java as well, and we're using them. I'm not saying low allocation code can't be done in C# or Java. But these languages don't give some nice tools that are present in C++ and Rust - in particular RAII and automatic reference counting.
    - masklinn 7y ago
      > I'm not saying low allocation code can't be done in C# or Java. With modern tools and APIs it should even be possible to write alloc-free work loops in C# (though you're probably writing pretty alien C# at this point). AFAIK it'll remain impossible in Java until the "value types" effort bears fruits. A few months back a study of sort made the rounds implementing a network driver in multiple languages (Ixy), C# did extremely well in it (better throughput than Go and near competitive with C and Rust at higher batch size though it was way behind on latency) while Java was pretty much in the dumps. In one of the reaction threads (don't remember if it was on HN or Reddit) one of the people involved explained the discrepancy between Java and C# by not being able to go under ~20 bytes of allocation per packet forwarded in Java. There were other odd / interesting results from the effort e.g. Rust was slightly slower than C, in investigating that they found out Rust executed way more instructions (especially significantly more stores) but had significantly higher IPC and much higher cache hit rates. ixy also tried just converting the C code to Rust (using C2Rust) then compiling it and that turned out to be very slightly faster than the C code, which was funny: https://github.com/ixy-languages/ixy-languages/blob/master/Rust-vs-C-performance.md https://github.com/ixy-languages/ixy-languages/blob/master/R...