6 ms·

A lot of ad-tech companies build ad-servers in C, because the latency is so crucial in that context.

geocar 11y ago
Specifically it's garbage collection. e.g. node.js has no problem getting 20k/sec per core, but a stall at the wrong time kills every pipelined HTTP request that follows (until you tear down the connection and restart it).
- deleted 11y ago
  [deleted]
- ex3ndr 11y ago
  20k/sec is a joke performance. Java/Scala with Akka handles millions of packages per sec and this is performance.
  - geocar 11y ago
    40k/sec http requests (no pipelining) per core is about as fast as it gets unless you move TCP into user space. Reports of "millions per sec" are usually talking about messages on established channels across all CPUs. If you're actually aware of a java based web server that can beat even 150k http requests measured by wrk or similar on local host I'd like to see it.
    - deleted 11y ago
      [deleted]
    - zzzcpan 11y ago
      > 40k/sec http requests (no pipelining) per core is about as fast as it gets unless you move TCP into user space. I think it can be micro-optimized beyond that. With predictions to avoid unnecessary syscalls, with syscalls grouped together to make cpu more efficient for the rest of the time it spends in event loop, and if it's possible to modify kernel a bit - with batching syscalls together to make them very cheap.
      - geocar 11y ago
        At some point, your complexity gets bigger than simply coding a state machine that operates directly on the network buffers themselves: That is to say, I suspect that if micro-optimisations can double our performance, they will be more complicated than just writing a customised ring0 that implements HTTP directly inside the network driver. Here is how I'm looking at it: • 10Gb/sec network port • 4k max requests and responses • == 1.3 million HTTP requests per second. Now the problem is that main memory is not much faster than our fastest network: About 15Gb/sec, so what we're talking about here is code and state staying entirely in L1, and streaming the network buffers across the CPU, and responding in one pass, to get that 1.3 million optimal performance. My dash server gets ~135k HTTP requests per second on localhost (I should be able to approach 300k/sec over a network if I ever get around to it). That's 22% of our optimal performance, and a lot better than any other HTTP server I'm aware of. At this speed, one of those micro-optimisations `writev()` is actually slower than `write()` -- likely because the code path is shorter in the simpler codebase -- but it illustrates my concern nicely: That we are close to that break-even point with the optimisations we can make. If we make our server bigger and more complicated, it might not make our programs any faster. That suggests to me that the solution is actually fewer, simpler syscalls, not more, bigger ones.
        vegardx 11y ago
        What kind of "main memory" are you talking about? Regular, consumer grade memory, will have a bandwidth at least ten times faster than your 10Gb/s network interface. Change the bit to a byte and you're a little closer.
- ktRolster 11y ago
  Worth also mentioning that ad servers tend to have massive RAM requirements (again, for speed). GC in a 30GB JVM can take 10 minutes. To handle it, companies mark the boxes as 'inoperable' when they are in GC mode and remove them from the cluster until thy are ready to return. All of this is motivation to rewrite everything in C.
  - paulddraper 11y ago
    10 minutes is nowhere even close to my experience, especially because Oracle/OpenJDK has incremental GC.
pjmlp 11y ago
Or they don't know any better. I took part in a few projects that replaced high throughput servers handling mobile network traffic from C++ to Java.
- faint_coder 11y ago
  Or maybe THEY don't know any better way to optimize/use C++ instead of switching to Java
  - pjmlp 11y ago
    Given that I remember the days when C and C++ compilers generated code worse than a junior Assembly programmer, I always find such comparisons interesting. Not that they aren't true, rather their validaty depends a lot of programmer skillset and compilers being used.
  - deleted 11y ago
    [deleted]
- goldenkey 11y ago
  Java is garbage unless you specific 20 -D options to bend it to your will.
  - pjmlp 11y ago
    I could say the same thing of compiler optimisation flags.
    - goldenkey 11y ago
      Pretty sure -O3 and we are set. False dichotomy. Java has a lot of users but sheep still get slaughtered.
- eva1984 11y ago
  Can confirm. I work at two ad-tech related companies, one large one medium, both use Java from the beginning. And I know other company using Python/Golang as well. Didn't know C(not C++) is particular popular until today. Note that ad company are pretty business focused, add or remove features for big clients are pretty common, so development efficiency matters a lot.

Specifically it's garbage collection. e.g. node.js has no problem getting 20k/sec per core, but a stall at the wrong time kills every pipelined HTTP request that follows (until you tear down the connection and restart it).

deleted 11y ago
[deleted]
ex3ndr 11y ago
20k/sec is a joke performance. Java/Scala with Akka handles millions of packages per sec and this is performance.
- geocar 11y ago
  40k/sec http requests (no pipelining) per core is about as fast as it gets unless you move TCP into user space. Reports of "millions per sec" are usually talking about messages on established channels across all CPUs. If you're actually aware of a java based web server that can beat even 150k http requests measured by wrk or similar on local host I'd like to see it.
  - deleted 11y ago
    [deleted]
  - zzzcpan 11y ago
    > 40k/sec http requests (no pipelining) per core is about as fast as it gets unless you move TCP into user space. I think it can be micro-optimized beyond that. With predictions to avoid unnecessary syscalls, with syscalls grouped together to make cpu more efficient for the rest of the time it spends in event loop, and if it's possible to modify kernel a bit - with batching syscalls together to make them very cheap.
    - geocar 11y ago
      At some point, your complexity gets bigger than simply coding a state machine that operates directly on the network buffers themselves: That is to say, I suspect that if micro-optimisations can double our performance, they will be more complicated than just writing a customised ring0 that implements HTTP directly inside the network driver. Here is how I'm looking at it: • 10Gb/sec network port • 4k max requests and responses • == 1.3 million HTTP requests per second. Now the problem is that main memory is not much faster than our fastest network: About 15Gb/sec, so what we're talking about here is code and state staying entirely in L1, and streaming the network buffers across the CPU, and responding in one pass, to get that 1.3 million optimal performance. My dash server gets ~135k HTTP requests per second on localhost (I should be able to approach 300k/sec over a network if I ever get around to it). That's 22% of our optimal performance, and a lot better than any other HTTP server I'm aware of. At this speed, one of those micro-optimisations `writev()` is actually slower than `write()` -- likely because the code path is shorter in the simpler codebase -- but it illustrates my concern nicely: That we are close to that break-even point with the optimisations we can make. If we make our server bigger and more complicated, it might not make our programs any faster. That suggests to me that the solution is actually fewer, simpler syscalls, not more, bigger ones.
      - vegardx 11y ago
        What kind of "main memory" are you talking about? Regular, consumer grade memory, will have a bandwidth at least ten times faster than your 10Gb/s network interface. Change the bit to a byte and you're a little closer.
ktRolster 11y ago
Worth also mentioning that ad servers tend to have massive RAM requirements (again, for speed). GC in a 30GB JVM can take 10 minutes. To handle it, companies mark the boxes as 'inoperable' when they are in GC mode and remove them from the cluster until thy are ready to return. All of this is motivation to rewrite everything in C.
- paulddraper 11y ago
  10 minutes is nowhere even close to my experience, especially because Oracle/OpenJDK has incremental GC.

pjmlp 11y ago

Or they don't know any better. I took part in a few projects that replaced high throughput servers handling mobile network traffic from C++ to Java.

faint_coder 11y ago
Or maybe THEY don't know any better way to optimize/use C++ instead of switching to Java
- pjmlp 11y ago
  Given that I remember the days when C and C++ compilers generated code worse than a junior Assembly programmer, I always find such comparisons interesting. Not that they aren't true, rather their validaty depends a lot of programmer skillset and compilers being used.
- deleted 11y ago
  [deleted]
goldenkey 11y ago
Java is garbage unless you specific 20 -D options to bend it to your will.
- pjmlp 11y ago
  I could say the same thing of compiler optimisation flags.
  - goldenkey 11y ago
    Pretty sure -O3 and we are set. False dichotomy. Java has a lot of users but sheep still get slaughtered.
eva1984 11y ago
Can confirm. I work at two ad-tech related companies, one large one medium, both use Java from the beginning. And I know other company using Python/Golang as well. Didn't know C(not C++) is particular popular until today. Note that ad company are pretty business focused, add or remove features for big clients are pretty common, so development efficiency matters a lot.