3 ms·

just out of curiosity -- for ad-tech real time bidding, what's your network latency and bandwidth like? I can buy that C/C++ is needed if you are colocated to a

just out of curiosity -- for ad-tech real time bidding, what's your network latency and bandwidth like? I can buy that C/C++ is needed if you are colocated to an exchange, but if you're bidding online, those few fractions of an ms you save in C/C++ vs node.js you could have also saved by locating your server closer to whatever ad-tech exchange you're bidding on.

synthc 6y ago
when I worked in adtech, we located the servers close to the exchange. This indeed made much more difference to the latency than code optimizations.
cmrdporcupine 6y ago
The issue I had back in this line of work was with garbage collection and was an issue in the 99th percentile. When you have an expectation from the exchange that you respond in under 80ms, and 25-50ms of that is eaten up by transport, you don't have a lot of time to mess around. So you spend the first chunk of time optimizing I/O and how you're accessing it. Then you start looking at computation -- improving caching, etc. At a certain point you start noticing in your graphs that you're on average doing quite well. But there's those hiccups every Nth request... And now you're in the game of fighting with your language's garbage collection algorithm...
- politician 6y ago
  At this level of scrutiny, replacing algorithms is appropriate and if the algorithms are built into the language then replacing the language may be one* way to clear the issue, but you have to do your homework to get there. *sync.Pool, arena allocation strategies
- user5994461 6y ago
  One clean and dirty way to help with that. Try to improve allocations where possible. There's often plenty of small allocations happening in Java that can be avoided (string usage is one major driver). Less allocations means less frequent garbage collections. Then one hack is to disable the garbage collection. Let the software run up to consume all the 100 GB of memory of the server and crash and restart. There is no pause of garbage collection when there is no garbage collection. If it's not enough, the last resort is to write native code or switch to C++.
joelbluminator 6y ago
Do u guys mean adtech or real time trading?
- cmrdporcupine 6y ago
  Realtime bidding on ad exchanges

synthc 6y ago

when I worked in adtech, we located the servers close to the exchange. This indeed made much more difference to the latency than code optimizations.

cmrdporcupine 6y ago

The issue I had back in this line of work was with garbage collection and was an issue in the 99th percentile. When you have an expectation from the exchange that you respond in under 80ms, and 25-50ms of that is eaten up by transport, you don't have a lot of time to mess around. So you spend the first chunk of time optimizing I/O and how you're accessing it. Then you start looking at computation -- improving caching, etc. At a certain point you start noticing in your graphs that you're on average doing quite well. But there's those hiccups every Nth request... And now you're in the game of fighting with your language's garbage collection algorithm...

politician 6y ago
At this level of scrutiny, replacing algorithms is appropriate and if the algorithms are built into the language then replacing the language may be one* way to clear the issue, but you have to do your homework to get there. *sync.Pool, arena allocation strategies
user5994461 6y ago
One clean and dirty way to help with that. Try to improve allocations where possible. There's often plenty of small allocations happening in Java that can be avoided (string usage is one major driver). Less allocations means less frequent garbage collections. Then one hack is to disable the garbage collection. Let the software run up to consume all the 100 GB of memory of the server and crash and restart. There is no pause of garbage collection when there is no garbage collection. If it's not enough, the last resort is to write native code or switch to C++.

joelbluminator 6y ago

Do u guys mean adtech or real time trading?

cmrdporcupine 6y ago
Realtime bidding on ad exchanges